1. Most speakers charge over USB
A portable speaker has a lithium-ion battery inside and charges from a USB socket. There is no cell to buy, and a replacement is a repair rather than a swap.
2. Some take AA cells
A smaller speaker, or a party speaker, may take AA cells instead. Those are usually disposable alkaline, and the speaker plays until they are flat.
3. A speaker that only works when plugged in
That means the battery is worn out. A rechargeable pack usually lasts two to four years before it holds too little charge to be worth having.
4. Do not keep it full for months
A lithium cell left sitting at a full charge for a long time ages faster than one kept at about half. Top it up every couple of months if you use the speaker only now and then.
5. Charge it dry
A speaker that has been in the rain or the pool should be dry before you plug it in. Let the port dry out fully, and never charge a wet speaker.