Somewhere in the product description there will be a battery type if it is AA or AAA then you will need these types of battery. If it says internal lithium then that usually means it has a non removeable rechargeable battery.
An important consideration is how long the mp3 lasts between charges or battery changes as you don't want to be continually connecting it to the pc or charger. Cheaper mp3s tend to have poor battery life compared to a better product such as Sonys. Many of their players last up to 30 hours on a single charge.
This player has internal battteries
http://www.microdirect.co.uk/(15898)SumVision- Platinum-Pro-2GB-MP4-Player-Supports.aspx
This one uses AAA type battery
http://www.microdirect.co.uk/(14333)Sumvision- MP3-Player-Card-Reader-with-LCD.aspx
This Sony is said to last 30 hours on a single charge, read the third review down mentions the long battery life
http://www.amazon.co.uk/Sony-NWZA815B-2GB-vide o-walkman/dp/B000VD565A