I wouldn't go near the PSP if your not going to play games on it for 3 reaons,
1. It's massive when compared to an MP3 player, I wouldn't want to be lugging that around.
2. It has little or no on board storage, you need memory cards to store the music on which are not cheap. �68 for 1Gb for example
3. It's expensive when you take in to account the memory requirements. Say for example the Apple iPod Nano which has 4Gb of memory at �174, the same spec PSP would be �451, it gets even worse when you compare to a iPod Video which has �30Gb at �210 compared with the same spec PSP at �2219.
Also unless you have a wireless network you won't be able to use the web browser, and if you do then personally I'd prefer to look at the internet on my PC rather than a small(ish) screen.
I'm not slagging off the PSP but it's pointless buying one as a music player only.