A cheap Dell will likely fit the bill. Apple iMac is an even better option in terms of simplicity, but may be too expensive for them, for their first computer if they just want basic tasks.
I'd also throw in the idea of using Ubuntu instead of Windows. It's totally free, more secure, and comes with a Word processor, spreadsheet, music player, web browser, email, everything.
It may be a better idea than Windows because they won't have constant nagging of Windows updates etc. to contend with, and it would run more smoothly on a lower-spec computer than Vista or even XP will.
The only problem with this idea is that their friends and family who can show them how to use it likely will be used to only Windows themselves.