The central idea is that you can't see several apps at once (it's a small screen, what's the point?).
However, most apps remember where you were when you exited it. So for example, I can read an email, and see a link to a website. If I touch that link, I'll get taken to the Safari browser, and have that link load. Once I'm done with that, I can press the home button, then touch the Mail icon again, and I'm immediately shown the email that I was just checking. I can now delete or move that email or whatever, and move onto the next email.
The 4 apps that are by default at the bottom of the screen (that is, Phone, Mail, Safari, iPod) are all background-capable apps, meaning that when you close them, they can continue to run in the background. Mail can be set to check for new email every 30mins, for example. iPod can keep playing your music through headphones or the internal speaker while you check your email or browse the web in Safari.
All the other apps don't actually run in the background. When you exit them by pressing the home button (to go back to the home screen), they close. But they do usually save their state, as I said above. This is simply to save battery life --- you don't want people accidentally leaving all kinds of apps open (especially non-Apple ones that may not have been as well programmed), because they'll eat into your battery life.