When using a client app like Outlook Express to check your email, there are 3 main acronyms and abbreviations that you'll see:
POP: move email from server to your machine
IMAP: synchronise email on server and your machine
SMTP: send email
POP is older and not as good. Most offer IMAP too now. Basically, if you want to check your email you need to use either POP or IMAP. POP moves the mail to your computer, meaning that it'll disappear from the server eventually. IMAP keeps it on the server too (for instance, Gmail's servers), just making sure that the email on your computer matches that on the server.
IMAP is much better for technical reasons, but to you it means that you can check your email on multiple devices and they'll all stay in sync with each other.
And you'll always be told to tell Outlook Express what SMTP server you want to use -- this is the server that sends your mail. Gmail has their own, your ISP will have their own, etc.
The Gmail interface is great, but the downside is that you don't actually store email on your own computer. If something goes wrong with Gmail you lose everything (and in the terms and conditions, you'll find that it won't be Google's fault).
So, what I do is use IMAP too. In the Gmail settings you can enable IMAP for your account. There are good help files there too, telling you step by step how to set up Outlook Express.
Once done, you can check your email with either the web interface or with Outlook Express. Using this method though, your mail will also be in Outlook Express too, stored on your computer. So if Gmail goes belly-up, you'll still be ok.
(Postscript: I'd recommend you use Thunderbird over Outlook. It's free, and is safer and better. It also stores your email in a better format that's easier to recover if something goes wrong.)