Hold your finger on the app you want to delete. The icons will start to shake and an x will show on each of them.
Touch the x to delete the app, then press the home key to return to normal. You will also need to delete/disable the app in itunes to stop it reloading when you sync.