If you are not wireless then what will be happening is most likely that you are using your broadband at the sametime your neighbourghs are using their own broadband along with potentially several others in your street perhaps. That's why Broadband providers only offer you 'up to' 4Mb, 8Mb, 16Mb etc. Depending on how many are sharing then this is speed is not always possible.
If you are wireless then you want to go into your settings and at the very least add a 5-digit WEP code. This is the most basic form of internet setting for the wirelless routers.
There will probably also be a setting that will actually show who is all connected to that router so if your neighbour for instance have named their PC with something you might recognise and have been using your wireless connection then that name would show up.
Hope this helps