If you had a filing cabinet you would want to keep certain files in each folder so that you can easily find them. All the folders will fill up your filing cabinet but in those folders there is lots of room for more files.
On a hard drive, when these allocated spaces are full, the additional data will be dumped anywhere there is room. Your computer will eventually get slower and slower because much of the CPU's processing is kept busy looking for files in unfamiliar territory.
Defragmenting will re-organise the hard drive so that Windows can once again access the data rapidly. You might have noticed, If you have a lot of stuff of on a HDD the allocations are different as , say, when you first got the computer. In the beginning of defragging all the data on teh disk is scanned to estimate how and where Windows would like the allocations, that's why the old ('98) defragging used to hang so long on 10%. Fortunately, XP is more organised and defrags a lot faster.