You can use the iPod as a Portable HDD (Hard disk drive)
This means it can store files, and folders, just like your computer.
In iTunes, set your iPod into Disk Mode, an icon for the iPod will appear on your desktop (Mac) or in your My Computer folder (Windows). Drag everything you wish to copy (in this case, your My Music folder if using Windows, or your iTunes folder on a Mac) onto the iPod.
Disconnect the iPod, connect it to the new computer, set it into Disk Mode again, drag My Music off the iPod. Set it OFF Disk Mode, and voila, all your music is on the new computer.
Why spend money on tiny USB memory sticks when your iPod's memory is like 100x larger!