The registry is basically an index of where everything is on your computer, together with what links everything together. So, for example, it includes a record of which drivers to call upon when you use devices such as printers and scanners. It also knows which application to launch when you double-click on a file and where to find the font files to display documents and web pages.
However it doesn't include actual programs (other than some Windows default programs) or data such as documents, pictures, videos, etc. So it's a relatively small (but essential) file which can easily fit onto a USB memory stick.
I'm happy enough to just keep back-ups of data such as pictures and documents, together with things like my browser bookmarks. If my computer crashes completely I can reload all of my software (such as word-processing and image manipulation software) onto a new computer and still have all of the really important stuff available to me again. (OK, it takes about a day to get a new computer up and running with all of my software reinstalled, and things like printer drivers in place, but I can live with that).
Many people are like me and only back up the really critical stuff (such as irreplaceable photographs or key business documents). However if you really want a complete copy of everything on your computer (so that you can simply set up a new computer exactly as if it was your old one) then you need to clone the disc (as per my third link above).