From a physics perspective cosmic rays are different from X-rays in a couple of ways.
Firstly they are more energetic. Although both are electromagnetic X rays are lower frequency and slightly less energetic. Cosmic rays also collide in the upper atmosphere and produce secondary cosmic rays and various other subatomic particles such as muons.
When talking about radiation to laymen doctors and scientists look for a way to try to explain that radiation carries a risk but it's very small. They often latch on to X-rays because they are a form of radiation that everybody is familliar with.
It's an analogy and like most analogies it gets the fundamental point across but when you start asking difficult questions it doesn't work as well as it might.
Radiation dosages are measured in milli or micro sieverts.
It's a measure that tries to catagorise the effects of radiation on the body.
1 Sievert (Sv) might cause nausea, 3 SV would kill 50% of people in 30 days.
an X-ray is 0.03 to 0.1 miliSieverts or 30,000 times less than that lethal dose.
Flying at altitude is calculated at 0.0001 milisieverts per hour
http://www.hps.org/publicinformation/ate/faqs/ commercialflights.html
(penultimate question)
So actually it's a lot less