The rocket would take off and burst with stars, dependant on the stars loaded into the rocket.
Some firework effects have a positive oxygen balance. In other words, they rely almost entirely on the oxygen in the oxidiser present in the mixture.
Some have a negative oxygen balance and rely on the presence of atmospheric oxygen for their effect.
For example the willow effects that just look like tendrils of sparks hanging in the air, are fine particles of charcoal or aluminium that rely on atmospheric oxygen to continue glowing after the initial burst.
Plain coloured stars such as reds and greens would be likely to burn as on earth, with the addition of a bit more oxidiser in the mixture..