The problem TV producers don't seem to grasp is that the appeal of the individual is bound up in the appeal of the show.
So Len Goodman appeals on Strictly, but that does not mean that Len Goodman appeals without the Strictly format around him. They may well find that his imagined 'appeal' will evaporate outside that comfort zone.
I strongly suspect Noel Fielding will find the same on Bake Off - he may appeal to his niche comedy audience, but I seriously doubt he will cut it on a mainstream show with such an accepted style as Bake Off.