TTT, you twice above refer to "the BBC", as if that organisation alone had suggested the debates. In fact, these were mooted by OFCOM, the organisation which represents all the communications media, not just the one you yourself dislike. Accordingly, there IS a challenge and Cameron seems intent on dodging it.
You ask, "Why would he go on?" Well, at the 2010 election, Gordon Brown certainly participated in the debates as PM and I'm sure you were one of the people who were only too keen to label him as "Bottler" a short time before that regarding opting out of a general election.
It seems Cameron is far more of a bottler than Brown ever was!
At the 2010 election, Gordon Brown