Because religion is everywhere, in everything. it cuts a very deep groove through our society, it is ingrained in our psyche, to the extent that we often dont even realise it... it just 'is' ... and thats wrong.
its in our marriages, births and deaths, schools, libraries, things are named after it, it is even how we swear, its deep in our language, our entertainment, our songs, its in our laws, in politics. its deep within our behaviour, how we've been brought up, guilt and shame and sin etc - the list goes on and on...
it is just 'the way it is' ... and many people dont like that - thats why they seek to oppose it, and ideally get rid of it.