Your boss may be threatened by your current liklihood of progressing up the ranks and their only line of defence is to ensure you gain the complainer/ troublemaker tag by tricking you into launching formal complaint procedures over an issue which is sufficiently annoying to motivate you to do so yet, ultimately, something trivial.
Things on the same level as race discrimination, sexual harrassment and so on need proper resolution and will not mark you down as one who complains unjustifiably.
If your work efficiency has been harmed (the book was reference material you need in order to perform your functions, say) then I think that justifies a complaint. *
If your work efficiency is unchanged then your case would be weaker, metely one of intrusion on "personal space" and, frankly, the company would regard it as their property, run under their rules. Get these clarified before you start anything which might show you up as ignorant of these rules.
If your work efficiency is improved, because the book was completely unrelated to work and other rearrangements also helped then, obviously, do not go ahead with the complaint: the boss will be getting 'brownie points' out of the situation.
* Showing up your boss, particularly a bad case of being so under-worked that they have time to tidy a desk - a task which could easily be ordered (delegated) later - will, naturally, have permanent and mildly corrosive effects on the team dynamic. If this harms the company's bottom line, ultimately it harms the prospects for you and your colleagues.
I am over-thinking, here, so you don't have to. Simply strike a healthy balance between being a doormat and asserting yourself but not so much so that you damage your employer's profit margin.