If the college or university takes public money, ( grants, etc ) it's public sector. Most colleges are, but a few ( such as crammers ) are not. Some colleges are entirely private - such as colleges belonging to large enterprises where they train their own staff.
'Public sector' is normally taken to mean councils, NHS, government (both local and national).
Schools /University/ college is more often described as ' the Education sector'.
No hard and fast rule though.
What is someone works in funding department of a college and someone works in clinical trials at a medical campus and a University - would this person be classed as working in public sector ?