The housing association are correct. A calendar year normally consists of 52 weeks and 1 day, with an extra day added on in a leap year. However a 'tax year' is always 52 weeks long (which is actually 1 day short of a calendar year, or 2 days short in a leap year) until such time as all of those missing days add up to a full week. Then, to get the system synchronised again, a 53rd week is added into the tax year.
Looking at it another way, the calculation in your second paragraph should really have '52-and-a-seventh' in place of '52' (because there are 365 days in a year, not 364) or '52-and-two-sevenths' in a leap year. So, in most years you pay slightly less than a true calculation would yield, so there's some 'catching up' to do periodically.
It complicates PAYE for employees too (and even more so for their employers, who have to check that their tax calculations are correct). This link confirms that there were 53 weeks in the 2018/19 tax year:
https://www.dataplanpayroll.co.uk/payrollblog/tax/when-is-a-52-week-year-not-a-year-when-there-are-53