He was born in the Bronx on May 9, 1949; Joel's family moved to Levittown, "a suburban housing development on Long Island", in 1950. It was one of the first planned suburban developments in the US and offered affordable housing to,essentially, New York's urban poor. Levittown was built on the edge of a traditional rural town - Hicksville - and came under that town's jurisdiction. Hence the confusion. There's an interesting newspaper article on Joel's references to Hicksville and Levittown and whether or not he's embarrassed or confused about his roots:
http://www.antonnews.com/hicksvilleillustratednews
/1999/08/13/news/
and there's a passionate response from one of that paper's readers:
http://www.antonnews.com/hicksvilleillustratednews
/1999/07/23/opinion/letter.html