I would make all the invites except for family for adults only, and then you can say personally to anyone that you dont mind them bringing along their kids. That way, people get the message that it is an adult party but you can make a few concensions as appropriate. A work colleague who was having a 40th did that (she said when she handed me my invite that she knew because I didn't have family close by that I had trouble getting a baby sitter and I could bring my kids along if I wanted to but that she wasn't saying that to everyone as only her family kids were invited).
When my neice got married, she excluded children appart from those involved in the ceremony - on my side, that meant that only my 1 year son and my 20 year old twin nephews were excluded, but my sisters little girl was, and on her husbands side, his aunt did not bring along her new born (she had to leave periodically to feed him) and his cousins who brought their 2 small girls along as they assumed that they were invited. I was really annoyed by this. The shoe was entirely on the other foot though when her aunt got married a couple of years later and her own son was not invited!
In the case of a wedding, I think hat you either invite them all or none - however, for a party, i think you do what suits your numbers.