I agree, in terms of easy access to the things you are most likely to want to see, staying in Manhattan makes the most sense (staying in the Lincoln Square area puts you within walking distance of a lot of stuff if you don't mind 15-30 minute walks. The Hampton Inn Times Square, where we normally stay, is actually about half way between Lincoln Center & Times Square).
You really don't want to spend a lot of time *under* Manhattan learning the ins & outs of the A train vs the D train, when you could be in Times Square. Yes, Manhattan hotels are pricey and it is all hustle & bustle, but isn't that why you go to NYC? For the high prices, hustle and bustle?
But, if you can't convince your husband to stay on the island, you could look in Brooklyn or the Bronx. A relatively small savings, though, for the extra travel time you'd be incurring. I kid (slightly) about New Jersey; Jersey City is just across the river, and they do have legalized gambling there. If being very far removed is the goal, look at Stamford Connecticut and take the train in with all the bankers and hedge fund managers.