i agree you should ask him - maybe he was just trying to give you an out - thinking you may not want to go and so you wouldn't feel obliged
i would tell him you would like to go, and see what he says - he might be made up that you want to.
my dad always says not to get him gifts, he acts all bothered that we spent our money and took time to get him something - yet one year i got him a smaller bottle of whiskey than usual (it was in a box and i didn't realise it was a smaller bottle inside - and he was offended that it was small, made a sarcastic comment... so you can't win sometimes, people just feel like they have to make a show of not wanting you to put yourself out, to show they don't expect it from you etc.
your stepdads 'reason' for saying what he said is not a reason - its just your perception of what he actually meant and you may be wrong ...
speak to him and if he is still weird about it then i would be a bit peeved too - but don't just take this vague conversation as fact
i have wondered in the past just how many lives are ruined or changed by assumptions, misunderstandings, misconstrued words etc, that are never resolved because people 'don't like to make a fuss' or pretend they're not bothered or think they are taking the higher ground by saying nothing.