Next time you're watching something together, just keep asking him, "What did she say?" "What did he say?" After a half hour, ask him what distracts him most, your questions or the sub-titles. Think he'll pass you the remote :-)
they're distracting even for me, Tilly - but not half as distracting as trying to read lips. Fortunately, my OH is also hard of hearing so doesn't object.