Or wherever they happen to be at the time. LB Johnson (JFK's Vice-President) was famously inaugurated aboard the plane he was on following JFK's assassination - I don't think there's any Constitutional requirement over where it takes place. Obviously you make it a big public occasion if you've got the time to plan it.