the patrol boats were unarmed, the sailors and Marines would have been armed with their personal weapons (rifle 5.56 or to give it it's popular name SA80), however they would only be able to open fire if they belived that they were going to be killed by the Iranians, or if the Iranians opened fire first, the Rules of Engagement are VERY stict on this matter and is enshrined in international law.
Each boat has an onboard GPS (Global Positioning Satalite system) which would show them their exact location, it is VERY unlikely that the RN would stray into Iranian waters, it is also unlikely that the Iranians will release the GPS's when the UK personnel are repatriated, they still have the patrol craft and GPS from 2004 when they illegally captured 8 RN personnel.
When captured they (the UK Forces Pers) are only obliged to give the "big 4" pieces of information, they are
1) Service number
2) Name
3) Rank
4) Religion
It is also very, very unlikely that they would admit they were in Iranian waters even if they were. Once again this is enshrined in international law, called The Law of Armed Conflict, including the Geneiva Conventions