Immediately following the Norman Conquest of England castles were built along the Anglo-Welsh border to fend of the 'barbarian' Celts. For several centuries this region of Marches - 'Welsh' land manned by English soldiers - was ruled by Marcher Lords who had nearly all of the rights that belonged to the King in other parts of the Kingdom.
Although technically in the control of England from 1067, it was not until Henry VIII's Acts Of Union, in the 16th century, that the Marches were declared separate counties and part of England.
The Laws in Wales Act 1535 treated the new county of Monmouthshire in a different manner to the other four creations; Brecknockshire, Radnorshire, Montgomeryshire and Denbighshire were each required to elect one Knight to Parliament in common with all other Shires "within the said Country of Dominion of Wales" whereas Monmouthshire required two Knights, as required by "all other Shires of this Realm of England".
Also the Laws in Wales Act 1542 specifically enumerates the Welsh counties as twelve in number, excluding Monmouthshire from the count.
To all intents and purposes, Monmouthshire only became Welsh when it came under the remit of The Welsh Office upon its creation in 1964.