British series from late sixties, often repeated, out on video/DVD. The line was spoken by Number Two. Patrick McGoohan played the title character, known as Number Six, imprisoned in a strange village. Google should reveal much...
It is from the series The Prisoner, used in the episode Dance of the Dead, spoken by a female obsever of The Prisoner when he asks her how long she's been there.