In most houses these days the cold water taps are fed direct from the mains. However, when the tap is switched off the water stays in the pipes until the tap is turned on. For drinking purposes I always run the tap for a short time, this flushes the pipe through and you get fresher water from the mains. The tap nearest the mains is usually the kitchen tap, and the bathroom tap is usually the furthest away - so you need to 'flush' the pipe a little longer. Also the bathroom tap often is not used for hours so the water will tend to have a stale taste.
Away from home you often see taps labelled "Drinking Water" or "Not drinking Water". The latter will have been stored in a tank, albeit for a short period.