Like you, I haven't heard all his albums but to tell you the truth I don't want to.
His greatest hits and live albums are fantastic, but I find his non-released singles very boring and samely.
I agree that Yellow Brick Road though does have some good non-singles, particularly Sweet Painted Lady. There really isn't enough songs about prostitutes!!!
And as you appear to be a fan, I have to say my Favourite Elton single ever is the relatively recent "I Want Love"