Forget the value of the items. You can only reject what the law allows you to reject. The buyer, (you) cannot just simply return goods. You can only do so (and be compensated for doing so), if the goods are not of satisfactory quality, or they have been misrepresented to you. So it's a yes or no answer.
If you have valid grounds for rejecting the goods under the Sale of Goods Act 1979 then yes, you can reject the the goods that are faulty/have been misrepresented, and keep the goods that meet the requirements of the 1979 Act. If not, you're stuck with them.
So you can't reject the two expensive items if it's the inexpensive item that has caused the problem, nor can you reject the less expensive item if only the more expensive items give rise to a claim.
You can't return the goods simply because both you and your partner have bought the same goods and you don't want 'doubles'.