Difference is the contrary of equality, in particular of objects.
Differences can only be stated on the basis of a comparison or categorisation. Since a complete comparison of objects or things is seldom possible in practice, only relevant or defining attributes are used for stating equality or difference. Similar objects are only different with respect to attributes of minor discriminative value.
In order for something to be different, you must have something to compare it to, usually this is your own standards or ideals. So whilst you may consider something to be different to you and therefore suspicious of it, it may only be because of your systemic belief that what you are is not indifferent, merely different itself to anyone who bears no similarity to you.
The same could be said of opinions, where there is always a difference. If you were indifferent, then you probably wouldn't care.