Yeah it's a pain, rival phone makers tend not to make it easy and it normally has to be done via outlook (or similar)
Transferring by sim card works if both phones can use the SIM card (so on the same network, or unlocked) but modern phones will store loads of extra details under each contact (Home No. Work No. Mobile No. Email addresses etc etc) but a sim will only store first name, last name and one phone number for each contact so if you use the extended contact info on the source phone the contacts get buggered up in various different ways (depending on how each phone deals with them) when you copy them over.