Just set up a new PC with Windows 8.1 and Office 2013 including Outlook. If I delete a message from my Inbox (on the new PC) it automatically removes the message from the server meaning they cannot be seen on any of my other devices.
I remember when I last set up a new machine there was an option to disable this facility (meaning I could view and delete a message on on device but it would still remain in the Inbox of other devices). That's certainly how my old machine is set up (I can view and delete a message using it but it remains in the Inbox when I look on my other machines). I cannot find that facility on Outlook 2013.
I'm only a poor relation and have only got Office 2003, this is what works for me.
Go to your email settings in Outlook and click on the 'More Settings' box. Then go to the 'Advanced' tab and put a tick in the 'Leave a copy of messages on the server' then OK. You need to do this for every email address your using.