As much as I could argue against this, I really don't have the energy to do so (Brexit debates are so boring). The right actually have a point but that is mainly down to the identity crisis that 'socialism' suffers from. The EU is actually an argument between the right-wing and liberals. But, for the sake of argument, I'm going to accept what you've said so hopefully now you will understand why I (and millions of others on the left-wing) have a problem with the EU. From an economical point of view the EU is very much capitalist/corporatist/neoliberal; this is very much against 'socialism'. I would love the world to be a 'democratic socialist' place, unfortunately that isn't going to happen any time soon. So my second preference is for the UK to be 'socialist' (not in the strictest definition of 'socialism' where the state owns all means of production, etc). Inside the EU, the UK is chained to the neoliberal rules and so, in order for the likes of me, to get what we desire we have to rely on the ECB, the IMF, the EU Commission and the majority of EU countries to turn 'socialist' hence it's better for a British socialist to be out of the EU in order to give 'socialism' a chance in the UK.
Having said that, it also depended/depends on having a 'socialist government in the UK, something that hasn't happened and is very unlikely to happen in the near future now Blair 2.0 is leader of Labour.