Windows here:
Software that communicated with broker had probably hard coded port, so I couldn't change the activemq.xml
I had turned on connection sharing for one of the connections, I've disabled it, and it helped, HUGE thaks to Bin S for this info as netstat didn't show this port as occupied!