Ultimately (and I have thought about this a lot, because I have also given up friends who are too different to me) you need to have things in common with friends. To be honest, I had NO Brexit supporting friends.
But I DID have 2 Trump supporting friends in the US. They were people I had worked with that seemed fine. But they were pro gun lobby, had racist opinions that I hadn't noticed while working with them and were VERY pro Trump. One of them, a rape survivor, supported the promotion of Brett Cavanagh.
It was really sad, and I told the one I really liked why I was doing it. And she was sad too. But then she made ANOTHER racist comment. So...