I am constantly surprised that his "I'm one of the lads with my pint and my fag" line convinces anyone. He's a very wealthy man with a public school background who is good at deceiving working class people that he has something in common with them (other than the pint and fag if that is the way they are inclined).
He has obviously annoyed some in the party who would like him to focus on getting us out of Europe, rather than banging on about pints, fags and immigrants. There would be just as many immigrants if we came out of the EU because the economy needs them. In fact there might be more, as the "points system" would allow in more well educated people from India etc than the present system does.