I'm in a quiz team of men (at 72, they are 7 years older than me) and I always refer to them as, 'I'm seeing 'the boys' tonight'. Mind you, I am the team captain!
Some men can use the words 'girls' in a demeaning manner, I agree, others as a compliment. Either I ignore it or react accordingly as to how the comment was intended depending on the situation. I could say,' That's flattering, I wish I was still a girl' or something like 'What do the other boys think?' The man/bloke/guy/fella/boy should get the appropriate message. In the situation mentioned here, probably better to be spoken to than ignored.