I rather agree with M0nica.
If your husband is genuinely not a ‘children’ person, let him go upstairs and say something along the lines of ‘Grandad is reading upstairs in the peace and quiet, that gives me loads more time with you, which is lovely’.
My husband adores his adult children and dotes on our grandchildren but in small doses. He doesn’t ‘do’ any of the hands-on school pickups, swimming, etc. But he is a huge financial support to both families, and loves to cook meals for us all.
I think there have been threads on here and on Mumsnet where the second husband or wife, more or less wont allow any contact with the previous spouse, children or grandchildren.
Your second husband doesn’t seem to be in that terrible camp. While it’s tempting to say LTB, there may be a reason why he feels uncomfortable around children, and it might come to light or it may not.
Have a wonderful Christmas whatever happens, enjoy yourself and if he chooses to hide upstairs, then so be it. ?