Didn’t you post about this before? It is difficult to gauge it just right with little ones. The fine line between fun, and basically...not fun.
I personally don’t go in for the grandchildren overnight thing, unless there was an emergency. Is there any need for them to stay at all at the moment? Particularly as the five year old is on edge. Surely it’s supposed to be enjoyable for them.
I think I said before, my husband can put his foot in it with the ‘ fun’ thing which just isn’t! We normally just say something like, “ silly grandad, what is he doing?”. Then ours go home, and it’s all forgotten next time.
Hope you manage to sort it.