I think taking on that level of child care can only be agreeable, if it's something the grandparents want to do. Should there be an expectation and that is then piled on without consideration for decreasing energy levels or ailments then naturally it will cause resentment. I can remember a full day of looking after the grandchildren when they were young, similar age gap as yours, and whilst still in my late 50s/60s at the time it could be exhausting, all that scrabbling around on the floor and outdoor activities, sometimes swimming, just knackered at the end of the day! I actually looked forward to taking them to the cinema, a chance to fall asleep. Now they're older, they amuse themselves much of the time if they're with us for a day or two.
I think you need to draw a line somewhere, and that needs to be a no to having them over night given you don't feel up to it. 4 and 6 months are very young, and it's quite feasible you will be up in the night with the baby, I would sleep in the spare room with the then baby if we had to do an over nighter, it just seemed easier, although going through the night with a baby is not something you want repeat once you've raised your own children. No need to apologise, you do masses for them already and sometimes you need to prioritise your own well being. My default line was often, "I've been your age, but you have yet to reach this age, so you can't possibly understand how age affects the depletion of energy levels" Suggest the other grandparents, if they are able, step up to the plate. Although I would add it would be their prerogative to say no too, childcare should be given willingly it's a bonus and should be received graciously without guilt tripping for the occasions when the grandparents turn down a request.