Do nothing, leave the grandparents alone, certainly do not drag the poor child around to their doorstep, like Little Lord Fauntleroy. If they do not want to fill the role of grandparents, nothing you do will change their minds.
Accept their decision. Leave them be, perhaps a photo of the children in their card at Christmas but nothing more.
The children will not suffer because one set, or even both sets of grandparents play no part in their life. My DH grew up with no functioning grandparents and plenty of children past and present have been in that situation and not suffered in any way. His parents just accepted it and so did he.
The one thing that will cause your children distress is you getting your knickers in a twist about trying to build a relationship where two of the people supposed to be involved are not interested in having one. You will never succeed.
Anyway why are you all so obsessed by building this relationship? Yes, I know it is the fairytale picture, children, 2 parents and 2 sets of doting grandparents. Just how many children actually have families like that? With divorce, remarriages, deaths, very few indeed. So stop seeking the unattainable.
Accept what you have got and give the uninterested grandparents and the confused little boy a break.