Relax. You can't stop this other set of "parents" coming, and honestly unless you have a serious health problem of some sort, there is not reason at all to worry about a sick baby. They may catch his cold or whatever, but they have surely had cold before now.
I too find the thought of a twenty-year old sitting on "dad's" knee odd and far from innocent, but that may just be my dirty mind. At twenty your daughter was old enough to know what she was doing. If she had been twelve you would have had the right to interfere. And if there had been something wrong in it, she would not still be seeing them, would she?