OK. I understand the points everyone is making, but essentially the problem seems to be about responsibility. Assuming the knitting group is not held in someone's house, nobody can just set up a group and off it goes. There's legal health and safety requirements to be followed.
The knitting group has to follow the health and safety guidelines for using the space. So if the person in charge of the knitting group has agreed someone in a wheelchair can join, then that person must make sure that the health and safety arrangements are in place to keep the wheelchair user safe (as well as the rest of the group).
The wheelchair lady and her carer need to know what they must do in the event of a fire (or the fire alarm going off). The carer needs to understand she is responsible for getting the lady in the wheelchair out of the building. This needs to be considered for all of the space used by the group, not just for the main room e.g. the toilet, the kitchen, etc.
So the carer has responsibilities and it seems to me that explaining to the carer about these responsibilities is the ideal way to make it clear to her that she can't just daydream the time away.