The agreement was not just for food, it included seeds, nursery stock and other plant material that have to meet EU standards.
Seed potatoes are different, import from outside the EU is prohibited, as the UK is leaving, agreement has not been reached yet. As we are now negotiating sensibly as long as seed potato producers agree to follow EU regulations they will be allowed, otherwise they will be prohibited.