Alice if there's a danger that he might embarrass you by wearing it in the presence of your friends and family, I think the kindest approach would be to ask him to put it on, with all the things he might wear it with, and give a carefully considered opinion along the lines of, 'Well, it's obviously good quality/value for money/ beautiful fabric (choose most appropriate), but that colour doesn't really flatter you/ the trouser length is a bit short/ I think the fabric might pill/ of course it's dryclean only' and perhaps offer to accompany him to get a replacement?
What was the garment by the way - a sort of reindeer jumper of the kind bought for him by Mark Darcy's mother in Bridget Jones 