Please don't take any notice of the surveyor, maybe he didn't get his oats the night before! 
When we moved two years ago, as I packed away our things, it sort of changed the home to a house, IYSWIM. We had been there 13 years, from me getting divorced and I know what you mean about it being your sanctuary, and I loved that home. It is where my children grew up, I got my freedom and had lots of new friends in, had a wonderful relationship and lots of wonderful memories. But I had to move on. I went round the house and kissed the walls on leaving (I have never told anyone that) and said thank you to it. I now love the home I have now though. It suits me better, although a chair lift and downstairs toilet would make it the best.
I have lived in a couple of houses which I have hated. We bought our second married home after being married seven years, really bad mistake. It was one of those houses which, no matter what you did to it, would never feel right. I was glad when I moved out. One when I was "at home" too was a bit cold and distant home, not warm and cuddly. But my mother wanted it so she got it. It probably didn't help though as she took us away from the area where we had grown up and all our friends were. Out of order imo.
If you know in your heart that you are doing the right thing, then as you pack up, the warmth and love with travel with you and your belongings. I wish you all the love and luck in the world in your new home. X