Sounds lovely to me. My DH built a long dry stone wall and in one section, put in a little fairy door with some colourful marbles cemented in at its base. Because the door is a beige sort of colour and set back in a few inches, it blends into the wall and you only see it when you look hard enough. He did this whilst I was on holiday one year and I thought it was wonderful.
I thinks it sounds quite nice. Perhaps the answer to making it tasteful is to choose subtle colours that enhance the colour of the stone and aren't immediately obvious, rather than using bright colours. Or you could graduate the colours, from subtle to bright, across the patio working from the sunniest area to the darkest.
My patio badly needs re-pointing. I have the 'brush-in and wet' stuff and plan to do it when the weather cools down. I have had the idea of setting in an occasional shiny bead, making sure they are not a slip or trip hazard. My garden is quite feminine, with lots of flowers, fairy lights and metal sculptures of peacock and cats. What do you think of the idea and how could I make it tasteful rather than tacky?