I broke my ankle last autumn, but found I loved walking with a stick, despite initial 'old lady' worries. It certainly eased my worries for the future. I could hook down plants to have a closer look, people skirted around me slightly so as not to be tripped up, and once I waved it at an annoying little boy, which was most satisfying! Getting up and down steps was an eye opener, as I had wondered how I would cope as my arthritic hip gets worse.
However, not once did it cross my mind that I might look like a sack of spuds! Wear fitted clothing rather than baggy loose clothes? I have a really nice olive green leather satchel bag that I used.
On reflection, I would say, wear fitted clothes, and make the bag a smart one. Then people will see you, and not the stick.