I would think it is safer to use your stick. I joined a Tai Chi class and it has improved my balance a lot, I can bend and stretch much more easily and stand on my toes, a previously lost art and that is very helpful for balance. I practice this when cleaning my teeth.
I. am in my eighties and use my stick but not all the time and it is amazing the difference in the behaviour of others. When I use my stick, cars will stop to let me cross roads, people offer to help me negotiate steps, gates and obstacles. No stick , no help offered. I do get rather cross as I am quite capable of looking after my self and resent the tone of voice used as if I am half witted but they are kind and caring so I bite my tongue and smile.