Most men who use testosterone actually report a decrease in anxiety.
Men who use testosterone that aromatize to excess (ie convert the testosterone to estrogen) are often the ones who complain of increased anxiety. You should have your doctor check your estradiol levels before you do anything else.