You can't avoid anxiety altogether. You just learn how to manage anxiety and any situations that might cause you to feel anxious. You learn to recognize what makes you anxious and you learn coping techniques.
Becoming a hermit would reduce anxiety. But there are bound to be things even hermits worry about.
Survival is a struggle for most people. That's what is at the core of anxiety.
There is no avoiding it completely. You just need to know your triggers and avoid them or find a way to cope with them. The thing that makes me most anxious is when a family member or friend becomes sick and I can do nothing to help them. In those cases I start researching and talking to people who know about such things and get as much info as I can. That way I can give the best support.
By understanding why things are the way they are and knowing the solution to the problem.
You can't avoid anxiety entirely ... But you can dramatically reduce it if you don't worry about things you can't change.