@randomcitizen123 It seems to me that you are a good nature'd person that has concerns over other people and how they are/think.
Without knowing your age it is difficult to give an idea as to the potability of anxiety or not.
At the rip old age of 42 I have (for many years now) always been concerned about what others think, but not so much that it would cause me personal issues.....
for example: I used to be extremely conscious of my size (build) ... to the point that if I was sitting by the river on a summers day I could hear others joking about my stature.... This however did not stop me from going the the river to enjoy my time there.
I would suggest that you may need to learn not to over think about what others are saying or thinking about you. If there is something that they are saying (negatively) about you but not saying it to your face then they are the one's that have the problem because they can't speak directly to you. This would also suggest that they are not worth your time, effort or thought.
If on the other hand they mentioned to you what they were saying (good or bad) then maybe they are worth making time for.... because they have taken the time to communicate to you their thought's or feelings.
End of the day, don't waste your time on it.... :)