come now. americans are not the only ones to label each other politically, that happens everywhere - they just use different labels.
And actually, sometimes labels do matter, especially when we choose them ourselves. For instance, I am a libertarian. This is helpful because knowing that I am a libertarian will allow you to predict my position on things that haven't come up yet in debate.
Why do I accept the label libertarian? Because every belief of mine comes from consistent first principles. So yes, I am always against government intervention and/or power because I believe that it is coercive and immoral.
So you can ask yourself 'Does X issue involve coercive force?" and then predict my position on it fairly accurately.
If you don't find that any labels fit you, maybe it's because you don't have consistent principles. Maybe you have several core principles but they are in conflict with each other, and so you pick and choose when they matter to you based on the situation.
When you say "I think what I want about each topic" I hear "I decide what my principles are based on convenience". It doesn't give you moral high ground - it's actually a moral weakness.
This isn't to say that all labels are good, and that using labels disingenuously as a weapon should be condoned. Certainly people can use them to beat each other with. But its worth considering why we do use them, what value they do have, and what label you really think should apply to you. Because if you can't even label yourself accurately, then you might not know what you actually believe yourself.