What legally gives the Government power over me?

I dont remember signing any contract or swearing any allegiance to the government. I have not consented to be governed/ruled by them. What legally gives them power over me if I do not consent to it. Is their rule even Legal at all? How can I take it back and become sovereign over myself? (U.S)