Via the social contract, by infringing on the rights of another, one volunteers to temporarily suspend the right to freedom. A right cannot be taken away or lost as they are inherent to the human condition.
Actually, a problem with Social Contract theory is that it can't deal with free-riders. If you never agreed to the contract then you can't be obligated to its terms. So you can infringe on the rights of people who did ascent to the terms of the contract without being obligated to submit to any punishment.
