Yes, it does depend on which side is which, or on your perspective and frame of reference. Societies or governments may impose order that is reasonable and just; or "order" that is unjust and brutal.
To some degree what is just or unjust is a matter of perspective.... a good many of those who make their livings through criminal endeavors view government's interference in same as "unreasonable", lol.
However, the question of whether authority resorting to violence is a result of someone "committing assault" upon the common good is a complicated question. What is an assault on the common good?
Did you know that there are certain plants and animals that, if you get caught bringing them into the country (perhaps merely seeds or leaves of same) you have committed a FEDERAL FELONY and can be locked up for many years for this.... even if you had no idea the plant was on the list? Even if it was completely legal in the country where you bought it? Even if the presence of some of the plant matter in your luggage was entirely accidental?
Allow me to point something out.... at some point in your life, you have almost certainly committed a FELONY crime. That is, a crime you could be locked up for a year or more for. Now before you get indignant and protest that you are not a criminal, I'm not saying you are one... not in the usual sense. What I mean is there are literally so many laws on the books that Supreme Court Justices do not know them all without referring to a well-stocked law library, and they are not all reasonable and common-sense things!
You probably didn't mean to; you probably didn't KNOW it was a felony and STILL don't know, and would be SHOCKED to find you could spend years in prison for something that seemed reasonable or harmless... but it is most likely so. Such is the complexity of our society, and our lawmakers are not always reasonable.
Mutual consent probably SEEMS to work well... if you've spent most of your time around reasonable people with much to lose from non-compliance. Spend some time in certain other places and you'll see a very different picture of how well mutual consent works among people who don't hold it in high respect as you do.
Have you ever seen this video? It is accurate and quite compelling and well worth the time, and addresses much more than the title suggests...