Like any other ideology that is firm, it will not tolerate any dissent. I get it, once you get rid of all the undesirables, it seems doable, but even then I doubt it as human nature will not allow it. I can see it working if humans are totally genetically engineered into an ideology, but who wants that, it would be no different from a society of AI's or robots with no free will or anything resembling a "soul", or consciousness, if you prefer.
As far as tyranny or totalitarianism, even the most libertarian society would be called tyrannical and totalitarian by those who want more social or economic control of the economy, as is evident by how liberals view tax and social program cuts, especially if done by executive order (amazingly, they don't mind spending via the same method). Liberals see an authority figure cutting the size of gov't as authoritarian and tyrannical. I've heard Democrats and Republicans recently calling Trump a tyrant for wanting to bring troops home. I've seen one parent call the other insensitive and a tyrant for not allowing a sex change on a kid, while the other parent believes the opposite.
The core belief of libertarianism is the Non Aggression Principle, but even that simple statement is met with utter disgust from some people.
When we can't even agree on what freedom is, it's impossible for any ideology to flourish, the best we can do is what we have done, human achievement in the last 200 years (the last 50 years of stagnation not withstanding, but we have made the world rich which counts for something) is incredible, but it will not go on indefinitely if we can't get along.
There is some data on the fact that humans split down the liberal/conservative line pretty much 50/50, no matter education or anything else. The lines are blurred today as nobody really knows what those terms mean, as I've always been liberal minded but lately identify with most conservative ideals, even though I am an atheist.