Some Economic Analysis
When I complain about senators that have been in office for longer than I have been alive, a typical response is that if people did not like that senator, they wouldn't vote for him.
Of course the problem with this is that even though I may not like a particular senator or congressman, it does not mean that I will happen to be one of his constituents. Congress is composed by people who make national decisions but are only held accountable locally. What is in the interest of the people in his district may not be in the interest of the people in my district. In the jargon of economics, any influence that another state or district imposes on the nation is an externality that is imposed on the rest of the nation.
Now, I think it shouldn't be too controversial to say that those who have been in congress longer have more influence than those who have not. They have made more connections, have more experience playing the game on the Hill, and so on. If this is so, then every state and congressional district has an incentive to keep the same people in (all other things being equal) year after year, since they risk their representative losing sway if they replace him.
To the extent that this is true, competition between states for influence in national policy likely reduces the competition that representatives actually face for their position in their locality, once they are through the door.
I think we would all be better off if we put a cap on the national-level competition. Any cap would be better than none at all; a three or four term limit cap in the senate would still involve less time in office than many senators have enjoyed. Since everyone would be unable to spend more than a certain amount of time in office, there would be more local level competition for representation once a particular member of congress was on their last allowed term.
An Ethical Afterthought
I think there is something inherently immoral about the professional politician. From the standpoint of tradition, it is certainly not what the founding fathers wanted. George Washington stepped down after two terms voluntarily; with his popularity he could have remained and become much more powerful than he chose to be. When Thomas Jefferson did the same, they set the political expectations in a way that made it very difficult for anyone to break the pattern. They were great men; few after have shown similar restraint.
People should have their careers, their passions, and their families. If they choose to serve public office, then it should be only as a small interruption in the course of their lives--they should earn it based on the accomplishments and acclaim they have gained elsewhere.
If this strikes the reader as too idealistic, I will understand. Morality should always be dealt with in ideals; it doesn't mean that those who fall short of perfection are to be deplored. Even Washington and Jefferson had their faults and their ambitions, but they serve as great moral examples that all of the ambitious should aspire to.
It may be that the men who are in office are those who have been selected by the incentives and pressures inherent in the system; that those who would step down as Washington would do and are quickly replaced by those who won't. If so, then it seems reason enough to consider term limits, in the hopes of drawing in higher caliber characters, or at least limiting the time that lesser characters are able to make consequential decisions.