If the voucher money was enough to cover the cost of public education, there would be an affordable alternative. And the public schools would be made better by the competition with each other and with the private schools.
You're right that (some) private schools can be very selective, but I've never heard of one selecting students based on their economic status as long as they're able to pay the tuition.
It'd work something like this. The taxpayers pay the government (state or federal, but definitely not local) a certain amount of tax money to be spent on education. The government would then allocate something like 10-25% of it on individual public schools, and 75-90% on individual students. The government could apportion the 75-90% voucher money to be spent equally on students, resulting in the government granting every student $X per yera to attend a school of his choice.
The students/parents would then inform the government where they'll be attending school, and the government writes a check to that institution for $X. If the tuition is more than that, the family would be expected to cover the difference. As long as the voucher was enough to cover the tuition at public schools, no one would be left out.
This is more or less the way that our college system works, which is the best in the world.
Some of the parochial schools cost as little as $5,000. The number of affordable private schools would most likely increase if we switched to vouchers, because the public schools would be more competitive. The private schools would need to lower their cost to compete with the public schools and/or improve their education even more to differentiate themselves from these new-and-improved public schools.
But the public schools spend over $10,000 per year per student under the current system. Switching to vouchers wouldn't require a massive tax increase, it would just require smarter spending of the education tax money the government already receives.