I don't like the term "privilege," because it gives connotation that you've been given this wholly undeserved, special favor.
What I'd have preferred it be called is something like "un-disadvantage" because that really is the crux of the matter.
People who look white are "privileged" not because they are advantaged; but because they are not forced carry or to deal with the assumptions or even stigmas that society has about people of color. Especially in American society where it used to be the case where all entertainment, media, political, business, etc., etc. figures were white or looked white. It made it so that people who look white were just assumed to be the norm, while everyone else was different.
I'm white. My ethnic heritage is generic European. Even if my ancestors had been disadvantaged; the fact that I look white I know is a huge advantage to me in the USA and abroad. I noticed it most substantially when I have been abroad on work trips. Since I'm out of my comfort zone, I'm already paying way much more attention to my surroundings and social interactions. However when I've been in S. America, Mexico, India, SE Asia and some other areas of the world — the advantage is palpable to the point of it being uncomfortable. A lot of people will take one look and right off assume that because you are white and foreign, it means that you are rich, important or famous. The amount excessive courtesies and weird behavior I have gotten is absurd. (A lot of begging too.)
In the USA I do not feel it is as blatant. However, if I were caught in a sticky situation like say one that involved the police. I'm glad that I look like a generic white European.
And to say so is not "wagging fingers." It's a realistic assessment of how white-looking people are viewed by an American or even global audiences.