The short answer, as always, is that there is a history of discrimination by whites against blacks. So, for example, a "Miss black America" pageant and a "Miss white America pageant" would not be the same issue, when black women faced such exclusion from any real equal chance and participation from the dominant beauty pageant in the country, and this 'double standard' could continue even after the "Miss America" pageant became less discriminatory.
As for whites doing black voices or parts in entertainment: there's an issue of a history of discrimination against blacks there, and an unfairness to black entertainers being hired. That creates an injustice that is more sensitive about whites taking black parts than blacks taking white parts. As things become more fair, the double standards tend to go away.
But what is it that requires James Bond to be white? He's not a real historical figure.
When Daniel Craig was selected, there was some outcry, 'James Bond is not blonde', and that's without the race issue. But Craig won over most fans. There's no real difference for a black actor to do the same - or even a woman Bond, Jane Bond, including a black woman. It is a bit jarring for some fans, but it's an option that might or might not 'work', depending partly on the actor, and partly on fans' level of closed-mindedness.
But the real answer likely lies more in the marketing research showing that the franchise gets 'stale', and a change like this, in light of changing demographics, keeps it more 'fresh', and that the number of white fans who abandon the franchise might be outnumbered by the number of new fans of color who warm to the series. In short, money. The 'controversy' it causes tends to be seen by the owners as good because it's press and attention.
It doesn't make anyone 'wrong', except those whose desire for Bond to remain white isn't based on 'tradition' but racism; some would prefer keeping him white and male, some would prefer change. It's a matter of taste for both sides largely. Bond has already long had similar controversies, where his 'sexism' appealed to some men, while it alienated some people as women's equality gained ground, calling the character a dinosaur.