I do think this is appropriate though since a key characteristic of a black swan is that people can look right at it but not see it due to psychological biases.
Agreed, but "black swan to most people but white swan to a few" is harder to say than "gray swan".
For that matter, Nassim Taleb himself predicted that the American mortgage industry was, I think he wrote something like "a barrel of dynamite" or some words to that affect. But, for most people, the mortgage crisis was a black swan.
The original metaphor, though, was that no matter how many swans you looked at in Europe, and how carefully, you would have zero data to tell you that black swans were possible, until you went to another country and discovered them.