That's true in theory but often wrong in practice.

You could conceivably make products with such as thing as "premium plastic" (some companies/products do), but the overwhelming experience people have with plastic is that it's often cheap injection molding, breaks easily, doesn't look good, etc.

You could also build crap with metal, so of course there's nothing inherent to those materials.

But the perception isn't entirely wrong.