I suppose this is like when Anthropic was using “prompt modification, steering vectors, or parameter-efficient fine-tuning” to poison the work of people working in the LLM field, including academic researchers.
When the model is open weights you can even pass every token (including the chain of thought) though a fourth-party lightweight model like gpt-oss-safeguard to check that it has not become adversarial.
I feel like that's a threat that isn't super difficult to block. Unplug it from the internet, require it to go through an API intermediary to access web pages.
It could, but exposing that would doom the company entirely, and AI doesn't generate code with near the quality needed to get a model to mass adoption, insert malicious underhanded code, ensure that consistently looks innocuous enough to never be noticed, and- most importantly- actually exfiltrate data without being noticed. Once it is noticed, it's game over across the board.
You don't even need that, all models are susceptible to prompt injection. You already need to take extreme security precautions and assume all models can essentially behave like attacker-controlled rootkits.
This is such a common omission: the Chinese models are open, you can host them yourself on your premises. So privacy and independence.