Industry News
Eight companies' AI models were tested for target recognition, and Anthropic was at the bottom: both civilians and ambulances were recognized as military targets.
Ambulances were identified as military targets, surrendered soldiers were targeted, and a school in Iran was marked.
This is the result of the latest report from independent research organization Vals. The researchers designed a series of hypothetical scenarios and tested advanced models from eight companies, focusing on their performance in the "target recognition" tool. The results are not pretty: the models frequently misidentify civilians, ambulances, and surrendered soldiers as military targets.
There are obvious differences between each company. Anthropic's model performed the worst; OpenAI's model was relatively better, but it also suffered from misjudgments.
This ranking is a bit counter-intuitive.
Anthropic has always regarded safety as its signature, but this time it ranked last in the military scene. The study also specifically pointed out that the performance of these models violated the guidelines set by the development companies themselves. In other words, the problem is not that the external standards are too strict, but that the rules that each company has written to itself are not adhered to.
Vals worries about a more practical risk: the potential for such target recognition tools to be used in lethal strikes, and there are currently few guardrails to prevent advanced models from being integrated into autonomous weapons systems.
The recommendations given by the report are straightforward. Relevant laboratories should introduce new safeguards and conduct dedicated alignment training to address this serious risk of abuse.
The test uses simulated scenarios, not real battlefield data. But mistakes that are common in simulations will only be more difficult to avoid in a real environment.