Industry News
The test results showed everything in green; no one could understand how the system was able to function properly. Addy Osmani explained that this phenomenon could be described as a form of “cognitive surrender.”
Addy Osmani works on Claude Code at Anthropic, but his recent writings are actually reminding everyone not to rely too heavily on such tools.
He gave this phenomenon a name: “cognitive surrender” – the act of simply accepting the answers provided by an intelligent system without any further verification. The code looks neat and all tests pass, but after some time, no one in the team is able to explain why the system functions in that way.
He cited a study conducted by Wharton University: People who had access to AI accepted its incorrect answers in 73% of cases, and they even felt more confident when doing so.
This is not only the case for programmers.
He established six rules in total: Before starting to work on a task, first figure out what success actually means; come up with a preliminary answer of your own before comparing it with the results produced by the model; treat the differences observed between your answer and the model’s outcome as if they were written by a junior engineer; let the model challenge your own reasoning; carefully examine which commands the intelligent system executed during each test and what specific aspects those tests were intended to evaluate; and set aside one day each week when you do not use the intelligent system to write any code at all.
In my opinion, the fifth point in this list is the most practical. The tests written by intelligent agents often merely serve to prove that “the code has indeed done what it is supposed to do”; a green status does not necessarily mean that the logic is correct. Many people only look at the final green checkmark and never bother to examine the actual assertions used in those tests.
Perhaps the most difficult rule to follow is the sixth one. With the project schedule already set in place, your boss won’t give you a single “flexible day” for personal matters. Moreover, the decline in your skills won’t cause immediate problems; it won’t become apparent until the system fails or the intelligent agents can no longer function properly.
For a company that makes its living by selling programming agents, it seems somewhat contradictory when its engineers talk about such things. However, the rule itself doesn’t oppose the use of tools; it simply leaves the decision-making power in the hands of humans. This will also be the biggest difference between senior and junior members within the team in the future.