In a recent update, the capabilities of AI agents in professional domains, particularly in law and corporate analysis, have shown significant improvement. Previously, a benchmark from Mercor revealed that these AI systems were underperforming, achieving scores below 25%. This led to the conclusion that lawyers were safe from potential displacement by AI for the time being.
However, advancements in AI technology can occur rapidly. Just this week, the launch of Opus 4.6 by Anthropic has made waves in the AI community. This new model has recorded an impressive score of nearly 30% in initial trials, and when given multiple attempts, it averaged around 45%. The introduction of innovative features, such as "agent swarms," has likely contributed to this enhanced performance in complex problem-solving.
This remarkable leap from previous scores demonstrates that the development of foundational AI models is progressing at a swift pace. Brendan Foody, CEO of Mercor, expressed his astonishment at this rapid advancement, stating, "jumping from 18.4% to 29.8% in a few months is insane."
While a score of 30% indicates progress, it still leaves a considerable gap to reach full competency. Therefore, while lawyers may not face immediate threats from AI, they should remain vigilant about the evolving capabilities of these technologies.
