Researchers Advance AI Agent Testing While Claude Generates Valid Specifications

Researchers have made significant advancements in artificial intelligence, particularly in the areas of formal verification, multimodal knowledge editing, and self-evolving agent skills. Inductive Deductive Synthesis (IDS) has been developed to address the gap in formal guarantees of full coverage, achieving 7/7 in about 6.8 hours and $106 per spec on average. Agentic Proving for Program Verification has shown that Claude generates arguably valid specifications for 98.8% of problems and certifies implementations against correct ground-truth specifications for 87.5% of problems. Additionally, SkillOpt has been introduced as a systematic controllable text-space optimizer for agent skills, achieving best or tied results on all 52 evaluated cells. Energy per Successful Goal (EpG) has been proposed as a cross-layer measurement framework to redefine the unit of AI energy accounting, showing that agentic workflows consume 4.33x higher mean energy per successful goal than linear baselines.

The development of large language models (LLMs) has led to a growing body of work on knowledge-work AI, including coding, research, and healthcare. However, current knowledge-work evaluation and benchmark design still largely follow the logic of traditional NLP tasks. A three-step approach has been proposed to make explicit how benchmarked tasks represent the work claims attached to their scores, covering task mapping, tested settings, and scoring. The approach has been demonstrated through three benchmark case analyses: GDPval, OfficeQA Pro, and APEX-SWE. Furthermore, the Foundation Protocol (FP) has been introduced as a graph-first coordination layer for an emerging human-AI society, unifying heterogeneous entities and supporting native multi-party organization and event-based collaboration.

Researchers have also made progress in the area of strategic reasoning in large language models. GENSTRAT has been introduced as a procedurally generated strategic environment to evaluate model competence across six axes, including state space, temporal depth, information sensitivity, opponent modeling, risk, and brittleness. The capability profile and jaggedness measure have been proposed to provide a deployment-relevant diagnostic that the overall ranking alone cannot provide. Additionally, the theory of accountability boundaries in agentic ecosystems has been developed, introducing accountability assets and three boundary strategies: component, integrated, and dual-track.