Exgentic Advances General Agents While FIRE Enhances Financial Benchmarks

Recent advancements in AI are pushing the boundaries of general-purpose agents and world models, with new frameworks and benchmarks emerging across diverse domains. For general agents, a unified protocol and framework called Exgentic aim to systematically evaluate their performance across environments, showing they can generalize without domain-specific tuning. In the realm of world models, the "Trinity of Consistency" (Modal, Spatial, Temporal) is proposed as a defining principle, guiding the development of unified architectures and a new benchmark, CoW-Bench, for evaluating multi-frame reasoning and generation. For specialized agents, progress is seen in areas like financial trading, where fine-grained task decomposition in multi-agent LLM systems significantly improves risk-adjusted returns, and in route planning, where MobilityBench evaluates LLM agents on real-world mobility scenarios, highlighting challenges in preference-constrained planning.

Research into AI safety and reliability is also accelerating. CourtGuard offers a model-agnostic framework for zero-shot policy adaptation in LLM safety through adversarial debate, demonstrating adaptability to new governance rules. AgentBehavioralContracts (ABC) introduces a formal framework for specifying and enforcing agent behavior at runtime, bounding behavioral drift and improving detection of soft violations. For LLM reasoning, latent reasoning methods are analyzed, revealing shortcut behaviors and a trade-off between supervision strength and the ability to maintain diverse hypotheses. Furthermore, a decision-theoretic view of steganography is proposed to detect and quantify hidden information in LLM reasoning, addressing limitations of classical methods.

AI is also being applied to complex scientific and engineering challenges. In computer architecture, ArchAgent uses generative AI to discover state-of-the-art cache replacement policies, achieving speedups faster than human-developed policies. For mass spectrum prediction in metabolomics, FlexMS provides a flexible framework for benchmarking deep learning tools. In biology, LLMs are shown to significantly uplift novice users' accuracy on biosecurity-relevant tasks, even outperforming experts on some benchmarks, while also raising concerns about dual-use risks. For scientific idea generation, GYWI combines co-author graphs with retrieval-augmented generation to provide controllable context and traceable inspiration paths for LLMs.

New benchmarks and evaluation methodologies are crucial for advancing AI. ClinDet-Bench evaluates LLMs' ability to recognize determinability under incomplete information in clinical decision-making, revealing failures in premature judgments and excessive abstention. FIRE is a comprehensive benchmark for evaluating LLMs in financial knowledge and practical business scenarios. AMA-Bench focuses on evaluating long-horizon memory for agentic applications, introducing a new agent memory system that incorporates a causality graph and tool-augmented retrieval. For route-planning agents, MobilityBench offers a scalable benchmark using real user queries and a deterministic sandbox for reproducible evaluation.