CATArena Advances AI Agent Testing While Denario Simplifies Financial Research

Researchers have made significant progress in various areas of artificial intelligence, including neuro-symbolic systems, large language models, and predictive agents. A study on grounding vs. compositionality in neuro-symbolic systems found that symbol grounding is necessary but insufficient for generalization, and that reasoning is a distinct capability that requires an explicit learning objective. In contrast, a framework for optimizing machine learning by evaluating generated algorithms, OMEGA, has been introduced, which starts at idea generation and ends with executable code. Additionally, a live environment for training predictive agents with real-world outcome rewards, FutureWorld, has been proposed, which closes the training loop between prediction, outcome realization, and parameter update.

Large language models have been evaluated for their safety in robotic health attendant control, with a dataset of 270 harmful instructions and 72 LLMs showing a mean violation rate of 54.4%. A study on persuadability and LLMs as legal decision tools found that frontier open- and closed-weight LLMs respond differently to legal arguments, with implications for the feasibility of adopting LLMs in legal and administrative settings. Furthermore, a framework for agentic system for AI-readiness evaluation of heterogeneous scientific data, SciHorizon-DataEVA, has been proposed, which evaluates AI-readiness across four dimensions: Governance Trustworthiness, Data Quality, AI Compatibility, and Scientific Adaptability.

Researchers have also made progress in developing predictive agents, with a study on human-in-the-loop benchmarking of heterogeneous LLMs for automated competency assessment in secondary-level mathematics showing a marked "Architecture-compatibility gap". A framework for operating-layer controls for onchain language-model agents under real capital has been proposed, which reduces fabricated sell rules from 57% to 3% and increases capital deployment from 42.9% to 78.0% in an affected test population. Finally, a closed-loop inverse source localization and characterization framework, Distill-Belief, has been introduced, which decouples correctness from efficiency and reduces sensing cost and improves success, posterior contraction, and estimation accuracy over baselines.