AIRA 2 Enhances Research Agents While BeSafe Bench Uncovers Safety Risks

Researchers are developing advanced AI agents and frameworks to tackle complex tasks across diverse domains, from building-grid simulations to airport management and CAD generation. AutoB2G automates building-grid co-simulation using LLMs, improving grid-side performance by coordinating building-grid interactions. For airports, a semi-automated framework fuses expert knowledge engineering with LLMs to create machine-readable Knowledge Graphs, resolving data silos and semantic inconsistencies for Total Airport Management, with document-level LLM processing proving superior for capturing complex dependencies. In CAD generation, CADSmith employs a multi-agent pipeline with programmatic geometric validation, achieving a 100% execution rate and significantly reducing errors in text-to-CAD models.

To address domain bias in GUI agents, GUIDE uses real-time web video retrieval and automated annotation, improving agent performance by over 5% without model modification. This training-free, plug-and-play framework leverages a Video-RAG pipeline and an inverse dynamics paradigm to inject domain-specific expertise into agents. Meanwhile, BeSafe-Bench is introduced as a benchmark to uncover behavioral safety risks in situated agents across web, mobile, and embodied domains, revealing that even top agents struggle to balance task performance with safety constraints. AIRA_2 enhances AI research agents by overcoming bottlenecks in throughput, generalization, and LLM operator capability through asynchronous multi-GPU workers, a Hidden Consistent Evaluation protocol, and ReAct agents, achieving improved performance on benchmarks.

Furthermore, a new method called Process-Aware Policy Optimization (PAPO) stabilizes training by integrating process-level evaluation into reinforcement learning. PAPO decouples advantage normalization to compose rewards from both outcome correctness and reasoning quality, outperforming traditional outcome-only reward models on benchmarks like OlympiadBench.