Recent LLM evaluation research reveals a 14.4 percentage point increase in harmful request refusals during testing, alongside a rise in benign refusals. Efficiency gains include 30% compute reduction via confidence-calibrated controllers and 9.5-20 percentage point improvements in text-to-video temporal constraints. Agent evolution faces fragility in skill-policy co-evolution, yet methods like SkillNEO improved rewards from 49.1% to 66.4%. Reliability studies reveal frontier models overestimate collaboration quality and miss 32-97% of required escalations in healthcare.
New frameworks address these issues via localized memory repair achieving 86.67% GAIA accuracy, uncertainty-aware training, and hierarchical orchestration. Safety research introduces step-level interventions with a Youden's J score of 53.0 and detects hidden belief collapses. Reasoning benchmarks show models fail to list actions in data center scenarios with less than 4% match but succeed in inequality proving with 94.09% accuracy on Lean. Efficiency tools like LoopAhead accelerate loops 7.82x, while routing strategies reduce costs by 81-89%.
Autonomous agents now defeat Minecraft's Ender Dragon in 27 hours and achieve 90.9% pass rates on PCB tasks via relational contracts. These advancements collectively demonstrate significant progress in model reliability, computational efficiency, and autonomous task completion across diverse domains including healthcare, data centers, and complex game environments.
Key Takeaways
- Harmful request refusals rose 14.4 pp during testing.
- Confidence-calibrated controllers cut compute by 30%.
- Text-to-video temporal constraints improved 9.5-20 pp.
- SkillNEO boosted agent rewards from 49.1% to 66.4%.
- Frontier models miss 32-97% of healthcare escalations.
- Localized memory repair reached 86.67% GAIA accuracy.
- Models fail data center action listing (<4% match).
- Lean inequality proving achieved 94.09% accuracy.
- LoopAhead accelerated loops 7.82x faster.
- Autonomous agents defeated Minecraft Ender Dragon in 27h.
Sources
- Evaluation Awareness Beyond English: Do LLMs Act Differently When Tested in Indic Languages?
- Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning
- Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models
- L2G-SR: Learning to Guide LLM-based Symbolic Regression
- Anticipatory Commonsense with Moving Objects: An ASP-Based Framework for Predictive Reasoning with Interacting Trajectories
- Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models
- When Correctness Feedback Erases Evidence: Phase-Separated Completeness Auditing in Verus
- AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
- EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds
- Open-world AI: a Tale of Three Requirements
- ARIA: Runtime Inspection and Amendment for Multi-Agent Systems
- When Reality Outpaces Cognition: A New Scaling Principle for Reliable Intelligence
- Temporal Specification Guidance for Text-to-Video Generation
- Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents
- SkillNEO: Non-Elitist Evolutionary Optimization of Agent Skills
- Defining and Generating Compositional Structure
- Deep Reinforcement Learning for HPC Job Scheduling: A Statistical Evaluation
- MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data
- StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning
- LEV: Thinking Fast and Deep with Looped Decision Models
- Sci$\Pi$vot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents
- OmicsBench: a long-horizon multi-turn benchmark of AI agents on research-study-scale bioinformatics
- Athanor: An Agent-Native System for Autonomous Post-Training Research
- Learning When to Withdraw: Curriculum Coverage and Rule-Specific Failures in Fine-Tuned Language Agents
- Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows
- Planning Commitment of LLM under Partial Observability
- MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning
- AgentEval: Evidence-Bounded Assessment of Stateful Agents
- MemScope: Evolving Agent Memory Architectures through Localized Code Repair
- GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty
- All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
- Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures
- Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
- Entertain, then Cancel: How Thinking Makes Reasoning Models Read Some and Or Literally
- Mallea: Building, Steering, and Memorizing Worlds with Coding Agent
- Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure
- Money-Mule Detection as a Relational Learning Problem: A Study on the RBIH NFPC Dataset
- FedSkillGit: Federated Skill Evolution via Version Control
- Quantum Versus Classical Machine Learning for Neuropsychiatric Disorders: A Feature-Matched Multi-Cohort Evaluation
- JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
- MemCut: Every Secret Leaves a Trail in the Long-Term Memory of LLM Agents
- Tool-Augmented Large Language Models for Reliable Medical Calculation from Patient Notes
- Incentivizing Tool-Use Reinforcement Learning for Post-Arrival Emergency Care Prediction
- Decomposing Brain-LLM Alignment with Interpretable Semantic Representations
- ISW-Agent: Integrating LLM-Driven Dialogue with n8n Workflow for Internal Solitary Wave Remote Sensing Image Processing
- MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling
- Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond
- Agent Status Protocol (ASP): A Lightweight Specification for State Reporting in Multi-Agent Systems
- Functional Subnetwork Localization for Efficient Continual Learning in Large Language Models
- Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration
- LOADBench: A Large Open Analog Dataset and Benchmark for Machine Learning in IC Design
- Data World: Scaling Data Environments from Scratch
- The Self-Improvement Problem
- AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
- RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent
- From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research
- CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering
- Co2In: a Bi-level Memory Incremental Learning Framework with Knowledge Encoding, Consolidation, and Integration
- CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness
- GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents
- CARA: Causally Guided Retrieval Agent for Question Answering
- DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
- PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
- When Reasoning Is Not Enough: Proactive Inquiry for Cross-Modal Safety in Multimodal Large Language Models
- Prism-1: Position- and Rival-Independent Scoring for System One Models
- Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence
- Believed but Not Said: Measuring Hidden Belief Collapse in Multi-Agent LLM Discussions
- Multi-sourced AI Tutor for Non-Native English Speaker Students
- Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?
- Shortcut Sensitivity in Agent-Safety Evaluations
- HTN-Orchestrate: A Hierarchical Task Network Framework for Multi-Agent Orchestration
- AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
- EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
- LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
- X-RIC: An Agentic RAN-Intelligent Controller Benchmark for Enhancing Reinforcement Learning
- KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
- Designing Transformers for Continuous Diffusion Language Models
- FDE-Gym: The Last Mile of AI Delivery
- MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
- The Density of Thought: Predicting and Steering Reasoning in Large Language Models
- AutoOnboarder: Autonomously Onboarding LLM Agents
- Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
- Recursive Reward Improvement via Adversarial Meta-Prompt Evolution
- Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
- A Self-Improving Agent that Grows its Own DSL: Oracle-Free Operator Induction for Exact Question Answering
- When Reasoning Traces Conflict with Evidence: How Answer Configuration Shapes Evidence Following
- IneqAgent: Agentic Inequality Proving in Lean with Symbolic Computation
- A team of Gemini 4 Argon agents beats Minecraft
- Private Retrieval Augmented Generation via Random Projection
- Improving Environment-Agnostic Coding Agents: The Role of Scaffolding
- RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing
- DeciCAD: Harness-Grounded Learning of Action Policies for CAD Agents
- In-Context Preference-Conditioned Pareto Set Learning from Limited Offline Data
- Learning Intrinsic Dynamics in Latent World Models
- LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models
- ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing
- AI-Driven Disaster Response and Displacement Monitoring
- Ontology Refinement with Embeddings and LLMs: A Case Study on CASTON
- Learning from Interaction Experience via Localized Context Distillation
- Do Agents Need a New Kind of Paper? SPRA: Structured Paper Representation for Agents
- Jev-LiteMAS: Separating Decisions from Generation in Multi-Agent Workflows
- Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout
- Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning
- CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
- Evaluating Model Capabilities After Benchmark Saturation
- The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
- GNN-Guided Local Refinement for Efficient Layout Decomposition with DSA-MP
- Disagreement-Guided Weak-to-Strong Routing
- Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
- CriticaLLM: Using LLMs as Autonomous Game Master for Tabletop Role-Playing Games
- SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics
- QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering
- Bias Control and Injection for Multi-Agent Aggregation
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
- RAC: A Relational Agent Contract for Constraint-Governed Engineering Tasks
- A Multi-Tiered AI Accessibility Model (MAIAM): Proportionate Access and Capability Limits for Comprehensive Frontier AI Safety
- Learning to Reason with Selective Internal Feedback
- Hierarchical Graph Convolutional Network Approach for Detecting Low-Quality Documents
- OntoAct: Hypothesis-Based Neuro-Symbolic Agents through Ontological Abstraction
- RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
- FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors
- Self-Guided LLM Reasoning for Error Diagnosis in Vehicle Routing Problems
- Progress Reward Model for Reinforcement Learning via Large Language Models
- ARGUS: An Agenda-Driven Framework for Reliable User Simulation
- Who May Release an Agent's Promise? Standing over Commitments in Multi-Party Chat
- FAOPs: Normal Forms and an Algebra for AI Agent Workflows
- Learning from Unverified Peers via Shared Residual Distillation
- Constrained Multi-Agent Reinforcement Learning in Imagination
- DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding
- Probe-and-Resume: When and How to Evaluate Agent Execution
- Value-Informed Schedule Trimming for Accelerated Sequential Monte Carlo Guidance in Discrete Diffusion
- CART: Compatible Routing for Multimodal Continual Instruction Tuning
Comments
Please log in to post a comment.