ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation

While LLMs increasingly assist individual users, there is a critical need for agents that can proactively manage complex, multi-party collaboration. However, the scarcity of systematic evaluation methods for these group dynamics limits the development of AI capable of effectively supporting teams. Here, we present P RO M EDIATE 1 , the first testbed for evaluating proactive AI mediator agents in complex, multi-topic, multi-party negotiations. P RO M E - DIATE consists of two core components: (i) a simulation environment based on realistic negotiation cases with a plug-and-play proactive AI mediator, capable of flexibly deciding when and how to intervene; and (ii) a socio-cognitive evaluation framework with a new suite of metrics to measure consensus changes, intervention latency, mediator effectiveness, and intelligence. These components establish a systematic framework for assessing the capability of proactive AI agents in multi-party settings. Our results show that a socially intelligent mediator agent outperforms a generic baseline, via faster, better-targeted interventions. In the P RO M EDI - ATE -Hard setting, our social mediator increases consensus change by 3.6 percentage points compared to the generic baseline (10.65% vs. 7.01%) while being 77% faster in response (15.98s vs. 3.71s). In conclusion, P RO M E - DIATE provides a rigorous, theory-grounded testbed to advance the development of proactive, socially intelligent agents.