Open-source benchmark LEEROY lets AI agents team up to clear a World of Warcraft dungeon
Leeroy: Making AI agents play World of Warcraft
Developers have open-sourced LEEROY, an AI agent evaluation benchmark designed to test long-horizon planning and collaboration by having LLM-based agents team up to tackle World of Warcraft dungeons. Because single-turn model inference takes significant time, the agents generate Lua scripts executed in real time by headless clients and make decisions based on structured data parsed directly from network protocols. In actual testing, a team of five agents with no predefined coordination instructions spontaneously coordinated, communicated skill usage, and cleared the Deadmines dungeon in 17 hours and 41 minutes. The project's source code is now available on GitHub.
The project proposes an asynchronous scripting architecture to bypass model inference latency, offering a transferable engineering reference for evaluating multi-agent collaboration in complex, real-time environments.
Source:Hacker News · AI · pthm.dev