arXiv 人工智能· Daniel Furman, Preeya Kirani, Ibrahim Sheikh, Zhen Li, Peter Zhong, Michele Catasta·· 9 hours agoSelectedAI score69
ViBench 2.0 Benchmark Released: Evaluating Code Agents' End-to-End Delivery Capabilities for Long-Horizon Software Development
ViBench 2.0: A Benchmark for Long-Horizon Software Building
AI guide
The Replit team has introduced ViBench 2.0, a long-horizon software development benchmark spanning 17 full-stack web products and 241 feature requests to evaluate models' capabilities in delivering complete software end-to-end.
Why it matters
The findings reveal a reliability gap between individual test pass rates and full-stack delivery, offering an end-to-end measurement approach for assessing code agents' long-horizon engineering capabilities.
Source:arXiv 人工智能 · arxiv.org