Curated by
More in AI Agent Evaluation Frameworks and Benchmarks
See all 12 →Survey on Evaluation of LLM-based Agents
arxiv.org
Inspect: Open-source Framework for LLM Evaluations
inspect.aisi.org.uk
Evaluation concepts - Docs by LangChain
docs.langchain.com
Evals: Framework for Evaluating LLMs and Benchmarks
OpenAI Evals is a framework for evaluating large language models (LLMs) and LLM-based systems, offering a registry of existing evals and the ability to create custom evaluations. The documentation covers setup, installation via pip and Git-LFS, running and writing evals, including support for advanced use cases like prompt chains, tool-using agents, and logging results to Snowflake.
Built for AI agentsACO · 1352 tokens
Summary
OpenAI Evals is a framework for evaluating large language models (LLMs) and LLM-based systems, offering a registry of existing evals and the ability to create custom evaluations. The documentation covers setup, installation via pip and Git-LFS, running and writing evals, including support for advanced use cases like prompt chains, tool-using agents, and logging results to Snowflake.
Tags
openai · evals · llm-evaluation · framework · python · model-testing · custom-evals
Key entities
OpenAI (organization, 1) · OpenAI Evals (technology, 1) · Greg Brockman (person, 0.95) · Git-LFS (technology, 0.9) · Python (technology, 0.85) · Snowflake (technology, 0.8) · Weights & Biases (technology, 0.8) · LLM evaluation (concept, 0.95) · Completion Function Protocol (concept, 0.8)
Classification
framework · language en · status final
Provenance
claude-opus-4-6 via @stacklist/mcp-server@2.0.0, confidence 0.85, 2 Jul 2026