Search

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

The LLM Evaluation Framework

BISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.

AI Observability & Evaluation

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

Harness self-evolution: update targets, theory, evaluation, and reported evidence. Bilingual reading list and project website.

[EMNLP 2026] AgentCompass is an extensible open-source evaluation infrastructure for systematically assessing LLM/VLM agent capabilities.

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

Forecast evaluation library

Multi-agent research execution system: paper reproduction, benchmark evaluation, AutoResearch and claim-evidence graphs

C++ Mathematical Expression Parsing And Evaluation Library https://www.partow.net/programming/exprtk/index.html

Word highlighting, simultaneous editing, split views, math evaluation, un/grep, comment reformatting, UAC elevation, complete regexps (PCRE), Lua lexers, DPI awareness and more (XP+)

rs-poker is a rust library that includes all of the poker evaluation tools that you need from hand ranking and starting card enumeration to a full agent arena for self learning.

EdgeCloudSim: An Environment for Performance Evaluation of Edge Computing Systems

FAME Automates Malware Evaluation

Python package for the evaluation of odometry and SLAM

Expression evaluation in golang

Programmatic modification and evaluation of software

Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

STM32Cube MCU Full Package for the STM32F0 series - (HAL + LL Drivers, CMSIS Core, CMSIS Device, MW libraries plus a set of Projects running on all boards provided by ST (Nucleo, Evaluation and Discovery Kits))

Learn time-series anomaly detection from z-scores to state-of-the-art deep models: runnable notebooks, honest evaluation, real-world case studies.

An engine for symbolic manipulation and numeric evaluation of math formulas expressed with MathJSON

We introduce MKQA, an open-domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). The goal of this dataset is to provide a challenging benchmark for question answering quality across a wide set of languages. Please refer to our paper for details, MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering

PArametrized Recommendation and Ai Model benchmark is a repository for development of numerous uBenchmarks as well as end to end nets for evaluation of training and inference platforms.

A framework for few-shot evaluation of language models.

Stockfish NNUE (Chess evaluation) trainer in Pytorch

Evaluation and Tracking for LLM Experiments and AI Agents

An AI for playing NES Tetris at a high level. Based primarily on search & heuristic, with high quality board evaluation through value iteration.

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.