Search

LLM inference in C/C++

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc

Port of OpenAI's Whisper model in C/C++

A curated list of awesome C++ (or C) frameworks, libraries, resources, and shiny things. Inspired by awesome-... stuff.

Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)

C and C++ Compiler Support Page

lightweight, standalone C++ inference engine for Google's Gemma models.

A C++ header-only HTTP/HTTPS server and client library

Private on-device AI suite for Android. Fork of Google AI Edge Gallery with llama.cpp, whisper.cpp, stable-diffusion.cpp, GGUF import, voice chat, vision AI, on-device image generation, biometric lock, encrypted history, and CPU/NPU/GPU acceleration.

A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.

C++ ggml runtime hub for multilingual ASR models: Cohere Transcribe, Parakeet TDT, Voxtral, Canary 1B v2, etc, plus universal forced alignment via NeMo Forced Aligner-style CTC, and others. Fork of whisper.cpp.

A fast library for AutoML and tuning. Join our Discord: https://discord.gg/Cppx2vSPVP.

Abseil Common Libraries (C++)

A C++ library for interacting with JSON.

static analysis of C/C++ code

HF CLI extension to run local coding agent powered by llmfit and llama.cpp

A modern C++20 header-only library for advanced direct system call invocation.

A rclcpp-compatible true zero-copy IPC middleware that supports all ROS message types, including message structs already generated by rosidl.

React Native binding of llama.cpp

QP/C++ Real-Time Event Framework/RTOS implements event-driven Active Object (Actor) model combined with Hierarchical State Machines. Specifically designed for embedded systems (e.g., ARM Cortex-M MCUs).

The missing bridge between Java and native C++

cpprefjpサイトのMarkdownソース

中文的C++ Template的教学指南。与知名书籍C++ Templates不同，该系列教程将C++ Templates作为一门图灵完备的语言来讲授，以求帮助读者对Meta-Programming融会贯通。(正在施工中)

Python bindings for llama.cpp

C++ Insights - See your source code with the eyes of a compiler

The World's First Universal Mod Loader for Unity Games compatible with both Il2Cpp and Mono

Mixed-effects models in R using S4 classes and methods with RcppEigen

Your personal engine for running open source models locally. Use Go for hardware accelerated local inference with llama.cpp directly integrated into your Go applications via the yzma module. Kronk provides a high-level API that feels similar to using an OpenAI compatible API. Kronk also provides a model server to run local work

Python bindings for whisper.cpp

Control panel for VLLM, Sglang, llama.cpp, exllamav3

C++ Client Libraries for Google Cloud Services

A tool interoperate between CoreCLR and Il2Cpp at runtime

C++ port of ZXing

React Native binding of whisper.cpp.

Community recipes for serving LLMs on RTX 3090. Multi-engine (vLLM, llama.cpp, SGLang) and model-agnostic. Currently shipping Qwen3.6-27B configs for 1× and 2× cards.

Universal cross-platform tokenizers binding to HF and sentencepiece

Fresh builds of llama.cpp with AMD ROCm™ 7 acceleration

A modern C++ runtime for building fast and reliable applications.

Go with your own intelligence - Go applications that directly integrate llama.cpp for local inference using hardware acceleration.

Source code for the book Real-Time C++, by Christopher Kormanyos

💃 Dance with LLM in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.

Home Assistant LLM integration for local OpenAI-compatible services (llamacpp, vllm, etc)

High-level C++ D-Bus library for Linux designed to provide easy-to-use yet powerful API in modern C++

ananicy-cpp-rules for CachyOS

A zero dependency web UI for any LLM backend, including KoboldCpp, OpenAI and AI Horde

On-device AI for Android — LLM chat (GGUF/llama.cpp), vision models (VLM), image generation (Stable Diffusion), tool calling, AI personas, RAG knowledge packs, TTS/STT. Fully offline, zero subscriptions, open-source.

C++17 templates between [stl::vector | armadillo | eigen3 | ublas | blitz++] and HDF5 datasets

The C++ REST SDK is a Microsoft project for cloud-based client-server communication in native code using a modern asynchronous C++ API design. This project aims to help C++ developers connect to and interact with services.