Search

SGLang is a high-performance serving framework for large language models and multimodal models.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

Visualize, query, and stream to train on multimodal robotics data.

Official Code for OpenVLHarness: Real-World Visual Agentic Tool‑Use with Multimodal Memory

Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage Less.

fast rust coding agent, extensible in any language, multimodal inputs, deep provider integrations.

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

🤖 Type-safe, provider-agnostic TypeScript AI SDK for streaming chat, tool calling, agents, and multimodal apps across OpenAI, Anthropic, Gemini, React, Vue, Svelte, and Solid.

Open, lightweight multimodal decision models for text, images, and video. Train and run them yourself.

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

Build realtime multimodal AI agents with Node.js

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

LightlyStudio - The Unified Data Platform for Multimodal ML

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

AI powered open source recommender system engine supports classical/LLM rankers and multimodal content via embedding

A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

A curated list of medical LLMs, multimodal systems, datasets, benchmarks, and more. 🏥

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

🥂 Gracefully face hCaptcha challenge with multimodal large language model.

Open-source data management for multimodal AI. Query, trace, and govern with a lineage-native, format-agnostic lakehouse for agents and teams. Supports biological formats and registries - by the creators of Scanpy. 🍊YC S22

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

A collection of resources on applications of multi-modal learning in medical imaging.

A Toolbox for MultiModal Recommendation. Integrating 10+ Models...

On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

X-AnyLabeling: A lightweight, efficient, and unified cross-platform desktop application for annotating text, image, video, point cloud and multimodal data, combining versatile built-in tools with state-of-the-art AI models and flexible multi-format export.