Search

Heterogeneous GPU Sharing on Kubernetes

High-performance RL post-training infrastructure. Designed to achieve bitwise operator-level train-inference consistency across heterogeneous engines and extreme memory efficiency for GRPO, PPO, etc.

Specification for streaming massive heterogeneous 3D geospatial datasets :earth_americas:

HIP: C++ Heterogeneous-Compute Interface for Portability

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

Heterogeneous programming in Julia

Framework providing operating system abstractions and a range of shared networking and memory services for common modern heterogeneous platforms.

This is an open-source toolkit for Heterogeneous Graph Neural Network(OpenHGNN) based on DGL.

An Async SDR Runtime for Heterogeneous Architectures

Python actor framework for heterogeneous computing.

LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure

A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

A programmable Mixture-of-Models router for heterogeneous LLM inference

The developer-first platform for scaling complex Physical AI workloads across heterogeneous compute—unifying training GPUs, simulation clusters, and edge devices in a simple YAML

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

LLM speculative inference server for heterogeneous hardware & consumer GPUs

使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations