Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Python12,347 stars811 forksApache-2.0 bentomlfine-tuningllamallama2llama3-1llama3-2llama3-2-visionllmllm-inferencellm-opsllm-servingllmopsmistralmlopsmodel-inferenceopen-source-llmopenllmvicuna