# LiteLLM: Free Open Source Gateway to Manage All Your LLM Providers Source: https://www.youtube.com/watch?v=mwP4sdp7gW0 Recap page: https://rapidrecap.app/video/mwP4sdp7gW0 Generated: 2026-02-27T09:03:46.72+00:00 --- ## Quick Overview The video demonstrates how to deploy and utilize LiteLLM, a free, open-source LLM gateway, on the Elestio platform, covering setup, model configuration with OpenAI and Google AI Studio, team management for access control, and usage tracking in the analytics dashboard. **Key Points:** - LiteLLM functions as a unified proxy server allowing developers to call over 100+ LLMs via a single OpenAI-compatible API. - The tutorial shows deploying LiteLLM via Elestio by selecting it from the AI/GPU services catalog, configuring service details, and selecting a plan (e.g., Netcup provider, Medium-2C-4G plan). - New models, such as Google's Gemini models, can be added by selecting the provider (e.g., Google AI Studio) and inputting credentials in the Model Management section. - The platform supports creating 'Teams' to manage access control, allowing admins to assign specific models (e.g., All Proxy Models or specific OpenAI models) and set budget/rate limits for team members. - Virtual keys, which can be assigned to specific teams, are created and used in the Playground to test connectivity and model responses across different providers like OpenAI and Gemini. - The Usage section provides detailed observability, including total spend ($0.0003 spent in the example), request metrics (4 successful requests, 172 total tokens), and spend breakdown by provider (Gemini and OpenAI). - The video concludes by showing how to manage member permissions within a team, controlling access to API endpoints like key generation and deletion. ![Screenshot at 00:06: The LiteLLM homepage diagram illustrates its function as an AI Gateway that provides developers with unified access, fallbacks, and speed tracking across 100+ LLMs using the OpenAI format.](https://ss.rapidrecap.app/screens/mwP4sdp7gW0/00-00-06.jpg) **Context:** This video serves as a tutorial demonstrating the deployment and practical use of LiteLLM, an open-source LLM gateway, hosted on the Elestio cloud platform. LiteLLM aims to simplify access to numerous large language models (LLMs) by providing a unified API interface, enabling features like cost tracking, fallbacks, and rate limiting across various providers like OpenAI and Google AI Studio. ## Detailed Analysis The video guides the user through setting up LiteLLM on the Elestio platform, emphasizing its role as a free, open-source gateway to manage diverse LLM providers via a single API. The initial steps involve selecting LiteLLM from the Elestio service catalog (00:06), configuring the deployment settings (provider Netcup, plan Medium-2C-4G at 00:58), and waiting for the service to deploy (01:15). Once deployed, the user accesses the LiteLLM dashboard (01:23) and proceeds to the Model Management section (01:57) to add new models. The tutorial adds an OpenAI model by selecting the provider and using 'All OpenAI Models (Wildcard)' (02:20), and then adds a Google AI Studio model, specifically 'gemini-3-flash-preview' (03:24), demonstrating how to input credentials and test the connection (02:56). The video then moves to Access Control, showing how to create a Team named 'Elestio' (03:44) with specific model access (All Proxy Models) and budget limits ($100 weekly reset) (04:17). Within the team settings, member roles (Admin/User) and specific API permissions can be configured (04:49). Finally, the user tests the setup in the Playground (05:54), using a Virtual Key (06:06) and switching the model to 'gemini-3-flash-preview' (06:31) to confirm functionality. The session concludes by viewing the Usage dashboard (07:08), which shows $0.0003 spent across 4 successful requests using both Gemini and OpenAI models, confirming that LiteLLM successfully routes and tracks usage across providers. ### LiteLLM Introduction - Free open-source proxy server - Provides unified API to call 100+ LLMs - Compatible with OpenAI format ### Deployment on Elestio - Select LiteLLM service - Choose provider (Netcup) and plan (Medium-2C-4G) - Deployment confirmed via email (01:16) ### Model Management - Navigate to Models + Endpoints (01:57) - Add Models by selecting Provider (OpenAI, Google AI Studio) - Input API Key and test connection (02:56) ### Team & Access Control - Create Team named 'Elestio' (03:40) - Assign model access (All Proxy Models) and set budget limits ($100 weekly reset) (04:17) - Configure individual Member Permissions for API endpoints (04:49) ### Testing and Observability - Use Virtual Key in Playground (05:54) - Test chat functionality with Gemini model (06:31) - Review Usage dashboard showing $0.0003 spend, 4 successful requests, and usage split between Gemini and OpenAI (07:14) ![Screenshot at 00:06: The LiteLLM homepage diagram illustrates its function as an AI Gateway that provides developers with unified access, fallbacks, and speed tracking across 100+ LLMs using the OpenAI format.](https://ss.rapidrecap.app/screens/mwP4sdp7gW0/00-00-06.jpg) ![Screenshot at 01:12: The service creation wizard shows configuration steps, including selecting the 'LiteLLM' service and the 'MEDIUM-2C-4G' plan from the Netcup provider.](https://ss.rapidrecap.app/screens/mwP4sdp7gW0/00-01-12.jpg) ![Screenshot at 02:22: The 'Add Model' screen shows the selection of 'OpenAI' as the provider and 'All OpenAI Models \(Wildcard\)' as the LiteLLM Model Name\(s\).](https://ss.rapidrecap.app/screens/mwP4sdp7gW0/00-02-22.jpg) ![Screenshot at 04:50: The Member Permissions tab displays granular control over API endpoints \(e.g., /key/generate, /key/update\) that team members can access.](https://ss.rapidrecap.app/screens/mwP4sdp7gW0/00-04-50.jpg) ![Screenshot at 07:14: The Usage dashboard displays key metrics, showing $0.0003 total spend, 4 successful requests, 172 total tokens, and usage breakdown by provider \(Gemini and OpenAI\).](https://ss.rapidrecap.app/screens/mwP4sdp7gW0/00-07-14.jpg)