Open models are still most cheaply hosted on a cloud, with batching and 24/7 use. API rates from developer lab are generally fair. Self hosting does have some significant tangible benefits though: Fine tuning for domain specific to organization, and not letting LLM provider train from your prompts/answers, followed by competing with your organization in the future as a result of “distilling your IP”.
Open models are still most cheaply hosted on a cloud, with batching and 24/7 use. API rates from developer lab are generally fair. Self hosting does have some significant tangible benefits though: Fine tuning for domain specific to organization, and not letting LLM provider train from your prompts/answers, followed by competing with your organization in the future as a result of “distilling your IP”.