Everybody’s doing it. I guess I need to do an AI, too. In my home, I have a few different tools that use generative AI and LLMs. I talk to my Home Assistant Voice Preview voice assistants which leverage a self-hosted Ollama running llama3.2. I use OpenWebUI, Tabby for as an experimental coding assistant. I use DeepInfra for larger models that don’t fit on my own GPU.
However, my problem is that each program supports different providers and models. Some support OpenAI style APIs to any provider, some only support Ollama APIs. If I wanted to forward my Home Assistant queries to DeepInfra, it wasn’t easy to do because there wasn’t an integration. If I wanted to change the model that Tabby uses between different models, I had to redeploy the service.