Ollama

A local runtime for large language models, packaging weights, templates and parameters so one command pulls a model and serves it with no cloud account. It exposes an HTTP API on a local port, speaks its own interface and the OpenAI-compatible one, and handles concurrency and GPU offloading so apps can share a resident model.

Developers use it for private chat, code assistance and prototyping against models that never leave the workstation, and it is a common backend for chat UIs and agents.

On your own hardware the prompts never go to a third party, you pay only for the GPU you already own, and a copy of the model directory rebuilds the setup. It can run alongside other AI services on your own infrastructure.