Today I spent another half-hour curl-ing the latest Mistral release while colleagues in Slack bragged about integrating yet another API in 5 minutes. It got me thinking—why do I still bother?
Reason #1: Control
When the model lives on your machine, you know exactly what it's doing. No surprise TOS updates like those cloud providers pushed last week. A GitHub repo is all the documentation I'll ever need.
Reason #2: Performance
Sure, setting up torch + CUDA sometimes feels like black magic. But once it works? Three times lower latency than their "optimized" endpoints. Verified with Llama 2 13B using LoRA adapters.
"But clouds scale!" my product manager says. Then I show them the egress traffic bill.Anyone else still downloading models manually? Or am I the last dinosaur in the API age?