Running on your hardware is always going to be less efficient than running in a data center. You can run small models on green data centers - best of both worlds (unless you need privacy then run local).
Ever since I switched to Cortecs I’m spending like 20 cents / month for my LLM usage cuz I mostly use small models and try using them responsibly.
Running on your hardware is always going to be less efficient than running in a data center. You can run small models on green data centers - best of both worlds (unless you need privacy then run local).
Ever since I switched to Cortecs I’m spending like 20 cents / month for my LLM usage cuz I mostly use small models and try using them responsibly.
That’s why you do it at work on a secured laptop you own that’s on their network. Modern problems. They pay electricity and all the jazzzzzzzz.