What tools and projects do you use for your local LLM chats and agents?

I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.

  • nitrolife@hikki.teamEnglish
    2·
    6 days ago

    front: open-webui skill servers: dockling - docs parsing for llm, searxng - searchengine for llm, open-terminal - linux terminal for llm (inside container) base: llama-swap Inside llama swap: qwen3-14b-q5, translator-nemo-12b, tower-plus-9b, translator-gemma4-26b, qwen3.6-35b-a3b llama stack: ROCm

    Inside homeassistant: wyoming-faster-whisper and wyoming-piper for my Home Assistant Voice

    and some self-writed daemons for jnovels and translation.

    Hardware: RX9070XT for llama-swap . AMD epyc + 128 GB ram for skills and wyoming-faster-whisper and wyoming-piper… and SteamOS gaming, of course. =)

    UPD: if you want easy way and run all in oneshot use https://github.com/Niko1221/Strata