Have you tried

Outline?

I recently set it up and I’m very impressed.

This is not difficult to achieve at all with tools like sed or awk. But unless you provide a concrete example input file or files, all we can do is point to those tools.

Seems like the router doesn’t like how the headers are passed on. You could try:

login.router.lan {
    reverse_proxy 192.168.1.1:80 {
        header_up Host {upstream_hostport}
    }
}

https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#headers