So first off, yeah their nginx is not the greatest example. Most of it is fine but check out their ansible example

So first off, I don't see HTTPS. Federation will not work without HTTPS, that's in the troubleshooting guide. You'll need to get https up and running and have a valid cert. Certbot is very easy to get up and running, I followed this guide

I don't know if it matters, but add all of the extra security items from the example guide around HTTPS, I believe Lemmy does depend on some of the extra parameters like ssl_ciphers and protocols to make sure requests are using the correct ones.

One key nugget of information is that in your http block you should add resolver 127.0.0.11 ipv6=off;, which tells nginx to use 127.0.0.11 as it's DNS, which is imperative if you are using docker host names. That IP is Docker's internal DNS, so things like http://lemmy and http://lemmy-ui work.

server_name should be the external tld. For example, mine is poptalk.scrubbles.tech.

My entire proxypass for / is as follows, I think yours should work, but this is known working:

     location / {

          # The default ports:
          # lemmy_ui_port: 1235
          # lemmy_port: 8536

            set $proxpass "http://lemmy-ui:1234";
            if ($http_accept ~ "^application/.*$") {
               set $proxpass "http://lemmy:8536";
            }
            if ($request_method = POST) {
               set $proxpass "http://lemmy:8536";
            }
            proxy_pass $proxpass;
            rewrite ^(.+)/+$ $1 permanent;
            proxy_http_version 1.1;
	    proxy_set_header  X-Real-IP $remote_addr;
            proxy_set_header  Host $host;
            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
	    proxy_set_header  Upgrade $http_upgrade;
            proxy_set_header  Connection $connection_upgrade;
        }

Make sure you're also redirecting pictshare (which I believe is deprecated, but if you have issues with pictures:

        location ~ /pictshare/(.*)$ {
          return 301 /pictrs/image/$1;
        }