I'm sorry but if you aren't using automated renewals then you are not using let's encrypt the way it's intended to be used. You should take this as an opportunity to get that set up.
Ours is automated, but we incur downtime on the renewal because our org forbids plain http so we have to do TLS-ALPN-01. It is a short downtime. I wish let's encrypt would just allow http challenges over https while skipping the cert validation. It's nuts that we have to meaningfully reply over 80...
Though I also think it's nuts that we aren't allowed to even send a redirect over 80...
our org forbids plain http
is redirecting http to https also out of the question? because let's encrypt HTTP-01 accepts http -> https redirects:
Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates.
They in fact refuse to even do a redirect... it's monumentally stupid and I've repeatedly complained, but 'security' team says port 80 doing anything but dropping the packet or connection refused is bad...
Forgive my ignorance but why would that incur a downtime?
The only way I can think of for downtime to happen if you switched certs before the new one was signed (in which case ..don't) or am I missing something?
It also strikes me as weird that LE requires 80 but does allow insecure 443 after a redirect. Why not just do/allow insecure 443 in the first place?
the TLS-ALPN-01 challenge requires a https server that implements generating a self-signed certificate on demand in response to a specific request. So we have to shut down our usual traffic forwarder and let an ACME implementation control the port for a minute or so. It's not a long downtime, but irritatingly awkward to do and can disrupt some traffic on our site that has clients from every timezone so there's no universal '3 in the morning' time, and even then our service is used as part of other clients '3 in the morning' maintenance windows... Folks can generally take a blip in the provider but don't like that we generate a blip in those logs if they connect at just the wrong minute in a month...
As to why not support going straight to 443, don't know why not. I know they did TLS-ALPN-01 to keep it purely as TLS extensions to stay out of the URL space of services which had value to some that liked being able to fully handle it in TLS termination which frequently is nothing but a reverse proxy and so in principle has no business messing with payload like HTTP-01 requires. However for nginx at least this is awkward as nginx doesn't support it.
Thanks for the explanation!
Though it ought to be possible to only respond with the new self-signed cert when LE does the challenge and with the previous, properly signed cert otherwise.
I found https://codeberg.org/neilpang/acme.sh/wiki/TLS-ALPN-without-downtime which demonstrates one method to achieve that but I lack practical experience judge whether that's optimal.
I've got it setup automated on all my external domains, but trying to automate it on my internal-only domain is rather tedious since not only do I NOT want to open a port for it to confirm, but I have 2 other devices/services on the network not behind my primary reverse proxy that share the same cert.
What In need to do is setup my own custom cron that hits the hosting provider to update the DNS txt entries. Then I need to have it write and restart the services that use the cert. I've tried to automate this once before and it did not go so smoothly so I've been hesitant on wasting time to try it again... But maybe it's time to.
What would be ideal is if I could allow it to be automated just by getting a one time dns approval and storing a local private/public key to prove to them that I'm the owner of the domain or something. Not aware of this being possible though.
Depends on which DNS service you are using, a plugin might already exist that would do it for you. e.g. I use cloudflare for DNS and certbot is able to automatically set the txt record.
Technically my renews aren't automated. I have a nightly cronjob that should renew certificates and restart services, but when the certificates need renewal, it always fails because it wants to open a port I'm already using in order to answer the challenge.
I hear there's an apache module / configuration I can use, but I never got around to setting it up. So, when the cron job fails, I get an email and go run a script that stops apache, renews certs, and restarts services (including apache). I will be a bit annoying to have to do that more often, but maybe it'll help motivate me to configure apache (or whatever) correctly.
Debian Stable
You could try using the DNS challenge instead; I find it a lot more convenient as not all my services are exposed.
Shit like this is why building websites is no longer fun for me like it was back in the 90s and 2000s. There's way too much security shit to worry about now.
"You can set up your own email server at home, for fun!"
-- The 90's, Probably.
Lol. I'm kinda sad I missed out on that expressive time of making websites when I was growing up. You're right, now everything is very homogenized and there's a billion botswarms just waiting for you to be 3 seconds late to a security update so they can zombify your site for...
(Flips papers) Crypto somehow... it's always crypto.
Internet crime isn't even cool anymore. Lol
I still think the web would have been better off if certificates were signed and part of a web of trust like in GPG/PGP. It wouldn't stop sites from using trusted CAs to increase their trust levels with browsers, but it would mean that tiny websites wouldn't need to go through layers of mandatory bullshit and inconvenience. Also means that key signers could have meaningful business relationships rather than being some random CA that nobody has a clue about.
minor panic, oh, "2028".
Might as well adjust the setting now. I had that same feeling for something they changed several years ago and never got around to changing it til all my stuff went down lol.
YES! Keep cutting it down!
Revocation is a lost cause and if you don’t automate you deserve what you get.
I have multiple self hosted services at home which are impossible to automate because they are not accessible from the internet without VPN. And some even don't have internet access. Still me and my roommates are using them through a valid domain that points to the local address enabling https. Some services require https to function at all. After log4j i'll never again open a "normal" port 80 or 443 to my local net. So thanks i guess. 90 days was annoying already. Great it works out for you
Use DNS validation
The solution is to not use Http based validation of the cert, but use dns based validation. Possibly combined with a wildcard cert for your whole domain. This is what I do for internal services on my LAN, along with split DNS so that the internal services aren't even listed in public DNS.
Dude, you need to figure out a way to automate that. It’s no way to live.
I agree, but it's impossible to convince my less tech savy roommates and friends to let me install a root certificate. "That sounds like i could read all their private messages", lol. Just let me have my certificate for https in my local net. I don't need to be "even more" secure. I get that that's necessary for public services, but surely not for local selfhosting. I don't even have a port open other than wireguard. And i would not even care "if a roommate hacks/gets access to a guests voice commands for home assistant." (Not complaining at you but at this trend. I do think my use case is valid)
You are gonna laugh if i tell you how i partly automated this workaround. A script changes the (dyn) dns entries of all subdomains to point to my public server in a datacenter. There, it ssh's in and requests the certificates with certbot. Then, it restores the dns entries and downloads and installs the certificates in the local net. Still requires manual supervision and sometimes intervention. My domains do not support automated dnssec. I don't have time to secure my local net enough to feel good about opening ports. If all certificate lifetimes get shorter, i'll either have to switch my domain provider or give up selfhosting for other people.
Can you not issue your own certificate? I guess it depends on how many devices and what types of devices need to connect. It's be a one time effort per device (importing your own self signed cert) versus one time effort per service per X days.
I did that for myself a few years back. But i can't convince my roommates, let's not even speak of guests, to install a (my) root certificate. My android phone still complains about "possibly supervised network traffic" since back when i installed my root ca. Maybe there is another solution im not aware of, but i can't think of any
It's so infuriating to me that there isn't a way to just encrypt traffic without verifying it's part of a chain. By all means, give a nag warning in browsers, but ugh, I think that ship has long since sailed. Plus, realistically, you'd need just as many scary warnings to deter the average user that they might be getting MITMed.
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
-
No low-effort posts. This is subjective and will largely be determined by the community member reports.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!