35
What's the biggest docker footgun you've experienced? (programming.dev)
submitted 1 year ago by ishanpage@programming.dev to c/devops@programming.dev
30 comments fedilink hide all child comments

I'll start:

When I was first learning to use Docker, I didn't realize that most tutorials that include a database don't configure the database to persist. Imagine my surprise when I couldn't figure out why the database kept getting wiped!

you are viewing a single comment's thread
view the rest of the comments
[-] object_Object@programming.dev 19 points 1 year ago* (last edited 1 year ago)

The biggest footgun I encounter every time I set up a raspberry pi or other linux host for a side project is forgetting that Docker doesn't do log rotation for containers' logs by default, which results in the service going down and seeing a sweat inducing ENOSPC error when you ssh in to check it out.

You can configure this by creating /etc/docker/daemon.json and either setting up log rotation with log-opts or using the local logging driver (it defaults to json) if you're not shipping container logs anywhere and just read the logs locally. The local driver compresses the logs and automatically does log rotation:

{
  "log-driver": "local",
  "log-opts": {
     "max-size": "10m",
     "max-file": "3"
  }
}
[-] sisyphean@programming.dev 7 points 1 year ago* (last edited 1 year ago)

TIL. Thank you! (Now I will ssh into all my VPSes and set this up!)

(cool username btw)

[-] vegetaaaaaaa@lemmy.world 7 points 1 year ago

I prefer this method:

{
  "log-driver": "syslog",
  "log-opts": {
    "tag": "docker.{{.Name}}"
  }
}

This way container logs are forwarded to /var/log/syslog, which already contains all other services logs, and has sane rotation rules by default (and it allows rsyslog to manage log forwarding/shipping if needed).

[-] object_Object@programming.dev 3 points 1 year ago

Thanks, good to know! I had no idea about the tags. Looks like there's a lot more variables available.

I just reread the docs on the log drivers - they mentioned that as of docker 20.x local logs now work with all drivers as it buffers the logs locally as well. I think this is probably why I hadn't explored the other drivers before - couldn't use docker-compose logs.

this post was submitted on 01 Jul 2023
35 points (100.0% liked)

DevOps

1677 readers
1 users here now

DevOps integrates and automates the work of software development (Dev) and IT operations (Ops) as a means for improving and shortening the systems development life cycle.

Rules:

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

founded 1 year ago
MODERATORS
Ategon@programming.dev
RandomDevOpsDude@programming.dev