There's real usability benefits too. I've collected some anecdotes from Reddit:

Rootless podman is my first choice for using containers now, it works fantastically well in my experience. It's so much nicer to have all my container related stuff like volumes, configs, the control socket, etc. in my home directory and standard user paths vs. scattered all over the system. Permission issues with bind mounts just totally disappear when you go rootless. It's so much easier and better than the root privileged daemon.

and,

If you are on Linux, there is the fantastic podman option "--userns keep-id" which will make sure the uid inside+the container is the same as your current user uid.+

and,

Yeah in my experience with rootless you don't need to worry about UID shenanigans anymore. Containers can do stuff as root (from their perspective at least) all they want but any files you bind mount into the container are still just owned/modified by your user account on the host system (not a root user bleeding through from the container).

finally,

The permissions (rwx) don't change, but the uid/gid is mapped. E.g. uid 0 is the running user outside the container, by uid 1 will be mapped to 100000 (configurable), and say 5000 inside the container is mapped to 105000. I don't remember the exact mapping but it works roughly like that.