ulimit and systemd LimitNOFILE — why ulimit -n inside a unit doesn't stick

What is nofile and where it lives

nofile is the maximum number of open file descriptors per process. That’s not just regular files — it covers sockets, pipes, stdin/stdout/stderr, logs shipped through journald — everything counts. When nginx or a Go app crashes with too many open files, this is the limit to blame.

Limits live at three levels:

LimitNOFILE in a systemd unit

In a unit file the directive looks like this:

[Service]
LimitNOFILE=65536

You can set both soft and hard limits at once, separated by a space:

LimitNOFILE=65536:1048576

The first value is the soft limit, the second is the hard limit. If you specify only one, it becomes the soft limit and the hard limit is taken from the system maximum.

Why ulimit -n in ExecStart doesn’t work

The typical mistake is trying to set the limit directly in the launch command:

[Service]
ExecStart=/bin/sh -c 'ulimit -n 65536 && exec /usr/bin/myapp'

ulimit -n inside ExecStart does not take effect on the process systemd tracks as MainPID. systemd sets limits before ExecStart runs, and the shell wrapper already operates in a context where the limit is fixed. Worse, ulimit -n may fail with Operation not permitted if the requested value exceeds the hard limit systemd assigned.

If you need to change the limit, use the [Service] directive — not a shell wrapper.

How to check applied limits

Three ways, from simple to authoritative:

# 1. From inside the process
cat /proc/self/limits | grep "Max open files"

# 2. For a specific PID
cat /proc/<pid>/limits | grep "Max open files"

# 3. What systemd sees for the unit
systemctl show myservice.service -p LimitNOFILE

systemctl show reports exactly what systemd applied at fork() — that’s the authoritative source. /proc/<pid>/limits is what the kernel sees for the process. If they disagree, the problem is in an intermediate layer (PAM, container runtime, sudo).

For debugging ExecStart, you can add:

[Service]
ExecStartPre=/bin/sh -c 'cat /proc/self/limits | grep "Max open files"'

This shows the limits before the main process starts — what systemd set for the service.

PAM limits vs systemd limits

On most modern distros (RHEL 8+, Ubuntu 20.04+, Debian 11+) systemd does not invoke pam_limits.so for system services. Limits from /etc/security/limits.conf are not applied to unit files. They only work for login sessions (SSH, local login, su).

If a service isn’t started through systemd (e.g., via supervisor, docker --ulimit, or directly), LimitNOFILE in the unit file doesn’t apply at all. In Docker that’s --ulimit nofile=65536:1048576, in supervisor the stdout_maxbytes and similar options aren’t related to nofile directly — you configure it at the OS or container level.