Why disk usage tools are slow on macOS

A disk usage tool never reads your files. Sizes live in the file system's records, so the whole job is listing folders and asking how big each entry is. How a tool asks, and how many threads ask at once, turns out to be almost everything.

The benchmark

szt against dua, dust, diskus, gdu, du and ncdu, each with its default settings, on a Mac Studio (M2 Ultra). How it was run is at the end.

One call per folder, not one per file

Most tools list a folder's names with readdir and then make a separate call (stat, lstat or fstatat) for every file to learn its size. macOS has a better way: getattrlistbulk returns names, types and sizes for hundreds of entries at once. szt lists every folder that way, so a file never gets a system call of its own.

The per-file calls add up: in a profile of du on the projects folder, 60% of its CPU time goes to fstatat. The system calls each tool's binary imports show who does what:

dua is the other tool built on getattrlistbulk, which is why it's the close one in the benchmark.

More threads made it slower

With the per-file calls gone, what's left is opening each folder before listing it, and that has a ceiling. Here's szt on the projects folder with different thread counts:

A single thread spends much of its time waiting rather than working, so the first few threads help a lot. Up to eight the scan keeps getting faster. Past that it gets slower while the kernel works harder and harder: 24 threads burn 200 seconds of CPU time to finish in 10.6. A profile at 24 threads shows why: 93% of the time is spent inside open, the call that opens a folder so it can be listed. Opening a folder goes through vnode and name-cache bookkeeping that every process on the Mac shares (on this machine the vnode table was full, 263,168 entries out of 263,168, so an open often has to evict an old entry first). Past a handful of threads, they mostly wait for each other.

So szt stops at eight threads. dua's authors arrived at the same number: its thread count defaults to 8 on macOS and to the number of cores elsewhere. Most other tools start a thread per core or more, which on this 24-core Mac means 24 threads for dust and gdu and 72 for diskus. It shows in their CPU time: 122 to 211 seconds for the projects folder, against szt's 12.

Tuned, the per-file calls still cost about double

To tell the two effects apart, here are the other tools at their best thread count on the same Mac:

With threads out of the picture they still take 1.8 to 2.4 times as long, and that's the price of asking about every file separately.

Efficiency cores don't help

Apple silicon mixes fast performance cores with slower efficiency cores. macOS runs background-priority threads on efficiency cores only, which makes it possible to try different mixes on one machine:

Four efficiency threads on top of four performance threads barely help (3.09 s against 3.17 s), and more threads than cores is a little slower. So szt starts one thread per performance core, as macOS reports in hw.perflevel0.physicalcpu, up to eight: four on a base M-series chip, and up to eight on the Pro, Max and Ultra.

How the benchmark was run

Try szt

szt is free and open source, for macOS 15 and later. Run it in any folder to get a tree of where the space went.