Skip to main content

Raymii.org Logo (IEC resistor symbol)logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

ncdu - for troubleshooting diskspace and inode issues

Published: 29-10-2017 | Author: Remy van Elst | Text only version of this article


Table of Contents


In my box of sysadmin tools there are multiple gems I use for troubleshootingservers. Since I work at a cloud provider sometimes I have to fix servers thatare not mine. One of those tools is ncdu. It's a very usefull tool when aserver has a full disk, both full of used space or full of used inodes. Thisarticle covers ncdu and shows the process of finding the culprit when you're outof disk space or inodes.

If you like this article, consider sponsoring me by trying out a Digital OceanVPS. With this link you'll get $100 credit for 60 days). (referral link)

ncdu

On the homepage ncdu describes itself as:

Ncdu is a disk usage analyzer with an ncurses interface. It is designed tofind space hogs on a remote server where you don't have an entire graphicalsetup available, but it is a useful tool even on regular desktop systems. Ncduaims to be fast, simple and easy to use, and should be able to run in anyminimal POSIX-like environment with ncurses installed.

When the root disk is full, a server will behave strangely. Some daemons stillwork, others will work for a bit and some refuse to start at all. If you useext4 the server will boot up most of the time since there is a bit of reservedspace in the filesystem.

Space and inodes

A disk can be full because of too much space being used. You can check that withthe df -h command:

root@s1:~# df -hFilesystem      Size  Used Avail Use% Mounted on/dev/vda1        20G   19G     0 100% /

A disk can also be full because there are to many files and folders. You've ranout of inodes. df -i shows you that:

root@s1:~# df -iFilesystem      Inodes   IUsed  IFree IUse% Mounted on/dev/vda1      1305600 1305600      0  100% /

In this case one of my own servers has ran out of inodes. I know where they are,one of the users has no quota and has synced an entire 10 years of Maildir,which resulted in too many small files. If I wouldn't have known where thestorage was being used, ncdu would be my go to tool for finding out.

Yes you can use du and df. du takes a long time and it doesn't tell youwhere the inodes are used. A shell loop with find for finding inodes has thesame problem. You also can't easily navigate the results.

ncdu gives a graphical overview of the files and folders, sorted by most spaceused. In the lower right corner there is a counter for the number of files inthe current folder. Using that counter, you can navigate around the filesystemand see where the millions of inodes are used. Sorting by inode usage can alsobe done with the C key. No other tool I know of has that option.

ncdu also allows exporting the results. You can then see them on anothermachine. Usefull if you have a weird ARM nas with terminal display issues. ncduruns remotely without the TUI, scp over the results file and use the localterminal to find out what's wrong.

Installing ncdu

ncdu is available for almost every operating system imaginable and probably inyour package manager:

apt-get install ncduyum install ncdu

If you have trouble with a full disk or your package manager doesn't work, thedeveloper even has static binaries on the page:

Sometimes you can remove a few files to make space for this download, where thepackage manager doesn't work.

Using ncdu

ncdu

Start up ncdu. If you don't give any parameters, it will scan the currentfolder. If you give a filesystem path, it will scan that:

ncdu /

Will start a scan of the entire system. When the scan is finished, use the arrowkeys and ENTER to navigate around. Files and folders can be deleted right awayusing d. You can change the display from the bar to percentage or both usingthe g key. Sorting based on filesize with s, by name with n and by inodeswith C. That option is very usefull when your disk has run out of inodes.

ncdu is not real time. If you remove files outside of ncdu you need to run arescan.

Tags: articles, bash, disk, monitoring, ncdu