Today I wanted to install some software using "apt-get install", when it suddenly reported that there was no space left on a device. df -h reported me, that my root file system was almost full, so I looked what I could do and finally moved some big files to another partition. Although my system had then plenty enough free disk space, "apt-get" kept reporting low disk space. So I asked DuckDuckGo and Google for help and got it: Apparently even a file system with enough free space can be full - if it ran out of inodes! df -i then showed me, that almost 100% of all inodes were used. So I was curious what caused those troubles. On [1] I found a quite useful command which I even "pimped" a little bit:
$ for i in /*; do echo $i has `find $i | wc -l` files; done
which outputs something like
/bin has 153 files
/boot has 262 files
and so on.
Or use
$ for i in /*; do echo `find $i | wc -l` files in $i; done
if you would like to have the numbers first:
153 files in /bin
262 files in /boot
Piping the output to "sort -n" does sorting, but you have to wait for the command to complete:
$ for i in /var/*; do echo `find $i | wc -l` files in $i; done | sort -n
1 files in /var/crash
1 files in /var/local
...
600 files in /var/log
11066 files in /var/lib
Using that command, I found out that "/usr" has many files, so I replaced "/*" by "/usr/*" and figured out that /usr/src was the really bad directory on the file system. I didn't notice yet that removing a Linux kernel doesn't remove the according headers. This must be done manually. To see all currently installed header packages, it's possible to use
$ dpkg --list | grep linux-headers
Those packages have really huge numbers of files! For Linux 3.2.0-80, there are two directories in /usr/src: "linux-headers-3.2.0-80" and "linux-headers-3.2.0-80-generic" which together have 22042 (!) files! And I had 28 of those on my system...
Conclusion: After "apt-get remove linux-headers-3.2.0-24 linux-headers-3.2.0-25-generic linux-headers-3.2.0-26-generic..." I had 167098 free inodes, which should be safe for the upcoming weeks. In future, I will monitor this value to avoid troubles.
[1] http://www.ivankuznetsov.com/2010/02/no-space-left-on-device-running-out-of-inodes.html
