Sometimes happens a linux machine runs out of space, there are many reasons for it, external attacks, large uploaded files, misconfigured log files, ecc…Running out of space can sometimes lead to file corruption problems, so it’s a good idea addressing the problem as soon as possible. While on windows machines we have many graphic tools to find large files, in linux we have all we need inside our kernel, but as usual, as a good linux sysadmin you need some command line tool tips, or finding those large file to delete can be tricky.
First of all find which partition you need to inspect:
and move there positioning yourself on the root.
then take a look at your directories (and they subdir) size:
du -sk *|sort -n
You will have something like this:
folders are sorted from smaller to bigger.
Look at the folders size and try to guess where you need to clean some space, usually a sys admin knows his servers and can find out oversized directories at the first glance. We’ll move into /home for a deeper inspection:
let’s see the files:
if we can’t find anything wrong we can type
du -sk *|sort -n
again and follow the subfolders for a deeper inspection until we find our large file/files to be deleted, issuing
will delete the file
will delete a directory and its contenet, use the commands carefully.
Check with df again your partition usage until you can free out the needed space.
Sometimes there’s a big difference between what the “df” command says and what “du-sk” returns, the reason is “df” returns the disk allocation, while “du” the file dimensions. You can check if there are deleted files issuing this command:
lsof |grep '(deleted)'
finding deleted files still used by other programs when deleted. They still use space but do not show up in the du command. As soon as you restart the system or stop the program they leave.
PREVENT THE PROBLEM FROM HAPPENING
You can use a simple script to prevent your disk from going full:
you can find some more info about the script here:
set up a cronjob:
press the insert key, then add this line:
*/5 * * * * path_to_script
then press ESC key
and type :wq then return key to save and exit.
The script will run every 5 minutes checking space on your partitions and sending an email to “ADMIN” address once the space will go lower then the percentage in “ALERT”.
Remember to setup ALERT, ADMIN and eventually EXCLUDE_LIST in the script.