Solving Sophos Virtual Appliance Disk Space Issues

If you've built a few VMs for Sophos UTM (and Sophos SUM/ACC4), you may have come across a problem with the /var/storage partition filling up every few days. Constant emails with messages like "Data Disk is filling up - please check. Current usage: 86%" and "[INFO-154] Data Disk is filling up - please check" clog your inbox.

The culprit is usually the postgresql database, and while the situation can be temporarily resolved by running /etc/init.d/postgresql92 rebuild from the console, the effects of doing so can be undesirable. In the case of ACC/SUM4, rebuilding the postgres DB will remove all global objects; this is not good since it takes considerable time to create and deploy the global objects (although an import and deployment should be trivial, as name and data parameters are identical). Sophos has stated that there is no way to grow partitions after installation, and after having been bit by this issue a few times, I decided to confirm their statement.

The particular Sophos virtual appliance I am attempting to resize here is the SUM4/ACC. Let's see what is possible. Everything here should be done in the appliance console.

Step 1: Increase the size of the virtual appliances disk in the hypervisor and reboot the VM.

Step 2: Login, run cfdisk /dev/vda replacing vda with the virtualized disk in your installation. This can be attained by running 'df' and looking at the filesystem column. The numbers indicate the partition position; use only the device name.

Step 3: At the bottom of the list, you should see a 'free space' area on the disk. Whittle your way through cfdisk to create a new partition in this space of the 'linux' type, and then write the changes. Take note of the partition number in cfdisk, and exit.

Step 4: Run partprobe to reread partitions.

Step 5: Create a new ext4 partition on the partition that you created by running mkfs.ext4 /dev/vda9 (vda9 was the partition in my case, but use the partition number of yours from above).

Step 6. Run df, and take note of the partition that is causing problems. In my case, this /dev/vda5, and was mounted at /var/storage. 

Step 7. Close all processes that have open files on this partition. Run lsof | grep /var/storage but replace the /var/storage portion with the mount point of the partition you need more space on. In my case, this involved stopping postgresql (/etc/init.d/postgresql92 stop), and a few others. I couldn't find the init scripts for some of the processes, so I ended up running for i in `ls /etc/init.d`; do $i stop; done; and stopped everything. The system was still functional, however the odd process was still using file resources on /var/storage; specifically, exim and smtpd.bin. I'm not too concerned with the integrity of those services on the ACC/SUM4 appliance so I left them running.

Step 8.  Mount the new partition, but replace the new partition ID with yours.

sum:/root # mkdir /mnt/newstorage

sum:/root # mount /dev/vda9 /mnt/newstorage

Step 9. Copy from the old to the new partition: rsync -aAXv --exclude="proc/*" --exclude="dev/*" /var/storage/* /mnt/newstorage

Step 10. Edit /etc/fstab, and replace the partition for the mount to your new partition on the line that mounts the old partition. You can obtain the the UUID for the device using blkid. I ran blkid /var/sda9 >> /etc/fstab and corrected the new line in fstab.

Step 11. Reboot and cross fingers.

This worked for me. I didn't resize the partition,  I created a new one. I don't expect any more issues from this. I'd be interested in hearing if this works for UTM as well, so let me know if you try it. This didn't take more than 5 minutes. 

Your original partition is still there, and short of grabbing lvresize as a static binary (I can't determine the distro in use by Sophos, nor can I find a package manager), there isn't much you can do to reclaim that space. Since you had problems to begin with, your partition was probably only a handful of gigabytes to begin with.