I've a VirtualBox VM which configured a very large hard disk size (bigger than host). By my mistake, a program on the VM generated lots of log files and the VDI file size keeps growing until there is no space on the host.

Now I've deleted the log files but the VDI file size are not getting smaller after using VBoxManage.exe modifyhd "C:\Virts\mybox-i386.vdi" compact

Is there a way to really compact the VDI file size? Thanks!

up vote 469 down vote accepted

You have to do the following steps:

  1. Run defrag in the guest (Windows only)
  2. Nullify free space:

    With a Linux Guest run this:

    dd if=/dev/zero of=/var/tmp/bigemptyfile bs=4096k ; rm /var/tmp/bigemptyfile
    

    Or:

    telinit 1
    mount -o remount,ro /dev/sda1
    zerofree -v /dev/sda1
    

    With a Windows Guest, download SDelete from Sysinternals and run this:

    sdelete.exe c: -z
    

    (replace C: with the drive letter of the VDI)

  3. Shutdown the guest VM

  4. Now run VBoxManage's modifymedium command with the --compact option:

    With a Linux Host run this:

    vboxmanage modifymedium --compact /path/to/thedisk.vdi
    

    With a Windows Host run this:

    VBoxManage.exe modifymedium --compact c:\path\to\thedisk.vdi
    

    With a Mac Host run this:

    VBoxManage modifymedium --compact /path/to/thedisk.vdi
    

This reduces the vdi size.

  • 13
    For the next person, my command ended up looking like this: "C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyhd "C:\Users\daniel\VirtualBox VMs\....\thedisk.vdi" --compact – Daniel Dec 13 '13 at 17:57
  • 35
    According to the zerofree Linux utility manpage (manpages.ubuntu.com/manpages/natty/man8/zerofree.8.html), zerofree should be better than dd for this job. dd would not been recommended because "it is slow", "it makes the disk image (temporarily) grow to its maximal extent", "it (temporarily) uses all free space on the disk, so other concurrent write actions may fail". Zerofree is available on Ubuntu Linux via apt, or you can compile it yourself. – Dakatine Feb 20 '14 at 13:16
  • 19
    It's fun that the the manpage of zerofree states that with dd other concurrent writes will fail, but zerofree needs the filesystem to be mounted read-only! *duh* – Madarco Feb 25 '14 at 12:04
  • 7
    Tip: Put the two commands on one line like so: dd ...; rm /bigfile, this will minimize the time with a full disk in case you're not waiting for the dd to complete. – jlh May 14 '14 at 17:45
  • 18
    @Dakatine Using VirtualBox 4.3.10, the disk image file did not grow to its maximal extent. VirtualBox is clever enough to not bother writing all zero blocks to the physical disk. – jlh May 14 '14 at 17:50

If running the command in the accepted answer produces an unhelpful error message like this

VBoxManage.exe: error: Cannot register the hard disk 'thedisk.vdi'
{aaaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeee} because a hard disk 'thedisk.vdi'
with UUID {aaaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeee} already exists

Simply run the command by the UUID instead of the filename:

VBoxManage.exe modifyhd {aaaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeee} --compact

I'm on a Windows 7 host with Windows guests, Here is a batch file I wrote to Compact all of the VDIs in a folder tree

echo off
mode con:cols=140 lines=200
cls
:: see https://forums.virtualbox.org/viewtopic.php?p=29272#p29272
:: How can I reduce the size of a dynamic VDI on disk?
:: but that page says to use sdelete -s which is suboptimal. 
:: use -z as per http://technet.microsoft.com/en-us/sysinternals/bb897443.aspx

:: First run the sdelete -z c: inside the VMs that zero-out all the free space
:: THEN run this batch file 

Title Compacting Free space on Virtual Machine VMs

:: http://ss64.com/nt/for_r.html
:: http://stackoverflow.com/questions/8836368/windows-batch-file-how-to-loop-through-files-in-a-directory/8836401#8836401

Setlocal EnableDelayedExpansion
:: http://ss64.com/nt/delayedexpansion.html ... 
:: Notice that within the for loop we use !variable! instead of %variable%.

For /R %CD% %%G IN (*.vdi) DO (
 set ohai=%%G
 set lastfive=!ohai:~-5!
:: Skip snapshots which are named {guid}.vdi
 if NOT !lastfive!==}.vdi (
 echo .
 echo Compacting %%G
 "C:\Program Files\Oracle\VirtualBox\VboxManage.exe" modifyhd "%%G" --compact )
 )

pause 
exit

I left the links in the comments so you can (sort of) tell how it works.

edit

Well, after all that, I tried the CloneVDI tool and it did a good job in much less time and in one click.

  • 5
    You would think on this sort of site there would be some kind of syntax highlighting for DOS but no. It looks much prettier in Notepad++ – CAD bloke Jan 9 '15 at 10:26
  • 1
    @CAD_bloke that would require a parsing engine and when you consider the number of different languages posted on SE you are looking at a HUGE project. Just think how many versions and dialects of DOS there are for example and that's before you even get to Linux etc. – Caltor Sep 17 '15 at 9:58
  • Very good point. Ironically it is highlighted on the stack exchange iOS app. – CAD bloke Sep 17 '15 at 10:32
  • 2
    Yeah CloneVDI is much better and faster way for personal use – VarunAgw Nov 14 '15 at 9:51

Debian guest on Windows host using discard/TRIM.

This isn't a direct answer per se, as I'm addressing the problem, not the question. Instead of periodically compacting the image, this solution uses discard to automatically remove unused blocks in the host's VM disk image.

This solution requires a guest filesystem that supports continuous TRIM. The Arch Linux wiki has a list of filesystems supporting TRIM operations.

FDE and cryptoroot are specifically not covered, as there are security concerns and none of the other solutions to this question would allow compacting either. The Arch Linux wiki has information about TRIM and dm-crypt devices.

In theory, this will work for all Linux guests on VBox hosts using VDI storage.

Host configuration

With VBox exited and no VMs running, add discard support to your disks by setting both discard and nonrotational for each disk in the config file for the VM. At this time discard is not in the GUI, but nonrotational is exposed as the "Solid-state Drive" checkbox. (ref: vbox forums, discard support)

<AttachedDevice discard="true" nonrotational="true" type="HardDisk" [..other options..] >

Boot the VM up, and verify that TRIM support is enabled:

sudo hdparm -I /dev/sda | grep TRIM

Guest Configuration

If LVM is in use, change the discard setting in /etc/lvm/lvm.conf. (ref: debian wiki, lvm.conf example)

devices {
...
    issue_discards = 1
}

In fstab, add the discard option to the filesystems you wish to auto-discard (ref: debian wiki, fstab example)

UUID=8db6787f-1e82-42d8-b39f-8b7491a0523c   /   ext4    discard,errors=remount-ro   0   1
UUID=70bfca92-8454-4777-9d87-a7face32b7e7   /build  ext4    discard,errors=remount-ro,noatime   0   1

Remount the filesystems to have them pick up their new options.

sudo mount -o remount /
sudo mount -o remount /build

Manually trim free blocks now with fstrim. fstrim uses the mounted filesystem, not the block device backing it. Instead of setting continuous discard in fstab, this could be done on a weekly cron. (The weekly cron is recommended for physical SSDs which may have questionable support for TRIM, but this is not relevant here since underlying SSDs are handled by the host OS. see: ssd trim warning).

fstrim /
fstrim /build

At this point, the size of the filesystems inside the VM and the size of the VM images should be pretty close in value.

Tested with:

  • Guest1: Debian 8.7, kernel: linux 4.8 grsec from backports, filesystem: ext4
  • Guest2: Debian 9 RC2, kernel: linux 4.9, filesystem: ext4
  • Host1: VBox 5.1.14, Win7, image fmt: VDI
  • Host2: VBox 5.1.14, Win8.1, image fmt: VDI

For MacOS Guest do this:

  1. Nullify free space in guest system:

    diskutil secureErase freespace 0 "/Volumes/Macintosh HD"
    

    (replace /Volumes/Macintosh HD with your drive name)

  2. Shutdown the guest VM

  3. Run this command to reduce VDI disk image size

    VBoxManage modifyhd /path/to/thedisk.vdi --compact
    

    OR

    VBoxManage modifymedium /path/to/thedisk.vdi --compact
    

I use this for my VDI image mounted to virtual Debian in Windows VirtualBox. It isn't a general solution, but it should at least give you a gist of what I do.

Commands in Debian:

root@debian:~# lsblk  # show partitions
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT 
sdb 8:16 0 128G 0 disk 
└─sdb1 8:17 0 128G 0 part /mnt/web  # THIS IS THE PARTITION OF INTEREST!
sda 8:0 0 64G 0 disk 
├─sda1 8:1 0 61,4G 0 part / 
├─sda2 8:2 0 1K 0 part 
└─sda5 8:5 0 2,7G 0 part 
[SWAP] sr0 11:0 1 56,3M 0 rom
root@debian:~# service mysql stop  # terminate all operations with partition
root@debian:~# service apache2 stop  # terminate all operations with partition
root@debian:~# umount /mnt/web  # unplug partition
root@debian:~# apt-get install zerofree  # install tool for filling in zeros to empty space
root@debian:~# zerofree -v /dev/sdb1  # fill with zeros
root@debian:~# poweroff  # shut down machine

Commands in Windows:

C:\Program Files\Oracle\VirtualBox>VBoxManage.exe modifyhd --compact "D:\VirtualBox VMs\web.vdi"  # convert zeros to empty space

Hope it helps :)

I don't want to enable TRIM support in OS, because every data deletion will force data compacting in VDI file, making guest system unusable when VDI file is on classic rotational disc. For me better is to perform compacting by hand e.g. once per month.

During normal compacting, VDI file content is copied to new file. This requires some (sometimes big) free space on host disc.

I've solution similar to pointed by Andrew Domaszek. It works very well even with NTFS (Windows10).

To do this:

  • create new virtual machine which boots with GParted Live CD (you can use your favorite Linux distro).
  • Edit machine settings and set SATA disc controller
  • Add existing VDI files which you want to compact
  • Change VDI based discs to be visible as SSD with TRIM support:

VBoxManage storageattach "gpared live" --storagectl "SATA" --port 0 --discard on --nonrotational on VBoxManage storageattach "gpared live" --storagectl "SATA" --port 1 --discard on --nonrotational on

  • start machine
  • In Linux root shell, mount NTFS partition mount /dev/sda2 /mnt
  • zero free space dd if=/dev/zero of=/mnt/bigfile
  • rm /mnt/bigfile
  • force compacting VDI without creating new file: fstrim -v /mnt

A very neat trick to supplement the accepted answer is that you can get away without doing any compacting at all after zeroing guest space, by using a compressed file system on the host (e.g. selecting to compress the folder of virtual drives on NTFS properties on a Windows host). This in fact has the benefit to save a lot more space because operating systems tend to hold a lot of repetitive text or binary files (e.g. a 30GB guest drive that had 15GB of space zeroed can turn to 4GB on the host drive).

Caveats include that drive access on the real hardware may increase and there is a slight increase in CPU usage.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.