Kelser Blog, Media, and News | Connecticut IT Consulting Blog

vSphere 6.0 Critical CBT (Backup) Bug

Written by Kelser | December 7, 2015

There is a fairly serious bug, in vSphere 6.0, which can make backup data corrupt and prevent you from restoring data. This issue is specifically targeted at those (most) who use CBT (change block tracking) to take efficient image-based backups of their environment.  If you need any assistance with applying or testing these patches and/or the integrity of your backups, please give your account representative a call. You can also contact us here.

 

The VMware KB for this article is here:  http://kb.vmware.com/kb/2136854

There is a patch available.  After applying the patch, you need to reset CBT on your VMs, so you can ensure backups that may have had missing data are fully consistent (i.e. the damage has already been done).  This is of particular importance if you are using an incremental-forever approach and/or have backup software that doesn't invoke a CBT reset during a full backup.  I recommend using PowerCLI to reset CBT because you don't have to shut the VM down to do it.  Many backup packages will re-enable CBT automatically.

Basic process for the reset/disable with PowerCLI (use at your own risk!): 

$vms = Get-VM
$cbtspec = New-Object VMware.Vim.VirtualMachineConfigSpec;$spec.ChangeTrackingEnabled = $false;
foreach($vm in $vms) {
   $vm.ExtensionData.ReconfigVM($cbtspec)
   $ss = $vm | New-Snapshot -Name 'CBT_Reset'
   $ss | Remove-Snapshot -confirm:$false;
   } 

 

Some other KB's for popular backup software packages: 

Unitrends:  http://support.unitrends.com/ikm/questions.php?questionid=1892

Veeam:  http://www.veeam.com/kb2075

Symantec/Veritas:  https://www.veritas.com/support/en_US/article.000097972

 

** Important Note for HP Gen9 Customers *

*If you're using an HP Gen9 server, please pay particular attention to another bug which MUST be reviewed prior to applying the CBT (or any other patches).  If you have a several servers to patch, I recommend using ether Update Manager to stop the application of the hpsa patch and/or VMware's PowerCLI w. Image Builder to create a customized image profile that won't touch the hpsa driver. 

VMware KB for Gen9 issue:  http://kb.vmware.com/kb/2120539