Migrating Hyper-V Virtual machines between clusters and forests

Word of Warning: Hyper-V and it’s components suck, this guide is sketchy at best as i’m writing it out of memory, and there’s probably something form the hours of debugging. If the guide is missing something crucial, please leave a comment so I can correct it. Verify and validate everything before running it in your production environment.

By jumping through all these hoops, I was able to live migrate VM’s (Linux and Windows) from one Hyper-V cluster managed by System Center Virtual Machine Manager (SCVMM) to another Hyper-V SCVMM Cluster, while neither being in the same Active Directory (AD) Forest, datacenter or network. A truly share nothing migration between Hyper-V clusters. Obviously you should have a good knowledge of what your about to do, do the research to avoid any mistakes.

This migration was completed successfully between Windows Server 2012 and 2012R2 Cluster with VMM 2012 management to a destination Windows Server 2016 Hyper-V Cluster with 2016 VMM management.

Pre requisits

So i’m going to assume that you have some Hyper-V, AD and Networking knowledge under your belt. So here are some things you need to make sure to configure first, to get stuff working;

1. A functioning Active Directory trust between yourdomain.tld and second.site. Validate that Domain Controllers (DC) from both forests are able to authenticate and make DNS queries.
2. Add your account as administrator on both site hyper-v hosts. The same account, for example administrator@yourdomain.tld
3. Validate that your hyper-v hosts can query the names of the other hyper-v hosts, for example vmhost.yourdomain.tld can query vmhost.second.site
4. Validate that your hyper-v hosts can contact each other, meaning that you can ping and enter a remote powershell session to each host.
5. Make sure your Hyper-V hosts user Kerberos Authentication (NOT CredSSP) as authentication protocol. This can be found in Hyper-V Settings -> Advanced Features.
6. If you use a firewall between the hyper-v hosts, make sure you have the appropriate ports open, or simply allow all traffic. The hosts do NOT need to be on the same subnet, as long as theres a route between them. You could even set up a pfSense gust VM on one of the hosts and add static routes to each host.
7. Make sure that your cluster/host can use the correct network for live migration. Using any available network option for Live Migration is a valid setting, but you can just choose the right one(s).

That’s quite the list, pretty basic stuff in the end but make sure you check off each one of them.

Migration of VMs

To migrate a VM between clusters and forests, you can’t use the Hyper-V GUI, and you can’t use VMM either. The only way to do it, is with powershell. GUI might seem to work at first, but it will fail with an error message that is false. Using the GUI will give you an error like:

The Virtual Machine Management Service failed to authenticate the connection for a Virtual Machine migration at the source host: The specified target is unknown or unreachable (0x80090303).
Event ID:21024

Before you migrate the VM, you must remove it’s cluster resource. Migrate the VM to the correct host you’ve tested everything on, then use Microsoft Failover Cluster Manager to remove the cluster role of the Virtual Machine (This won’t remove the VM, just the role). You can recreate the role if things fail and you need to make it highly available again. Any mounted medias (ISO or other) must be unmounted. It will migrate with the ISO file still mounted, but the host will go haywire at some point as the VMs mounted file can’t be found at the destination.

NOTE: Run powershell as Admin

$server = get-vm -name VMtoMigrateHere

Start off by saving your VM to migrate as the variable $server. You may also validate it with

if($server.count -eq 1 -and $server.GetType().name -eq "VirtualMachine")

We’re going to store the destination hyper-v server in the variable vmmHostName for later use. After that we’re going to query a list of Cluster Shared Volumes (CSV) as the target for our migration and select the first one as variable $destinationDisk.

$vmmHostName = "your.target.vmmhost"
$vmmstorage = getClusterStorage($vmmHostName)
$destinationDisk = $vmmstorage[0]

Once that is done, we need to start the migration by doing a compability check. This is done by using the Compare-Vm function, to find out if there are any issues in the configuration. Also this allows us to change any network settings of the VM, like changing your vlan of the virtual machine.

$DestinationFolder = $destinationDisk.FriendlyVolumeName+"\"+$server.Name
$myFQDN=(Get-WmiObject win32_computersystem).DNSHostName+"."+(Get-WmiObject win32_computersystem).Domain
write-host "Migrating VM " $server.name " to $DestinationFolder"
$moveVM = Compare-VM -Name $server.name -computername $myFQDN -DestinationHost $vmmHostName -includestorage -destinationstoragepath $DestinationFolder -ErrorAction Stop

$moveVM will contain the result, and we can make changes before actually initializing the migration. You can check for any incompabilities with the following command

if($moveVM.Incompatibilities.Count -gt 0) {
     write-host "The following errors were found during compare-vm check"
     foreach($check in $moveVM.Incompatibilities){
         write-host "##  " $check.message
     }

Network compability issues are most likely to occour, as the destination hyper-v cluster most likely doesn’t have the same name for the port groups. Network issues can be found with the following command

if ($moveVM.Incompatibilities | Where-Object {$_.MessageID -eq 33012}).count -gt 0) { do stuff here }

You can view the settings of adapters with the following commands, where $i is the number of the network adapter

write-host "Switch Name: " $server.NetworkAdapters[$i].SwitchName " >> " $moveVM.VM.NetworkAdapters[$i].SwitchName

write-host "OperationMode: " $moveVM.VM.NetworkAdapters[$i].VlanSetting.OperationMode

write-host "AccessVlanId: " $moveVM.VM.NetworkAdapters[$i].VlanSetting.AccessVlanId

You may change network adapter settings of the VM with the following example commands ($i being the adapter to modify)

$moveVM.VM.NetworkAdapters[$i] | Set-VMNetworkAdapterVlan -Access -VlanId $NewAccessVlanId

$moveVM.VM.NetworkAdapters[$i] | Set-VMNetworkAdapterVlan -Untagged

$moveVM.VM.NetworkAdapters[$i] | Connect-VMNetworkAdapter -Switchname $NewSwitchName 

Once everything is set, you can initiate the migration with the folloing command

move-vm -CompatibilityReport $moveVM -verbose

And there you go. You will get a nice little progress bar to show your VM is moving. You can have the VM running when you do this, and depending on your switch configuration, only a few packets are lost when the final handoff is done between the hyper-v hosts. Remember to re-apply any failover cluster roles or just use your VMM (Migrate VM) to make the virtual machine highly available again.