hyper-v

#windows #hyper-v #virtualization #debian #preseed #ansible #kubernetes

Table of Content

[[Hyper-V]]

https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/about/

What I want to do here is to use Hyper-V to run a few Debian servers and build another test Kubernetes cluster to play with.

[x] Enable Hyper-V on Windows 11 Pro
[x] Add current Windows user to Hyper-V admin group
[x] Create external virtual switch so that the VMs on Hyper-V are on the same LAN network
[x] Configure client DNS settings to fix network issue including host machine
[x] Prepare preseed.cfg file for automated Debian installation
[x] Download latest stable Debian netinst ISO image
[x] Run a simple http server to host preseed files
[x] Create Hyper-V VM (more than 3)
[x] Connect and start Debian installation
[x] Create post-installation checkpoint

enable hyper-v¶

Open powershell as admin and run below. The system will ask to reboot the machine.

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All

set permission¶

Using powershell hyper-v module requires privilege. Open powershell again as admin and run below to add the current user as hyper-v admin user.

Opening new powershell terminal didn't do any good, and I had to reboot my machine again to enable this privilege settings.

Add-LocalGroupMember -Group 'Hyper-V Administrators' -Member $env:USERNAME
Get-LocalGroupMember -Group 'Hyper-V Administrators'

powershell commands¶

https://learn.microsoft.com/en-us/powershell/module/hyper-v/?view=windowsserver2022-ps&preserve-view=true

# to display available commands
Get-Command -Module hyper-v | Out-GridView

Starting and stopping VM.

# start stopped VMs
Get-VM | where {$_.State -eq 'Off'} | Start-VM

# stop running VMs
Get-VM | where {$_.State -eq 'Running'} | Stop-VM

alias¶

$PROFILE

# alias - hyper-v
function StartVMAll { Get-VM | where {$_.State -eq 'Off'} | Start-VM }
Set-Alias -Name Start-VM-All -Value StartVMAll
function StopVMAll { Get-VM | where {$_.State -eq 'Running'} | Stop-VM }
Set-Alias -Name Stop-VM-All -Value StopVMAll

virtual switch¶

https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/connect-to-network

Hyper-V has three types of virtual switches -- external, internal, and private. Create an external switch to share your computer's network with the virtual machines running on it.

By the way, I renamed the network adapter to "eth0" by running following on admin powershell.

# current network adapter name in "-Name", which you can confirm by Get-NetAdapter
Rename-NetAdapter -Name "イーサネット 2" -NewName eth0

One warning before proceeding. Creating an external virtual switch immediately messed up my host machine networking, causing DNS to not to work. Confirm which DNS servers you want to use before executing New-VMSwitch and prepare Set-DnsClientServerAddress command except the interface index of the created vNIC.

# confirm client DNS settings
Get-DnsClientServerAddress

# Confirm the current available net adapter on host machine
Get-NetAdapter

# assuming the host machine has only one net adapter
# otherwise, specify the name of the net adapter to use in "-Name"
$net = Get-NetAdapter -Name $(Get-NetAdapter).name

# create a new virtual switch
New-VMSwitch -Name "External VM Switch" -AllowManagementOS $True -NetAdapterName $net.Name

# remove if you need to rollback
remove-vmswitch -name 'External VM Switch'

# confirm the interface index of the created virtual switch
Get-NetAdapter
Get-NetIPAddress -InterfaceAlias "vEthernet (External VM Switch)"

# confirm client DNS settings for the external virtual switch
Get-DnsClientServerAddress

# set client DNS settings, specifing the correct interface index
# in my case it was 30
Set-DnsClientServerAddress -InterfaceIndex 30 -ServerAddresses ("192.168.1.55","192.168.1.58")

Here is what I had from the beginning. I have WSL2 Ubuntu running and that should be why there is this VM switch named WSL.

╰─❯ Get-VMSwitch

Name                   SwitchType NetAdapterInterfaceDescription
----                   ---------- ------------------------------
Default Switch         Internal
WSL (Hyper-V firewall) Internal


╰─❯ Get-NetIPAddress -InterfaceAlias "vEthernet (Default Switch)"


IPAddress         : fe80::f4f7:1bd3:1d9e:50fb%19
InterfaceIndex    : 19
InterfaceAlias    : vEthernet (Default Switch)
AddressFamily     : IPv6
Type              : Unicast
PrefixLength      : 64
PrefixOrigin      : WellKnown
SuffixOrigin      : Link
AddressState      : Preferred
ValidLifetime     :
PreferredLifetime :
SkipAsSource      : False
PolicyStore       : ActiveStore

IPAddress         : 172.21.176.1
InterfaceIndex    : 19
InterfaceAlias    : vEthernet (Default Switch)
AddressFamily     : IPv4
Type              : Unicast
PrefixLength      : 20
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Preferred
ValidLifetime     :
PreferredLifetime :
SkipAsSource      : False
PolicyStore       : ActiveStore



╰─❯ Get-NetIPAddress -InterfaceAlias "vEthernet (WSL (Hyper-V firewall))"


IPAddress         : fe80::2d07:67ce:330d:9fe2%27
InterfaceIndex    : 27
InterfaceAlias    : vEthernet (WSL (Hyper-V firewall))
AddressFamily     : IPv6
Type              : Unicast
PrefixLength      : 64
PrefixOrigin      : WellKnown
SuffixOrigin      : Link
AddressState      : Preferred
ValidLifetime     :
PreferredLifetime :
SkipAsSource      : False
PolicyStore       : ActiveStore

IPAddress         : 172.25.176.1
InterfaceIndex    : 27
InterfaceAlias    : vEthernet (WSL (Hyper-V firewall))
AddressFamily     : IPv4
Type              : Unicast
PrefixLength      : 20
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Preferred
ValidLifetime     :
PreferredLifetime :
SkipAsSource      : False
PolicyStore       : ActiveStore

After the virtual switch creation.

╰─❯ Get-VMSwitch

Name                   SwitchType NetAdapterInterfaceDescription
----                   ---------- ------------------------------
Default Switch         Internal
WSL (Hyper-V firewall) Internal
External VM Switch     External   Intel(R) Ethernet Connection (12) I219-V


╰─❯ Get-NetIPAddress -InterfaceAlias "vEthernet (External VM Switch)"


IPAddress         : fe80::e760:51a1:78fe:ba7b%30
InterfaceIndex    : 30
InterfaceAlias    : vEthernet (External VM Switch)
AddressFamily     : IPv6
Type              : Unicast
PrefixLength      : 64
PrefixOrigin      : WellKnown
SuffixOrigin      : Link
AddressState      : Preferred
ValidLifetime     :
PreferredLifetime :
SkipAsSource      : False
PolicyStore       : ActiveStore

IPAddress         : 192.168.1.244
InterfaceIndex    : 30
InterfaceAlias    : vEthernet (External VM Switch)
AddressFamily     : IPv4
Type              : Unicast
PrefixLength      : 24
PrefixOrigin      : Manual
SuffixOrigin      : Manual
AddressState      : Preferred
ValidLifetime     :
PreferredLifetime :
SkipAsSource      : False
PolicyStore       : ActiveStore

gen1/gen2¶

https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/plan/should-i-create-a-generation-1-or-2-virtual-machine-in-hyper-v

Debian installation image¶

[[Debian]] on [[Hyper-V]]

https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/Supported-Debian-virtual-machines-on-Hyper-V

https://wiki.debian.org/WindowsServerHyperV

Download installation image file from the official site.

https://www.debian.org/CD/netinst/index.en.html

automated installation¶

https://www.debian.org/releases/stable/amd64/apb.en.html

Example preseed file is available here.

https://www.debian.org/releases/bookworm/example-preseed.txt

prepare a preseed file
host it on http server
create and start vm
on debian installation menu, type c to enter grub command line mode and run linux, initrd, and boot command

To compose a preseed file, download the example-preseed.txt and copy it for different machines.

As for the root and/or user password, use mkpasswd -m sha-512 {password string} to generate hashed password and use it in the preseed file. mkpasswd came with whois as described in the debian installation guide.

The following command (available from the whois package) can be used to generate a SHA-512 based crypt(3) hash for a password

To host the files, go to that directory and run python -m http.server 8080 to run and listen on port 8080.

grub command line

linux /install.amd/vmlinuz auto=true preseed/url={IPADDR of the server hosting the preseed file}:8080/{preseed file name} priority=critical ---
initrd /install.amd/initrd.gz
boot

directory on host machine¶

I wanted to use my disk space in F drive so here it is.

set-location f:\
mkdir hyperv

confirm services running¶

get-service -Name "vm*"

Start using hyper-v¶

https://learn.microsoft.com/en-us/powershell/module/hyper-v/new-vm?view=windowsserver2022-ps

Below is the powershell commands to use to create VMs.

I created VMs named vcp, vworker1, vworker2, and vworker3, completed the initial Debian installation, and created snapshot named PostInstallation.

I was thinking to create a Kubernetes cluster with these VMs. "vcp" is going to be the control plane and as per the Kubernetes requirement, I set 2 CPUs and minimum start memory byte of 2Gi. Without specifying minimum of 2Gi memory, hyper-v will adjust and lowers the memory size, then the Kubernetes kubeadm init will fail to setup a cluster.

$VMName = "vcp"
$VMName = "vworker1"
$VMName = "vworker2"
$VMName = "vworker3"
$VMName = "vworker4"
$VMName = "vworker5"
$Switch = 'External VM Switch'
$BasePath = "F:\hyperv"
$VMPath = "$BasePath\vm"
$VHDPath = "$BasePath\vhd"
$InstallMedia = "F:\hyperv\install_media\debian-12.4.0-amd64-netinst.iso"

$VM = @{
    Name = $VMName
    MemoryStartupBytes = 2GB
    Generation = 2
    NewVHDPath = "$VHDPath\$VMName.vhdx"
    NewVHDSizeBytes = 50GB
    Path = "$VMPath\$VMName"
    SwitchName = $Switch
}

New-VM @VM

# Set 2 CPU processors
Set-VMProcessor $VMName -Count 2

# Set 2GB memory for control plane, vcp
Set-VMMemory -VMName vcp -MinimumBytes 2147483648 -MaximumBytes 8589934592
# Set 1GB min memory for worker nodes
Set-VMMemory -VMName vworker1 -MinimumBytes 1073741824 -MaximumBytes 4294967296
Set-VMMemory -VMName vworker2 -MinimumBytes 1073741824 -MaximumBytes 4294967296
Set-VMMemory -VMName vworker3 -MinimumBytes 1073741824 -MaximumBytes 4294967296
Set-VMMemory -VMName vworker4 -MinimumBytes 1073741824 -MaximumBytes 4294967296
Set-VMMemory -VMName vworker5 -MinimumBytes 1073741824 -MaximumBytes 4294967296

# Change secure boot template to MS UEFI CA
#Set-VMFirmware -VMName $VMName -SecureBootTemplate MicrosoftUEFICertificateAuthority
Set-VMFirmware -VMName $VMName -SecureBootTemplateId "272e7447-90a4-4563-a4b9-8e4ab00526ce"

# Add DVD Drive to Virtual Machine
Add-VMScsiController -VMName $VMName
Add-VMDvdDrive -VMName $VMName -ControllerNumber 1 -ControllerLocation 0 -Path $InstallMedia

# Mount Installation Media
$DVDDrive = Get-VMDvdDrive -VMName $VMName

# Configure Virtual Machine to Boot from DVD
Set-VMFirmware -VMName $VMName -FirstBootDevice $DVDDrive

# Set "production" checkpoint mode
Set-VM -Name $VMName -CheckpointType Production

# Connect/Start the VM on Hyper-V Manager
# to start automated installation, type c to enter grub command line and
# run commands mentioned earlier

# create checkpoint post successful OS installation
Checkpoint-VM -Name $VMName -SnapshotName PostInstallation

# create checkpoint kube-ready
Checkpoint-VM -Name $VMName -SnapshotName KubeReady

# restore checkpoint
Restore-VMCheckpoint -Name PostInstallation -VMName $VMName -Confirm:$false

### clean up

# stop and remove the VM
Stop-VM -Name $VMName
# Stop-VM -Name $VMName -Force
Remove-VM -Name $VMName

# look for vmwp with username of the vm id, kill that process if stop-vm doesn't work

# clean up
set-location $BasePath
Remove-Item "vhd/$VMName.vhdx"
Remove-Item "vm/$MVName"

adding additional disk¶

# create virtual hard disk
New-VHD -Path F:\hyperv\vhd\directpv.vhd -SizeBytes 300GB

# create vm vworker5
$VMName = "vworker5"
$Switch = 'External VM Switch'
$BasePath = "F:\hyperv"
$VMPath = "$BasePath\vm"
$VHDPath = "$BasePath\vhd"
$InstallMedia = "F:\hyperv\install_media\debian-12.4.0-amd64-netinst.iso"

$VM = @{
    Name = $VMName
    MemoryStartupBytes = 2GB
    Generation = 2
    NewVHDPath = "$VHDPath\$VMName.vhdx"
    NewVHDSizeBytes = 50GB
    Path = "$VMPath\$VMName"
    SwitchName = $Switch
}

New-VM @VM

# Set 2 CPU processors
Set-VMProcessor $VMName -Count 2

# Set 1GB min memory for worker nodes
Set-VMMemory -VMName vworker5 -MinimumBytes 1073741824

# Change secure boot template to MS UEFI CA
#Set-VMFirmware -VMName $VMName -SecureBootTemplate MicrosoftUEFICertificateAuthority
Set-VMFirmware -VMName $VMName -SecureBootTemplateId "272e7447-90a4-4563-a4b9-8e4ab00526ce"

# Add DVD Drive to Virtual Machine
Add-VMScsiController -VMName $VMName
Add-VMDvdDrive -VMName $VMName -ControllerNumber 1 -ControllerLocation 0 -Path $InstallMedia

# Mount Installation Media
$DVDDrive = Get-VMDvdDrive -VMName $VMName

# Configure Virtual Machine to Boot from DVD
Set-VMFirmware -VMName $VMName -FirstBootDevice $DVDDrive

# Connect/Start the VM on Hyper-V Manager
#
# enter grub mode by typing "c" on installation menu and enter following 3 lines
# linux /install.amd/vmlinuz auto=true preseed/url=192.168.1.244:8080/vworker5-preseed.cfg priority=critical ---
# initrd /install.amd/initrd.gz
# boot

# Set "production" checkpoint mode
Set-VM -Name $VMName -CheckpointType Production

# Create checkpoint/snapshot and restore
Checkpoint-VM -Name $VMName -SnapshotName PostInstallation
Restore-VMCheckpoint -Name PostInstallation -VMName $VMName -Confirm:$false

# Add harddisk
Get-VMHardDiskDrive -VMName vworker5 -ControllerType SCSI
Add-VMScsiController -VMName vworker5
Get-VMHardDiskDrive -VMName vworker5 -ControllerType SCSI
Add-VMHardDiskDrive -VMName vworker5 -ControllerType SCSI -ControllerNumber 2 -Path F:\hyperv/vhd/directpv.vhd

Confirm the disk inside the VM vworker5.

root@vworker5:~# fdisk -l
Disk /dev/sdb: 50 GiB, 53687091200 bytes, 104857600 sectors
Disk model: Virtual Disk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: ABB825AA-D6CF-4AB0-872B-3AE899F23CDC

Device       Start       End   Sectors  Size Type
/dev/sdb1     2048   1050623   1048576  512M EFI System
/dev/sdb2  1050624   2050047    999424  488M Linux filesystem
/dev/sdb3  2050048 104855551 102805504   49G Linux LVM


Disk /dev/sda: 300 GiB, 322122547200 bytes, 629145600 sectors
Disk model: Virtual Disk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/vworker5--vg-root: 48.06 GiB, 51606716416 bytes, 100794368 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/vworker5--vg-swap_1: 980 MiB, 1027604480 bytes, 2007040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

tweaks made¶

disable auto checkpoint
set checkpoint type to production
change auto start action to none, and auto stop action to shutdown

https://www.reddit.com/r/HyperV/comments/moj25r/hyperv_vm_extremely_slow_disk_usage_always_at_100/

Hide the Search Bar on the Taskbar if it is currently displayed
Go into Services, Stop the Windows Search Service
Open Indexing Options
Click Advanced
Click Rebuild next to Troubleshooting

get VM network settings¶

# https://stackoverflow.com/a/27999072
get-vm | ?{$_.State -eq "Running"} | select -ExpandProperty networkadapters | select vmname, macaddress, switchname, ipaddresses | ft -wrap -autosize

Here is the example output when I built VMs initially starting with DHCP.

╰─❯ get-vm | ?{$_.State -eq "Running"} | select -ExpandProperty networkadapters | select vmname, macaddress, switchname, ipaddresses | ft -wrap -autosize

VMName                           MacAddress   SwitchName         IPAddresses
------                           ----------   ----------         -----------
k8s_vcp_1707806419070_95468      00155D01F41F External VM Switch {192.168.1.31}
k8s_vworker1_1707806649152_50802 00155D01F420 External VM Switch {192.168.1.32}
k8s_vworker2_1707806752033_33478 00155D01F421 External VM Switch {192.168.1.33}
k8s_vworker3_1707806821736_75591 00155D01F422 External VM Switch {192.168.1.34}
k8s_vworker4_1707806890423_84706 00155D01F423 External VM Switch {192.168.1.35}