Well folks it’s my long awaited blog post around moving my Homelab from FreeNAS to VMware VSAN.
Here are the steps I took to migrate my Home Lab GEN II with FreeNAS to Home Lab GEN III with VSAN.
- I am not putting a focus on ESXi setup as I want to focus on the steps to setup VSAN.
- My home lab is in no way on the VMware HCL, if you are building something like this for production you should use the VSAN HCL as your reference
The Plan –
- Meet the Requirements
- Backup VM’s
- Update and Prepare Hardware
- Distribute Existing hardware to VSAN ESXi Hosts
- Install ESXi on all Hosts
- Setup VSAN
The Steps –
Meet the Requirements – Detailed list here
- Minimum of three hosts
- Each host has a minimum of one SSD and one HDD
- The host must be managed by vCenter Server 5.5 and configured as a Virtual SAN cluster
- Min 6GB RAM
- Each host has a Pass-thru RAID controller as specified in the HCL. The RAID controller must be able to present disks directly to the host without a RAID configuration.
- 1GB NIC, I’ll be running 2 x 1Gbs NICs. However 10GB and Jumbo frames are recommended
- VSAN VMkernel port configured on every host participating in the cluster.
- All disks that VSAN will be allocated to should be clear of any data.
Backup Existing VMs
- No secret here around backups. I just used vCenter Server OVF Export to a local disk to backup all my critical VM’s
- More Information Here
Update and Prepare Hardware
- Update all Motherboard (Mobo) BIOS and disk Firmware
- Remove all HDD’s / SDD’s from FreeNAS SAN
Remove any Data from HDD/SDD’s . Either of these tools do the job
Distribute Existing hardware to VSAN ESXi Hosts
- Current Lab – 1 x VMware Workstation PC, 2 x ESXi Hosts boot to USB (Host 1 and 2), 1 x FreeNAS SAN
- Desired Lab – 3 x ESXi hosts with VSAN and 1 x Workstation PC
End Results after moves
- All Hosts ESXi 5.5U1 with VSAN enabled
- Host 1 – MSI 7676, i7-3770, 24GB RAM, Boot 160GB HDD, VSAN disks (2 x 2TB HDD SATA II, 1 x 60GB SSD SATA III), 5 xpNICs
- Host 2 – MSI 7676, i7-2600, 32 GB RAM, Boot 160GB HDD, VSAN disks (2 x 2TB HDD SATA II, 1 x 90 GB SSD SATA III), 5 x pNICs
- Host 3 – MSI 7676, i7-2600, 32 GB RAM, Boot 160GB HDD, VSAN disks (2 x 2TB HDD SATA II, 1 x 90 GB SSD SATA III), 5 x pNICs
- Note – I have ditched my Gigabyte z68xp-UD3 Mobo and bought another MSI 7676 board. I started this VSAN conversion with it and it started to give me fits again similar to the past. There are many web posts with bugs around this board. I am simply done with it and will move to a more reliable Mobo that is working well for me.
Install ESXi on all Hosts
Starting with Host 1
- Prior to Install ensure all data has been removed and all disk show up in BIOS in AHCI Mode
Install ESXi to Local Boot HD
- After the install I experienced the ESXi hanging at Boot – ‘Starting up Services – Running usbarbitrator start’
- Solution – Stop usbarbitrator service
- Setup ESXi base IP address via direct Console, DNS, disable IP 6, enable shell and SSH
- Using the VI Client setup the basic ESXi networking and vSwitch
- Using VI Client I restored the vCSA and my AD server from OVF and powered them on
- Once booted I logged into the vCSA via the web client
- I built out Datacenter and add host 1
- Create a cluster but only enabled EVC to support my different Intel CPU’s
- Cleaned up any old DNS settings and ensure all ESXi Hosts are correct
- From the Web client Validate that 2 x HDD and 1 x SDD are present in Host
- Installed ESXi Host 2 / 3, followed most of these steps, and added them to the cluster
Logon to the Webclient
Ensure on all the hosts
- Networking is setup and all functions are working
- NTP is working
- All expected HDD’s for VSAN are reporting in to ESXi
Create a vSwitch for VSAN and attach networking to it
- I attached 2 x 1Gbs NICs for my load that should be enough
Assign the VSAN License Key
- Click on the Cluster > Manage > Settings > Virtual SAN Licensing > Assign License Key
- Under Virtual SAN click on General then Edit
- Choose ‘Turn on Virtual SAN’
- Set ‘Add disks to storage’ to Manual
Note – for a system on the HCL, chances are the Automatic setting will work without issue. However my system is not on the any VMware HCL and I want to control the drives to add to my Disk Group.
Add Disks to VSAN
- Under Virtual SAN click on ‘Disk Management’
- Choose the ICON with the Check boxes on it
- Finally add the disks you want in your disk group
- Allow VSAN to complete its tasks, you can check on its progress by going to ‘Tasks’
- Once complete ensure all disks report in as healthy.
Ensure VSAN General tab is coming up correct
- 3 Hosts
- 3 of 3 SSD’s
- 6 of 6 Data disks
- Check to see if the data store is online
Migrating from FreeNAS to VSAN was relatively a simple process. I simply moved, prepared, and installed and the product came right up. My only issue was working with a faulty Gigabyte Mobo which I resolved by replacing it. I’ll post up more as I continue to work with VSAN. If you are interested in more detail around VSAN I would recommend the following book.