VMware Workstation 17 Nested vSAN ESA Overview

Posted on Updated on

In this high level video I give an overview of my #VMware #workstation running 3 x nested ESXi 8 Hosts, vSAN ESA, VCSA, and a Windows 2022 AD. Additionally, I show some early performance results using HCI Bench.

For more information around my VMware Workstation Generation 8 Build check out my latest BOM here

This entry was posted in ESXi, Home Lab, Storage, vCenter Server, Virtual SAN, vmware, VSAN, Workstation and tagged , .

Home Lab Generation 8 Parts List (Part 2)

Posted on Updated on

Today I’m releasing the parts that I plan to use for my Generation 8 Home Lab! I’ll be using three Dell T7820 Workstations for my ESXi Hosts. In this video I go through the BOM (Bill of Materials) or the parts I selected to build out my systems.

NOTE: Though I’ve installed ESXi 8 on these hosts, this in no way is my complete solution. For now ESXi 8 seems to run without issue but I have not tested it with vSAN ESA yet. I’ll be releasing more videos around settings and networking shortly.

UPDATE: 03/11/23 All parts seem to be working well though I’ve had a string of bad QLogic QL45212HLCU-DE 2 Port 25Gbe cards from eBay. After receiving 2 bad cards, I’ve decided to buy a new card from Server Supply. I’ve installed this card and its working fine. I recommend buying this card new vs. used. https://www.serversupply.com/NETWORKING/NETWORK%20ADAPTER/25%20GIGABIT/DELL/0NJFX_310672.htm

More and updated information can be found here – https://vmexplorer.com/home-lab-bom/ #intelxeon #optane #vmware #vexpert #cloud

This entry was posted in Other.

Home Lab Generation 7: Archive Page

Posted on Updated on

Today I’m retiring my Generation 7 Home lab as my Generation 8 lab will be posted in my Home Lab BOM section very soon. My Gen 7 Home Lab was a vSphere and vSAN 7 All Flash cluster and below was its final configuration. Some of the items below I’ll be reusing in Generation 8, and some of the items I will be selling off.

Overview:

My Gen 7 Home Lab is based on vSphere 7 (VCSA, ESXi, and vSAN), NSX-T, and vRealize Suite. Additionally, I use it for vSphere Nesting and Testing. It contains 3 x ESXi Hosts, 1 x Windows 10 Workstation,  4 x Cisco Switches, 2 x MikroTik 10gbe Switch, 3 x APC UPS, 1 x Synology 1621+, and 1 x Asustor Lockerstor 10.  Read further below for an itemized list.

NOTE: My Home Lab Generation 7 design will not fully work with vSphere 8.  Though I could force the install of vSphere 8 I have decided to upgrade my home lab. Please see my Home Lab BOM for more inforamtion.

HL7
HL-stats
perhost-storage
perhost-nic

ESXi Hosts: (Total cost for one host ~$1325 | Unless noted, the items this area are per host)

NOTE: This Home Lab Generation 7 has worked well for me with vSphere 7 BUT most likely it will need to be updated for vSphere 8 due to pCPU discontinuation. Please see this KB for further information https://kb.vmware.com/s/article/82794

  • Case:
  • Motherboard:
  • CPU: (These CPUs work on vSphere 7, but chances are they will NOT work with vSphere 8)
    • 2 x Xeon E5-2640 v2 8 Cores / 16 HT (Ebay $30 each)
  • CPU Cooler or Heatsink: (Bracket required to fit narrow 2011 cooler mounts)
  • RAM:
    • 256GB DDR3 ECC RAM (Mostly from Ebay | each host has a different brand)
      • 16 x Host1: ELPIDA 16GB 2Rx4 PC3L-10600R-09-11-E2 Registered ECC Memory DDR3
      • 16 x Host 2: SAMSUNG M393B2K70CM0-YF8 16gb Pc3-8500r 1066mhz 1.35v Quad Rank X4 Ecc Registered Cl7 Ddr3 Sdram 240-pin Rdimm Memory Module For Server
      • 16 x Host 3: HYNIX HMT42GR7BMR4C-G7 16gb Pc3-8500r 1066mhz Cl7 Quad Rank X4 Registered Ecc Ddr3 Sdram 240-pin Rdimm Memory Module For Server
  • Disks:
    • 1 x Dual M.2 PCIe Adapter Card for NVMe/SATA SSD (Amazon $15)
      • 1 x 512GB NVMe SSD | vSAN Cache | Sabrent Rocket 512 (B&H $69)
      • 1 x 240GB M.2 SSD | Boot Disk | Kingston A400 (Amazon $35)
    • 2 x Supermicro AOC-SLG3-2M2 PCIe NVMe (Amazon $55)
      • 2 x 2TB NVMe SSD | vSAN Capacity | Sabrent Rocket 2TB (Amazon $199)
    • 1 x 2TB SATA | Extra Space | Hitachi (Ebay $50)
  • Network:
    • Motherboard Integrated i350 1gbe 4 Port
    • 1 x MellanoxConnectX3 Dual Port (HP INFINIBAND 4X DDR PCI-E HCA CARD 452372-001)
  • Power Supply:
    • Antec Earthwatts 500-700 Watts (Adapters needed to support case and motherboard connections)
      • Adapter: Dual 8(4+4) Pin Male for Motherboard Power Adapter Cable (Amazon $11)
      • Adapter: LP4 Molex Male to ATX 4 pin Male Auxiliary (Amazon $11)
      • Power Supply Extension Cable: StarTech.com 8in 24 Pin ATX 2.01 Power Extension Cable (Amazon $9)

Storage (NAS):

Network:

Battery Backup UPS:

This entry was posted in Other.

How to upgrade a Dell T7820 to a U.2 Backplane

Posted on Updated on

In this video I show how I upgraded my Dell T7820 SATA backplane to a U.2 backplane. I’m doing this upgrade to enable support for 2 x #intel #Optane drives. I’ll be using these #Dell #T7820 Workstations for my Next Generation #homelab where I’ll need 4 x Intel Optane drives to support #VMware #vsan ESA.

Part Installed in this Video: (XN8TT) Dell Precision T7820 T5820 U.2 NVME Solid State Drive Backplane Kit found used on Ebay.

For more information around My Next Generation 8 Home Lab based on the Dell T7820 check out my blog series at https://vmexplorer.com/blog-series/

This entry was posted in Home Lab, Storage, vmware, VSAN and tagged , .

Announcing my Generation 8 Super VMware Workstation!

Posted on Updated on

I’ve been using VMware Workstation for many years and its been a great tool to have in my home lab plan. Over the last couple of years, I had been using multiple computers to do various tasks (VMware Workstation, PC, PLEX, Video Editing, etc.) and they seemed to work okay. Most of these were older PCs I had around that I repurposed. However, a follower of my blog made a very generous donation of several key components. It was enough to really get me thinking about creating a SUPER Workstation box. I wanted to make an all in one workstation that could handle everything I currently supported plus more. I started to evaluate what I wanted to accomplish and created a simple goal list.

Goals:

  • Support for 3 different Networks:
    • I wanted to have a 1Gbe and 10Gbe into my Home lab
    • The other connection would be used for basic networking
  • Run VMware Workstation:
    • Support several PC VMs
    • Support nested ESXi/vSAN 8 VMs, without having to force the install
    • Have enough resources to run everything from one box
  • Run PLEX Server
    • I wanted to make sure I had enough HDD space to run my PLEX media
  • Video Editing
    • Support x16 PCIe Video Card
    • From time to time I create videos and I wanted a system that could be performant around this.

From there I inventoried the parts I already had, chose new components, and created my Generation 8 VMware Workstation.

BOM:

To better clarify some of my component choices I created this video and announced my DREAM #vmware #workstation

I’m sure it will be updated as time goes and my most recent Bill of Materials (BOM) can be found here. https://vmexplorer.com/home-lab-bom/

This entry was posted in Other.

Working with Dell T7820 Disk trays

Posted on Updated on

As part of my Generation 8 ESXi/vSAN ESA Home lab build, in this video I go over how to work with the T7820 disk trays. They can be a bit tricky to work with, especially when adding or subtracting the 2.5″ carrier.

This entry was posted in Other.

First Look GEN8 ESXi/vSAN ESA 8 Home Lab (Part 1)

Posted on Updated on

I’m kicking off my next generation home lab with this first look in to my choice for an ESXi/vSAN 8 host. There will be more videos to come as this series evolves!

This entry was posted in Dell, ESXi, Home Lab, InfiniBand, Other, Video Blog, vmware, VSAN.

Dell T7820: CPU Upgrade Issues *Solved*

Posted on Updated on

Quick Read: Did you buy a used barebones Dell T7820 or upgrade your CPU from Xeon Silver to Gold and now the T7820 won’t boot? Are you getting the error Code Memory/RAM failure: 2 amber blinks followed by a short pause, 4 white blinks, long pause, then repeats:?

Solution: Ensure your BIOS is at least 2.6.3 or later.

More Details:

I recently purchased 3 used Dell Precision Workstations model T7820 from eBay. They will be the replacements for my next generation Home Lab with vSphere 8. These used T7820 are barebone systems that didn’t come with a CPU or RAM. Other than a vNivida Video card they were pretty much empty. After inserting a Xeon 6252 and RAM modules, they all powered on but only one would post. The other two simply gave me a blink code (2 amber blinks followed by a short pause, 4 white blinks, long pause, then repeats). It was an odd error as their manuals stated it supported the Xeon Gold 6252 and 2933Mhz RAM.

Looking at the Dell 7820 manual I soon found Dell code description that matched the blink code: Memory/RAM failure. I swapped the known working RAM and CPU between the systems but still the errors persisted. So at this point, I knew my RAM and CPU were compatible with the T7820 but why would only 1 of 3 them work?

I now put a focus on the one system that worked. I first checked its BIOS level. It was running Dell BIOS 2.6.3 and funny enough the BIOS release notes specifically called out fixing a memory speed issue. It read: ‘Supports memory speed of 2933 MHz with two memory modules per channel’

It appears that when the T7820 posts, it queries the CPU for its max memory speed rating. In my case the Xeon 6252 has a max memory speed is 2933Mhz. So, even if I put in 2666Mhz RAM it still will not work, because its asking the CPU for is max speed rating. Dell BIOS 2.6.3 fixes this issue allowing the for faster memory speeds to be supported.

Knowing this, the fix should be simple. All I need to do is update my BIOS to 2.6.3 or later. However, the issue is I can’t update or even check the BIOS version on my 2 non-working systems. To do this I’m going to need a Xeon CPU whose max RAM frequency is slower than 2933Mhz, it’s something I don’t have. Enter my eBay Seller oztech llc. to the rescue!

I’ve heard of so many bad experiences when working with eBay sellers, but working with the eBay vendor oztech llc has been an absolute pleasure and I would highly recommend them. They are very responsive, helpful, and knowledgeable around the product they sell. They had not heard of this issue before but were willing to help and after a short call explaining my issue, they promptly shipped out a Xeon Silver 4114 CPU that supported a max RAM speed of 2666Mhz and matching RAM. This should allow me power on my T7820s and update to the latest Dell BIOS (2.29.0)

When the Xeon Silver CPU arrived I did the following:

  • Prepare the T7820
    • Removed the existing Xeon Gold CPU and 2933 Mhz RAM
    • Plugged in the power to the system and then unplugged (don’t skip this step)
    • Cleared the CMOS
    • Installed the Xeon Silver CPU and 2666Mhz RAM in Slot one
    • Plugged in the T7820
    • It power cycled about 4 times as it adjusted the system settings, and then it booted!
  • Confirm Current BIOS Level
    • During boot I pushed F12 and went into setup
    • Confirmed its BIOS level to be 1.7.1
  • Updating the BIOS to 2.29.0
    • The T7820 BIOS update is designed to work with Windows 10.
    • I booted to Windows 10, and ran the 2.29.0 BIOS update
    • The update confirmed it was currently at 1.7.1
    • I ran the BIOS update
    • Once it was completed I checked the T7820 BIOS but it came up with BIOS 2.6.3
    • I ran the BIOS update again, it confirmed 2.6.3 was current and it would update the T7820 to 2.29.0
    • After a reboot I confirmed the BIOS to be 2.29.0
  • Finally, installing the Xeon Gold CPU
    • Powered down the T7820, removed the Xeon Silver CPU and RAM
    • Plugged in the power then removed, and cleared the CMOS
    • Inserted the Xeon Gold CPU / 2933Mhz RAM, powered on, and allowed it to adjust for the new components (reboots multiple)
    • Went into setup and confirmed it saw the Xeon Gold CPU
  • I repeated this process on my other T7820, only its BIOS was 1.4.1.
  • I had no issues with either system after I completed this process.

Why this odd BIOS Update behavior? Though I could not find any information in the 2.29.0 release notes, I can only assume that it had a requirement for systems to be on 2.6.3 to be able to update. This would explain why it updated the system to 2.6.3 first then it updated to 2.29.0

It took a bit of trial and error to figure out this issue but with the great help of oztech and their all out willingness to ensure my satisfaction I now have 3 working T7820s. Next I’m off to install ESXi 8 and update my Home Lab but that my readers will be a different blog post.

This entry was posted in Other.

Installing Intel U.2 Optane Disk using an adapter

Posted on Updated on

In this quick video I installed a Intel U.2 Optane disk into my Windows system using the StarTech U.2 to PCIe Adapter and then speed tested it with ATTO. I’ll be using these drives and adapters in my Next Generation 8 Super VMware Workstation that I am currently working on. Note the drive that I show being installed on to the StarTech card is different from the one I ran ATTO with.

** Please do keep in mind that the average PCIe slot has a 25 Watt Power Rating. Some systems have a higher wattage rating and some could be lower. The P5800X draws about 30 Watts and the P4800X about 23 Watts. Take this into account when using this adapter and plan accordingly. **

Products Show in this video:

Drive that was tested with ATTO: https://ark.intel.com/content/www/us/en/ark/products/201861/intel-optane-ssd-dc-p5800x-series-400gb-2-5in-pcie-x4-3d-xpoint.html

Drive that was shown being installed on the Adapter: https://ark.intel.com/content/www/us/en/ark/products/129968/intel-optane-ssd-dc-d4800x-series-750gb-2-5in-pcie-2×2-3d-xpoint.html

StarTech Adapter: https://www.startech.com/en-us/hdd/pex4sff8639

This entry was posted in Other.

Super VMware Workstation: Install tips for the Supermicro X11SPL-F

Posted on Updated on

In this quick video I go over a few of the installation tips when mounting this motherboard into a Phanteks Enthoo Pro case. I’m looking at this motherboard as a possible candidate for my next Super VMware Workstation / PLEX Server.

Motherboard: https://www.supermicro.com/en/products/motherboard/x11spl-f

Case: https://www.phanteks.com/Enthoo-Pro.html

This entry was posted in Other.