Home Lab Gen IV – Part III: Best ESXi White box Mobo yet?

Posted on Updated on

Initially, when I decided to start this refresh my Home Lab to GEN IV I planned to wipe just the software, add InfiniBand.  I would keep most of the hardware. However, as I started to get into this transformation I decided it was time for a hardware refresh too including moving to All Flash vSAN.

In this post, I wanted to write a bit more about my new mobo and why I think it’s a great choice for a home lab. The past workhorse of my home lab has been my trusty MSI Z68MS-G45(B3) Rev 3.0 (AKA MSI-7676). I bought 3 MƒSI-7676 in 2012 and this mobo has been a solid performer and they treated me very well. However, they were starting to age a bit so I sold them off to a good buddy of mine and I used those resources to fund my new items.

My new workhorse –

Items kept from Home Lab Gen III:

  • 3 x Antec Sonata Gen I and III each with 500W PS by Antec: I’ve had one of these cases since 2003, now that is some serious return on investment

New Items:

  • 3 x Gigabyte MX31-BS0 – So feature rich, I found them for $139 each, and this is partly why I feel it’s the best ESXi white box mobo
  • 3 x Intel Xeon E3-1230 v5 – I bought the one without the GPU and saved some $$
  • 3 x 32GB DDR4 RAM – Nothing special here, just 2133Mhz DDR4 RAM
  • 3 x Mellanox Connectx InfiniBand cards (More to come on this soon)
  • 4 x 200GB SSD, 1 x 64GB USB (Boot)
  • 1 x IBM M5210 JBOD SAS Controller

Why I chose the Gigabyte MX31-BS0 –

Likes:

  • Headless environment: This Mobo comes with an AST2400 headless chipset environment. This means I no longer am tied to my KVM. With a java enabled browser, I can view the host screen, reboot, go into BIOS, BIOS updates, view hardware, and make adjustments as if I was physically at the box
  • Virtual Media: I now can virtually mount ISOs to the ESXi host without directly being at the console (Still to test ESXi install)
  • Onboard 2D Video: No VGA card needed, the onboard video controller takes care of it all. Why is this important? You can save money by choosing a CPU that doesn’t have the integrated GPU, the onboard video does this for you
  • vSphere HCL Support: Really? Yep, most of the components on this mobo are on the HCL and Gigabyte lists ESXi 6 as a supported OS, its not 100% HCL but for a white box its darn close
  • Full 16x PCIe Socket: Goes right into the CPU << Used for the Infiniband HCA
  • Full 8x PCIe Socket: Goes into the C232  << Used for the IBM M5210
  • M.2 Socket: Supporting 10Gb/s for SSD cards
  • 4 x SATA III ports (white)
  • 2 x SATA III can be used for Satadom ports (orange) with onboard power connectors
  • 2 x Intel i210 1Gbe (HCL supported) NICs
  • E3 v5 Xeon Support
  • 64GB RAM Support (ECC or Non-ECC Support)
  • 1 x Onboard USB 2.0 Port (Great for a boot drive)

Dislikes: (Very little)

  • Manual is terrible
  • Mobo Power connector is horizontal with the mobo, this made it a bit tight for a common case
  • 4 x SATA III Ports (White) are horizontal too, again hard to seat and maintain
  • No Audio (Really not needed, but would be nice)
  • For some installs, it could be a bit limited on PCIe Ports

Some PICS :

The pic directly below shows 2 windows: Window 1 has the large Gigabyte logo, this is the headless environmental controls. From here you can control your host and launch the video viewer (window 2). The video viewer allows you to control your host just as if you were physically there. In windows 2 I’m in the BIOS settings for the ESXi host.

This is a stock photo of the MX31-BS0. It’s a bit limited on the PCIe ports, however, I don’t need many ports as soon I’ll have 20Gb/s InfiniBand running on this board but that is another post soon to come!

If you like my ‘no-nonsense’ blog articles that get straight to the point… then post a comment or let me know… Else, I’ll start writing boring blog content.

5 thoughts on “Home Lab Gen IV – Part III: Best ESXi White box Mobo yet?

    Ren said:
    September 28, 2017 at 8:44 am

    Would you still use this board at this point ? Why not use the atx boards they have ? Gives you more pcie slots. Would the atx board also be as compatible with esxi as this one ? Good tips !

    Like

      Matt Mancini responded:
      September 28, 2017 at 1:30 pm

      Thanks for the comment. Yes, I am currently using this board in my home lab. It is working out well. Originally, I did not plan to use the second PCIe slot as most of my networking would be done via a single high-speed InfiniBand Card and I would use the SATA ports for vSAN. However, after I bought the boards I choose to do an All flash vSAN and at that point, I decided to use the 2nd PCIe slot for an IBM JBOD controller. Here are the KEY areas I like about this board — Headless environment, this means I can totally remote control my hosts and give them access to virtual media too. The price point was number 2, you’d be very hard to find an ESXi compatible Motherboard with remote headless access for the price I got these for. Sure Supermicro makes some really great boards that can do what I described, but they are at a premium price. 3rd Lots of goodies… Internal USB, onboard power for SATA DOM, M.2 slots, lots of SATA connectors, mATX form factor, etc. I hope this helps and thanks for the comment!

      Like

      wtodd_h said:
      October 27, 2017 at 1:48 pm

      I’m curious to know what components of the motherboard did you validate against HCL support to determine this choice? I always find this hard, because using the VMware Compatibility Guide you don’t find consumer or even prosumer hardware on that list.

      Like

        Matt Mancini responded:
        October 27, 2017 at 4:08 pm

        Great question..

        As stated above you cannot find this Gigabyte motherboard on the VMware HCL but you can find some of its discrete components on the list. When choosing an whiteboard motherboard the more discrete components you can find the better chance for compatibility you’ll have.

        Here are a few examples of the components I directly found on the HCL
        CPU support Xeon E3 v5 CPU
        NIC support for the I210 Intel Chipset
        SATA support for the Sunrise Point AHCI controller

        Here are some items I assume will work well as they are part of Motherboards that are on the HCL.
        IPMI Support for the AST2400
        C232 Intel Chipset support

        So its a bit of a mix between what I am able to find and what other HCL motherboards are on the HCL.

        Now this technique is NOT recommended for vSphere Production systems you MUST be 100% on the HCL or you will have a hard time getting support for your systems. Also, its worth nothing that in my other blog posts I call out an incompatibility that I found with the UEFI BIOS boot.
        https://vmexplorer.com/2017/09/26/home-lab-gen-iv-part-iv-overcoming-installation-challenges/

        A simple disablement of the UEFI Bios and all worked perfectly. Again, this is not an HCL motherboard, but its pretty darn close.

        Like

    wtodd_h said:
    October 28, 2017 at 7:12 am

    Admittedly I suffer from analysis paralysis when trying to build a homelab or whitebox. Never finding the holy grail of compatibility, cost, and efficiency. Your website is a wealth of knowledge and I deeply appreciate the personal time you take with explaining and documenting your journey.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s