Home Lab – VMware ESXi 5.1 with iSCSI and freeNAS

Recently I updated my home lab with a freeNAS server (post here). In this post I will cover my iSCSI setup with freeNAS and ESXi 5.1.

Keep this in mind when reading – This Post is about my home lab. My Home Lab is not a high performance production environment, its intent is to allow me to test and validate virtualization software. Some of the choices I have made here you might question, but keep in mind I’ve made these choices because they fit my environment and its intent.

Overall Hardware…

Click on these links for more information on my lab setup…

  • ESXi Hosts – 2 x ESXi 5.1, iCore 7, USB Boot, 32GB RAM, 5 x NICS
  • freeNAS SAN – freeNAS 8.3.0, 5 x 2TB SATA III, 8GB RAM, Zotac M880G-ITX Mobo
  • Networking – Netgear GSM7324 with several VLAN and Routing setup

Here are the overall goals…

  • Setup iSCSI connection from my ESXi Hosts to my freeNAS server
  • Use the SYBS Dual NIC to make balanced connections to my freeNAS server
  • Enable Balancing or teaming where I can
  • Support a CIFS Connection

Here is basic setup…

freeNAS Settings

Create 3 networks on separate VLANs – 1 for CIFS, 2 x for iSCSI < No need for freeNAS teaming

CIFS

The CIFS settings are simple. I followed the freeNAS guide and set up a CIFS share.

iSCSI

Create 2 x iSCSI LUNS 500GB each

Setup the basic iSCSI Settings under “Servers > iSCSI”

  • I used this doc to help with the iSCSI setup
  • Only exception is – Enable both of the iSCSI network adapters in the “Portals” area

 

ESXi Settings

Setup your iSCSI vSwitch and attach two dedicated NICS

Setup two VMKernel Ports for iSCSI connections

Ensure that the First VMKernel Port group (iSCSI72) goes to ONLY vmnic0 and vice versa for iSCSI73

Enable the iSCSI LUNs by following the standard VMware instructions

Note – Ensure you bind BOTH iSCSI VMKernel Ports

 

Once you have your connectivity working, it’s time to setup round robin for path management.

Right click on one of the LUNS, choose ‘Manage Paths…’

Change the path selection on both the LUNS to ‘Round Robin’

Tip – After the fact if you make changes to your iSCSI settings, then ensure you check your path selection as it may go back to default

 

Notes and other Thoughts…

I had issues with freeNAS updating information on their web interface, even after reboots of the NAS and my PC. I moved to Firefox and all issues went away. I cleared my cache in IE and these issues were gone.

Using Jumbo Frames with the SYBA Dual NICs (SY-PEX24028) – Short Answer is NO I was unable to get them to work in ESXi 5.1. SYBA Tech support stated the MAX Jumbo frames for this card is 7168 and it supports Windows OS’s only. I could get ESXi to accept a 4096 frame size but nothing larger, however none of the LUNS would connect, once I moved the frame size to 1500 everything worked perfectly. I beat this up pretty hard, adjusting all types of ESXi, networking, and freeNAS settings but in the end I decided the 7% boost that Jumbo frames offer my wasn’t worth the time or effort.

Summary…

These settings will enable my 2 ESXi Hosts to balance their connections to my iSCSI LUNS hosted by freeNAS server without the use of freeNAS Networking Teaming or aggregation. By far it is simplest way to setup and the out of the box performance works well.

My advice is — go simple with these settings for your home lab and save your time to beat up more important issues like “how do I shutdown windows 8″ J

I hope you found this post useful and if you have further questions or comments feel free to post up or reach out to me.

About these ads

Posted on December 17, 2012, in ESX, ESXi, freeNAS, Home Lab, LIAN LI, Networking, SAN, vmware, Whitebox. Bookmark the permalink. 15 Comments.

  1. All work and no play – Thanks for the lab!

  2. Was there any special settings you needed to do set on the vSphere (v5.1) side to get the iSCSI devices to appear? Im having an issue with FreeNAS 8.3.0 where the iSCSI target is found, but no devices appear. Id appreciate any input you can provide. Thanks.

  3. Excellent post however I was wanting to know if
    you could write a litte more on this topic?
    I’d be very thankful if you could elaborate a little bit more. Kudos!

  4. Thank you for your excellent job. I’ll try to did the same in the next days.
    Can you confirm simultaneous access by both ESXi ? Because I’m wondering if it was possible without special configuration or I don’t know what.
    Thanks again. Pierre

    • Hey Pierre – I’m not sure what you mean by “special configuration” however the config is pretty simple. Setup basic iSCSI on FreeNAS with multi (NOT Link-Agg) NICs. Setup ESXi native Round Robin iSCSI setup and point to those NICs. So yes it will balance the load if you set it up correctly. You can see this in FreeNAS or vCenter Server monitoring. Thanks!

      • Thanks mattman5, I was’nt clear : “normally” (Microsoft saying) one iscsi initiator point one target and only one initiator because FAT can be corrupted if more than one computer access and modofy it’s content during concurrent operations. So my question was : can I have multiple iscsi initiators targetting one LUN whithout any data (FAT) corruption problem ?

      • I’m not using my using my free nas SAN with FAT style Luns.. my iSCSI LUNS are formatted with VMFS which allows for multi-initiators to communicate with the LUNs

  5. Did you made any perfomance test on your environment? I’m missing something on the configuration, I’m almost sure is a config issue ’cause it works fine on other hosts. Let me explain what my lab has:

    Windows 7 Client: Core i7 3820, 16GB RAM, 128GB SSD, 2TB+64GB SSD cache, 1xIntel NIC 1Gbps
    ESXi Server 1: Core i5 650, 8GB RAM, 160GB local HDD, 3x1Gbps Intel NICs
    ESXi Server 2: Athlon II X3 455, 8GB RAM, 160GB local HDD, 3x1Gbps Intel NICs
    FreeNAS Server: Pentium G2030, 8GB RAM, 4x1TB 7200RPM RAIDZ1, 2x1Gbps Realtek NICs

    The RAIDZ1 has a 2TB CIFS share, and a 600GB zvol for iSCSI (2 uplinks, 2x1gpbs).

    CIFS Performance with Windows: 92MB/s Read average
    iSCSI Performance with Windows (1 uplink, 1gpbs): 102MB/s average

    iSCSI Performance with ESXi 5.1, 1 path (1gbps): 35MB/s
    iSCSI Performance with ESXi 5.1, 2 path Round Robin (2x1gbps): 50MB/s

    dd on local: 260MB/s

    There is something wrong between ESXi and the FreeNAS server, the performance is awful.
    NIC model is discarded (Intel vs Realtek, because the Windows client is working near the gigabit maximum).

    Any ideas??

  6. Hi,
    I followed your setup: two different vlan per iscsi path. From vSphere, by looking at iSCSI initiator i see that under iSCSI Server Location there are two ips addresses (whichs great), but Target name are the same. (iqn.2011-03.com.lv-ny-san-03:tgt01) would that cause a problem?
    Thank you

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 41 other followers

%d bloggers like this: