vmware
Update to my Home Lab with VMware Workstation 8 – Part 1 Why
Over the couple of years my home lab running VMware Workstation 7 has served me quite well. I’ve been able to deploy VM’s as needed (Windows, ESX, etc) and attach it to my shared storage.
However as my role has changed to a TAM at VMware I’ve needed to depend on my lab to interface with my customers and keep my technical skills up. VMware supplies labs and access to many of their products and this approach is a good one, however there is something said for building it yourself.
Before I just start setting up software, hardware, SAN’s, network, etc I needed to reevaluate my current home lab and setup my new objectives / goals —
“To update a home lab one must look back to move forward” – Matt Mancini 
What were some of the objectives and goals I chose back then?
What do you want to accomplish by setting up a home lab or work test environment?
I wanted to setup a home lab with a couple of ESX servers and shared storage
I wanted to do this so I could testing basic ESX functions and new versions
What are the expectations for uptime and performance?
I’d like the system to perform well as a home PC and a test lab
I’d like to use it to do basic functions and run my lab quickly keeping delays to a minimum
What equipment do I have access to and what do would I need to purchase?
Have – High End PC, Home Network
Purchase – More RAM, Shared Storage, Workstation Software
I built the following environment based on my objectives and goals–
Hardware / Software –
Network –
Internal networks to support vMotion and ESX Management Network
External Single NIC to support VM’s
Storage —
iSCSI, NFS – To support ESX Hosts
CIFS Shares – for a Public Share
Single NIC from home lab to support connection to IX4
500GB HD – Boot OS and Some Workstation VM’s
300GB HD – For Workstation VM’s
160GB HD – For Workstation VM’s
Current VM’s –
Web Safe – XP VM’s used for surfing the web
Home VM – Used for Home / Personal Use
ESX 3.5 Cluster – built but rarely used
ESX 4.0 Cluster – Used but not fully configured
vCenter Server VM – Windows 2003 x32 running vCenter Server 4.0
Outcomes for 09/2011 –
Currently my home lab is struggling to keep up. As my role has changed the demands on my home lab have increased. The main constraint appears to be RAM, 8GB of RAM won’t cut it anymore. So now I’m faced with updating or replacing my home lab.
The Update Plan 10/2011 –
Generally the home lab is doing okay but it is struggling with local RAM. If I run more then 3 VM’s (1 – vCenter Server, and 2 – ESX) there isn’t really much left and paging begins.
I start to look to update my current Mother board (MoBo) to 16GB of RAM, it’s MAX. As I start to research 4 x 4GB DDR2, I find that DDR3 has consumed the market. and DDR2 is so far past its prime that updating it would cost about ~$250 for RAM.
It’s time for a major overhaul for my home lab – What are the new objectives and considerations?
What do you want to accomplish by setting up a home lab or work test environment?
I wanted to setup my home lab to be able to run One vCenter Server, 2 – ESX servers w/ shared storage iSCSI and NFS, maybe on other XP VM, at the same time with out paging to disk
I to test basic ESX functions, new versions, explore options for customers, work with beta code, etc
What are the expectations for uptime and performance?
I’d like the system to perform well as a home PC and a test lab
I’d like to use it to do basic functions and run my lab quickly keeping delays to a minimum
What equipment do I have access to and what do would I need to purchase?
Have – High End PC, Home Network, Shared Storage, Workstation Software
What newer hardware update should be considered?
i7 Core CPU, MoBo with 32GB support, USB 3.0, onboard mSATA, SSD Drives, SATA 6.0 Support
With these things in mind I decided on the following items…
Let chats a bit about the Buy items and reasons why I choose them…
Gigabyte GA-Z68XP-UD3 Mobo
Here are the key drivers why I choose this motherboard –
Price and options were my number one driver, I had looked at other MoBo’s but when I narrowed it down this one kept coming up…
4 x SATAIII – 6.0Gbs Support – Really the MoBo out there with this (Note – this MoBo uses 2 different Chipsets to achieve this (2 x on the Intel Z68 and 2 x on the Marvell chipset)
4 x SATAII – 3.0Gbs Support – Uses the Z68 Chipset
32GB RAM Max
USB 3.0 Support
Good PCI Support – PCI Express 2.0 x16 2 (x16, x8), PCI Express x1, 3 PCI Slots 2 (I did look at PCI Express 3.0 boards but at this point they are more of a fad then reality)
Here are the details on this board — http://www.newegg.com/Product/Product.aspx?Item=N82E16813128512
60GB SSD SATAIII aka 6.0Gbs
The key driver for this product was Speed and Price.
I got the “Corsair Force Series 3 CSSD-F60GB3A-BK 2.5″ 60GB SATA III Internal Solid State Drive (SSD)”
I got this drive at Fry’s for $80 and it stated it could do 500MB/s Read and Write.
Come to find out it can only do 250MB/s RorW however if you buy the GT Model it will.
Don’t take my word for it read about it on the Corsair forums — http://forum.corsair.com/v3/forumdisplay.php?f=188
Other Caveats – Don’t use SSD for your boot drive, before you buy read reviews
I’m not happy about this but the drive still preforms well and I use it to boot VM’s it flys!
Here are the deailts on the SSD – http://www.newegg.com/Product/Product.aspx?Item=N82E16820233223
16GB DDR3 RAM
Key driver here was Size and Price
Once again Frys had the best price and it got 16GB of RAM for $79
Here are the details on the RAM – http://www.newegg.com/Product/Product.aspx?Item=N82E16820233143
With new hardware in hand, let the fun and rebuilding begin – I’m off to Part II of this blog “Update to my Home Lab with VMware Workstation 8 – Part 2 Fun with a Windows 7 Installer“
Passed the VCP4
I recently passed my VCP4 today. It’s been a long road getting here but I’m glad it’s behind me.
I was slated to take it 12/2009 time frame, but we had 4 deaths in the family within a 4 month period!
Needless to say that delay caused me to miss the opportunity to take the test without taking training
However at some point I thought I would get the training I needed to take the VCP but this never came to pass and I remained uncertified until most recently.
Last Summer I took a job with VMware as a TAM and I got the vSphere 4 training I needed. Finally I can say I am a VCP4!
With this behind me it’s time to study for my VCP5, which needs to be completed by end of year 2011 :) or in less than 2 months!
What do I have to add for all of you taking the test?
– Read those documents, know your stuff, take the mock exams, follow the blue prints
– Notice this about the questions… There are plays on words and metrics
Example –
A VM has the following performance counters – paging reads 0 and ballooning reads 240
Or I’ve seen it as
A VM has the following performance counters – paging reads 240 and ballooning reads 0
Also, I’ve seen a bunch of play on words like changing is to isn’t – This changes the answer you choose.
So word to the wise – I believe they are doing this to counter potential VCP’s just studying questions from TestKing, Cerfityme, etc… So if you are, Good Luck – You’ll need it — Its plain and simple – Study hard = Passing the test
9 Great vSphere 5 documents everyone should read
Here are 9 Great vSphere 5 documents everyone should read! Enjoy!
- What’s New in VMware vSphere 5.0 Platform
- What’s New in VMware vSphere 5.0 Storage
- What’s New in VMware vSphere 5.0 Networking
- What’s New in VMware vSphere 5.0 Performance
- What’s New in VMware vSphere 5.0 Availability
- vSphere Storage Appliance
- What’s New in VMware Data Recovery 2.0
- What’s New in VMware vCenter Site Recovery Manager 5.0
- What’s New in VMware vCloud Director 1.5
18 of the best VMware Resolution Path KB Articles
You might be asking yourself what are a Resolution Path KB articles? Resolution path articles are product specific PDF’s that lead you to common answers.
They help you to resolve common issues visually and link you to the KB to help in resolution.
In their own words –
“Mind Maps are a visualization of our Resolution Path KB articles, which are collections of modular steps that can be used to solve tech support issues. Being modular, they can be re-used in other resolution paths. A good example is using the ping command to test network connectivity. This step is used in all kinds of troubleshooting procedures. Put a number of these steps together, and you have a method.”
Here is the URL to the articles –
http://blogs.vmware.com/kb/2011/08/index-of-resolution-path-kb-articles.html
Here’s a quick screenshot but basically expand a category, and choose a link to a KB.
Test Lab – Day 6 Xsigo Redundancy testing with ESXi under load (Final Post)
Today I plan to setup the environment to simulate high disk I/O and re-run most the Xsigo tests from Day 4. My thoughts behind these tests are the Xisgo ran well under a minimal load but how would it perform when stressed.
Note: This blog post only covers how I setup testing but doesn’t have the final results. See A change of plans below for more information.
The environment layout will stay the same (4 x ESXi 4.1 hosts, each host with 10 VM’s, 2 Xsigo VP780’s and one IOMega IX12 with a 4 spindle NFS Datastore)
I’ll auto create about ~40 Windows 2003 VM’s and use PassMarks Burin Test to simulate a NFS load.
I plan to drive the IOMega to ~1Gbs of sustained NFS I/O by driving each VM’s hard disk.
While the load simulation is happening I’ll re-run the testing from Day 4, and record the results.
I do wish we could simulate a larger load however our NFS datastores are only across 4 Spindles. If we simulate a heavier load I believe it will “tank” the datastore making our testing pretty much worthless.
Preparing for the load Test.. In this step I setup my default template
I spun up a Windows 2003 server and installed the Burn in Test program.
I launched the BurnIn Test, clicked on Preferences, uncheck all, checked Disks, and entered a duty cycle of 25
Tip: BurnIn Test can be used for a wide range of load testing including CPU & Network (See below)
I saved the configuration file in the default folder.
I repeated this process and created 4 more config files driving the Disk to duty cycles of 5, 10, 50 and 100.
Writing the batch file to launch the BurnIn Test config file was simple (see below) and I created one for each duty cycle.
Tip: Before changing this VM into a template I tested my batch files to ensure all was working well.
Next I wrote a simple vSphere PowerCLI script to create the VM’s, it nothing complex but it did the trick..
I simply opened the vSphere PowerCLI prompt and pasted in the following lines…
|
$server_address = “vCenter Server IP” $username = “Administrator” $password = “your password here” $template_name = “w2003Template” $datastore_name = “NFS” $customization = “W23K” $location = “LoadTest” $destination_host = “ESX Host 1 IP Address” New-VM -Name 2003VMT21 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT22 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT23 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT24 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT25 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT26 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT27 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT28 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT29 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT30 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization $destination_host = “ESX Host 2 IP Address” New-VM -Name 2003VMT31 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT32 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT33 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT34 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT35 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT36 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT37 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT38 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT39 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization New-VM -Name 2003VMT40 -Location $location -Template $template_name -Host $destination_host -Datastore $datastore_name -OSCustomizationSpec $customization
This pattern was repeated on the other ESX hosts until ~40 VM’s were created… |
Test Procedure…
Using the Burn In test program I will drive the utilization on the ~40 VM’s until 1Gbs of sustained NFS I/O is achieved.
I’ll use vCenter Server to ensure NFS traffic is passing through XSA and then start the tests below.
All of the ESX Hosts are attached to the IOMega via NFS, this means all traffic has to pass through the Xsigo and our network to get to the IOMega.
I used a Laptop, the ESXi console in tech support, vCenter Server mode to capture and validate the results.
Keep in mind this deployment is a SINGLE site scenario and for accuracy we allowed the environment to settle between tests.
A Change in plans.…
At this point the testing had to stop. Reason why – I put in my two weeks’ notice and had to focus on a transition plan and less on testing.
My hopes are if you’re reading this post it will give you a baseline for testing.
Test Lab – Day 5 Expanding the IOMega ix12-300r
Recently I installed an IOMega ix12-300r for our ESX test lab and it’s doing quite will
However I wanted to push our Iomega to about 1Gbs of sustained NFS traffic of the available 2Gbs.
To do this I needed to expand our 2 drive storage pool to 4 drives.
From a previous post we created 3 storage pools as seen below.
Storage Pool 0 (SP0) 4 Drives for basic file shares (CIFS)
Storage Pool 1 (SP1_NFS) 2 drives for ESX NFS Shares only
Storage Pool 2 (SP2_iSCSI) 2 drives dedicated for ESX iSCSI only
In this post I’m going to delete the unused SP2_iSCSI and add those drives to SP1_NFS
Note: This procedure is simply the pattern I used in my environment. I’m not stating this is the right way but simply the way it was done. I don’t recommend you use this pattern or use it for any type of validation. These are simply my notes, for my personal records, and nothing more.
Under settings select storage pools
Select the Trash Can to delete the storage pools..
It prompted me to confirm and it deleted the storage pool.
Next I choose the Edit icon on SP2_NFS, selected the drives I wanted, choose RAID 5, and pressed apply.
From there it started to expand the 2 disk RAID1 to a 4 disk RAID5 storage pool..
Screenshot from the IOMega ix12 while it is being expanded…
I then went to the Dashboard and under status I can view its progress…
ALL this with NO Down time to ESX, in fact I’m writing this post from a VM at the expansion is happening.
It took about 11 Hours to rebuild the RAID set.
Tip: Use the event log under settings to determine how long the rebuild took.
The next day I checked in on ESX and it was reporting the updated store size.
Summary…
To be able to expand your storage pool that houses your ESXi test environment with no down time is extremely beneficial and a very cool feature.
Once again IOMega is living up to its tag line – IOmega Kicks NAS!
Tomorrow we’ll see how it performs when we push a higher load to it.
Test Lab – Day 2 CLI with the Xsigo!
Yesterday I did about 90% of the hardware install. Today, Day 2, our Xsigo SE will be here to assist with the installation and configuration of the Xsigo to the ESX Hosts..
Today’s Goals..
- Install 2nd Xsigo VP780
- Install vmware ESXi 4.1 on 4 servers with Xsigo Drivers
- Configure both Xsigo vp780’s
Install 2nd Xsigo VP780…
Day 2 started out with a gift from Mr. FedEx, it was the parts we needed to install the 2nd Xsigo. Only yesterday afternoon we discovered we were missing some power cords and mounting brackets. A couple quick calls to Xsigo and viola parts are on their way. Props to Xsigo for a VERY quick response to this issue!
Based on the lessons learned from Day 1 we mounted the 2nd Xsigo VP780 and it went much smoother. Notice the WE part of installing the VP780, these things are heavy & large and you’ll need some help or a giant with huge hands to install them into a rack. See their install manual for more information.
When we powered them up I was amazed by the amount of air they moved through the device >> Very NICE!
Keep in mind at this point all the test lab hardware including the Xsigo fiber modules (2 x 10gig Fiber modules per device), and networking is mounted and interconnected…
Install vmware ESXi 4.1 on 4 servers with Xsigo Drivers…
You’ll need the Xsigo Drivers installed for ESXi to recognize the infiniband cards and for proper communication.
There are two installation options…
- Install ESXi 4.1 and add the Xsigo Drivers after the install.
- Download the drivers and re-master the ESXi ISO yourself (This is a good option if your building / rebuilding lots of servers)
We chose to re-master the ESXi ISO with the Xsigo drivers.
Here is the link to master the ISO
I won’t bore you with the details of installing ESXi, however the only gotcha I ran into was the Dell R5400 SATA RAID controller.
I setup a SATA RAID group, during the ESXi install it recognized the RAID volume, and ESXi installed to it without issue.
However after the reboot of the host it would not boot to this volume.
I didn’t have time to troubleshoot, for now we just broke the RAID group, reinstalled, and it worked perfectly.
ESXi Management NICS’s..
Our test lab network will be isolated from production network traffic. However, one of our servers will need to be in the production environment. We setup one physical NIC (pNIC) on to our production environment. This will allow us to temporarily transfer VM’s from production to test, we’ll then disconnect this pNIC and setup ESXi to use the Xsigo NIC for management.
(More to come on this on Day 3)
Configure both Xsigo vp780’s…
Configuring the vp780 was very simple. We attached a laptop to the Xsigo and in about 20 commands our Xsigo was up and running..
These are the basic commands we used to setup our pair of Xsigo’s (A and B), the commands below reflect B only.
The commands would be the same for the A Xsigo simply change the appropriate parameters…
NOTE: I don’t recommend you execute these commands in your environment, keep in mind these are for my reference ONLY… I also recommend you contact your Xsigo representative for assistance.
Here are the commands we executed..
Getting into the Xsigo VP780…
We used a standard Xsigo provided rollover cable plugged into Serial1. (Serial2 is for Tech / Debug – Don’t use)
We connected to the console via Putty or Absolute Telnet (COM Settings are 115200,8,1,None,None)
Tip: All default passwords are in the CLI Config Guide by Xsigo
Setup the Xsigo via the Wizard…
Once the connected we used the XgOS config Wizard and entered in the following..
|
Welcome to XgOS Copyright (c) 2007-2010 Xsigo Systems, Inc. All rights reserved.
Enter “help” for information on available commands.
Would you like to use the XgOS Configuration Wizard? [Y/n] Hostname: xsigo-b Domain: YOURDOMAIN.COM Is this Director to be designated as the IB subnet manager (leave as Y unless using an external, non-Xsigo subnet manager) ? [Y/n] Do you want this Director to send diagnostic data to Xsigo periodically? [Y/n] Please input the ‘root’ password: **** Please confirm the ‘root’ password: **** Please input the ‘admin’ password: ***** Please confirm the ‘admin’ password: ***** Please input the ‘recovery-password’: **** Please confirm the ‘recovery-password’: **** IP Address [static/DHCP]: 555.555.555.555 IP Address [static/DHCP]: Enter NTP Server 1: 555.555.555.555 Enter NTP Server 2: Enter Timezone [<Tab><Tab> for the list of Timezones]: America_Phoenix Welcome to XgOS Copyright (c) 2007-2010 Xsigo Systems, Inc. All rights reserved.
Enter “help” for information on available commands. admin@xsigo-b[xsigo] |
Now it’s time to setup the Xsigo…
Place the Xsigo into Trunk Mode..
Port 10 and Port 11 are the 10gig Fibre Modules; this command places them in Trunk Mode
set ethernet-port 10/1 -mode=trunk << Port 10 will be used for our IP Network (Vlans for Guests, vmotion, hosts, etc)
set ethernet-port 11/1 -mode=trunk << Port 11 will be used for our NFS
Rear of VP780
Ensure Trunk Mode is activated..
Use the command ‘show ethernet-port ‘
|
admin@xsigo-b[xsigo] show ethernet-port
name type state descr mode flags lag access-vlan vnics vlans ——————————————————————————- 10/1 nwEthernet10GbPort up/up trunk -s— 1 0 none 11/1 nwEthernet10GbPort up/up trunk -s— 1 0 none 2 records displayed |
Setup Phone Home for Support…
set system phone-home -customer-name=”YOUR COMPANY NAME HERE”
set system phone-home -contact-email-address=YOURNAME@YOURDOMAIN.COM
set system phone-home -contact-phone-numbers=”555-555-5555″
set system phone-home proxy [YOUR PROXY IP HERE] [PROXY PORT if needed, default is 3128]
Note: For this command the syntax is [PROXY IP Address] one space [PROXY PORT], don’t use ‘:’ to as the separator.
Once completed then check confirm your information…
Enter the command ‘show system phone-home’
|
admin@xsigo-b[xsigo] show system phone-home ——————————————————————————- enabled true freq weekly next Fri Jan 14 12:44:52 MST 2011 notify no strip yes alarm yes name COMPANYNAME email EMAIL@EMAIL.com phone 5555555555 copy p-host 555.555.555.555:3128 p-user ——————————————————————————- 1 record displayed admin@xsigo-b[xsigo] |
Check on the Phone Home Log….
admin@xsigo-b[xsigo] showlog phonehome.log
Wed Jan 5 17:30:33 MST 2011: Phone home successful to http://phone-home.xsigo.com:6522
Wed Jan 5 18:04:14 MST 2011: Phone home successful to http://phone-home.xsigo.com:6522
Wed Jan 5 18:04:38 MST 2011: Phone home successful to http://phone-home.xsigo.com:6522
[Press CRTL-C to Exit]
admin@xsigo-b[xsigo]
Tip: your log might be empty until it has something to send
Ensure your Physical servers are attached…
As expected all 4 servers are attached to this Xsigo.. (If they don’t show up here it could be an interconnect or ESXi issue)
Enter the command ‘show physical-server’ to view your connected servers.
|
admin@xsigo-b[xsigo] show physical-server ——————————————————————————- name localhost <<< This is the ESXi Hostname guid 2c903000b4df5 descr port xsigo-001397001:ServerPort2 << This is the Xsigo Port the Server is connected to os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64 << This is the version of ESX & Xsigo Driver version 2.7.0/3.0.0 server-profile << Notice this is blank, We configured it next ——————————————————————————- name localhost guid 2c903000b4ea5 descr port xsigo-001397001:ServerPort3 os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64 version 2.7.0/3.0.0 server-profile ——————————————————————————- name localhost guid 2c903000b4ea9 descr port xsigo-001397001:ServerPort4 os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64 version 2.7.0/3.0.0 server-profile ——————————————————————————- name localhost guid 2c903000b5095 descr port xsigo-001397001:ServerPort1 os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64 version 2.7.0/3.0.0 server-profile ——————————————————————————- 4 records displayed |
Create Server Profiles…
Creating a server profile enables you to assign devices to your specific host.
In our case we used the ESX Hostname as the Xsigo Server Profile name.
This will help us to keep the profiles well organized.
Keep in mind YOURSERVERNAME# equals your ESX Hostname and it will become your Xsigo Server Profile Name…
Long way to create a Server Profile…
add server-profile [server profile name]
View the new server profile…
|
admin@xsigo-b[xsigo] show server-profile name state descr connection def-gw vnics vhbas ——————————————————————————- YOURSERVER1 up/unassigned 0 0 1 record displayed |
Assign the server profile to a port on the Xsigo…
set server-profile YOURSERVER1 connect localhost@xsigo-001397001:ServerPort1
Short way to create a Server Profile…
add server-profile YOURSERVER2 localhost@xsigo-001397001:ServerPort2
add server-profile YOURSERVER1 localhost@xsigo-001397001:ServerPort3
add server-profile YOURSERVER1 localhost@xsigo-001397001:ServerPort4
Then use show server-profile to confirm your entries…
|
admin@xsigo-b[xsigo] show server-profile name state descr connection def-gw vnics vhbas ——————————————————————————- Yourserver3 up/up localhost@xsigo-001397001:ServerPort3 0 0 Yourserver4 up/up localhost@xsigo-001397001:ServerPort4 0 0 Yourserver1 up/up localhost@xsigo-001397001:ServerPort1 0 0 Yourserver2 up/up localhost@xsigo-001397001:ServerPort2 0 0 4 records displayed admin@xsigo-b[xsigo] |
Set Up and attach the virtual NICS to your server profile…
In this step we created our Xsigo vNICS, attached them to the appropriate server profiles, and the 10gig Modules.
When complete each of our ESXi servers will have 4 Xsigo vNICS.
(2 vNICs for IP Network, 2 vNICs for Storage network)
Decoding the command…
The command ‘add vnic xnb.yourservername1 10/1 -mode=trunk’ breaks down to…
add vnic << Add vNIC Command
xnb << The vNIC Name (xnb = Xsigo, IP Network, B Xsigo Device, Xsb = Xsigo, Storage Network, B Xsigo Device)
yourservername1 << Which profile to attach to
10/1 << Which Module on the Xsigo to attach to
-mode=trunk << What transport mode
These are the command we entered..
|
IP Network vNICS admin@xsigo-b[xsigo] add vnic xnb.yourservername1 10/1 -mode=trunk admin@xsigo-b[xsigo] add vnic xnb.yourservername2 10/1 -mode=trunk admin@xsigo-b[xsigo] add vnic xnb.Yourservername3 10/1 -mode=trunk admin@xsigo-b[xsigo] add vnic xnb.Yourservername4 10/1 -mode=trunk
Storage vNICS admin@xsigo-b[xsigo] add vnic xsb.Yourservername1 11/1 -mode=trunk admin@xsigo-b[xsigo] add vnic xsb.Yourservername2 11/1 -mode=trunk admin@xsigo-b[xsigo] add vnic xsb.Yourservername3 11/1 -mode=trunk admin@xsigo-b[xsigo] add vnic xsb.Yourservername4 11/1 -mode=trunk |
Results from ESXi…
Other Information…
Set System back to factory Defaults…
If needed, you can set the System back to factory Defaults by the following command.
When complete you will need to access the system via Serial Cable.
Here are the steps:
set system factory-default
confirm << type in Confirm, my Putty will exited and the system will shutdown
NOTE: This command will erase the configuration from the Xsigo. Do it with caution
Tip: Note this will cause the system to shutdown, this means someone will have to manually power it back on.
Upgrade the XgOS via USB…
Download the GOS 2.8.5 to a USB Stick..
We inserted the stick into the USB Port on the VP780, then executed this command
system upgrade file://usb/xsigo-2.8.5.xpf
Other Handy commands…
show system status
show system
show system version
show system warnings
show serial
show system info
history
CLI Fun…
One thing I like about the CLI for Xsigo is TAB at the end of the command (most modern CLI’s have this and it sure is handy)
If I type in set system phone-home[Press TAB] it displays possible completions and qualifiers and then it displays the last command I typed in.
admin@ xsigo-b[xsigo] set system phone-home [Press TAB]
Possible completions:
disable Disable phone home
enable Enable phone home
noproxy Don’t use HTTP Proxy
proxy HTTP Proxy config
snooze Hit the snooze button
[Optional qualifiers]
-contact-email-address Email address for Xsigo technical support to contact when a problem is discovered. (or ‘none’)
-contact-phone-numbers Telephone number for Xsigo technical support to contact when a problem is discovered. (comma separated, or ‘none’)
-copy-url URL to send audit copy to
-customer-name Customer name (or ‘none’)
-frequency Phone home frequency (relative to when it is set)
-notify Will Xsigo notify you when problems are detected?
-send-alarms Send major alarms to Xsigo?
-strip-private Strip private information from phone-home data
Repeat ‘?’ for detailed help.
admin@xsigo-b[xsigo] set system phone-home
Day 2 Summary..
The pair of Xsigo’s were very easy to configure and install. I enjoyed working with Xsigo CLI, it is very well thought out, and I plan do to write additional blog about it alone.
Besides for the very few and sometime self-inflicted gotchas things went smooth.
It was nice to have a Xsigo SE on site to assist with the initial install and I’m looking forward to tomorrow when we spin up some VM’s and then test!
Still to do…
- Copy vCenter Server & other VM’s from Production to this test environment
- Test, Test, Test and more testing..
Test Lab – The Plan and Layout with Xsigo, juniper, IOMega, vmware, and HP/Dell servers)
This week I have the pleasure of setting up a pretty cool test lab with Xsigo, juniper, IOMega, vmware, and HP/Dell servers.
I’ll be posting up some more information as the days go on…
The idea and approval for the lab came up pretty quickly and we are still defining all the goals we’d like to accomplish.
I’m sure with time the list will grow, however here are the initial goals we laid out.
Goals…
-
Network Goals
- Deploy the vChissis solution by Juniper (Server Core and WAN Core)
- Deploy OSPF Routing (particularly between sites)
- Multicast Testing
- Layer 2 test for vm’s
- throughput Monitoring
-
VMware Goals
- Test EVC from Old Dell QuadCores Servers to new HP Nehalem
- Test Long Distance vMotion & long distance cluster failures from Site1 to Site 2
- Play around with ESXi 4.1
-
Xsigo Goals
- Test Redundant Controller failover with vmware
- Throughput between sites, servers, and storage
Caveats…
- We don’t have a dual storage devices to test SAN replication, however the IOMega will be “spanned” across the metro core
- Even though this is a “Site to Site” design, this is a lab and all equipment is in the same site
- The Simulated 10Gbs Site to Site vChassis Connection is merely a 10Gbs fibre cable (We are working on simulating latency)
- Xsigo recommends 2 controllers per site and DOES NOT recommend this setup for a production enviroment, however this is a test lab — not production.
The Hardware..
2 x Xsigo VP780’s with Dual 10Gbs Modules, All Server hardware will be Dual Connected
2 x HP DL360 G6, Single Quad Core Nehalem , 24GB RAM, Infinband DDR HBA, gNic’s for Mgt (Really not needed but nice to have)
2 x Dell Precision Workstation R5400, Dual QuadCore, 16GB RAM, Infiniband DDR HBA, gNic’s for Mgt (Really not needed but nice to have)
6 x Juniper EX4200’s (using Virtual Chassis and Interconnect Stacking Cables)
Using BGInfo to help determine VM Guest partition alignment (Starting offset and allocation unit size)
We use BGInfo quite a bit to help us with basic VM guest information (IP Address, name, HD Space, etc)
Today I was thinking about the partition alignment (Starting offset and allocation unit size) on my Windows guests and wondered about their alignment.
There are programs made for this, some are quite good, but it wouldn’t be more convenient to have it come up with BGinfo?
I thought it would be especially handy because we have multiple templates some are aligned differently, and I could use this as a double check to those programs which correct this issue.
By adding the Starting Offset and volume block size(aka allocation unit size) to our BGinfo.bat file we can now see how are VM guest partitions are aligned!
From this screenshot we see 3 Volumes..
C: has a 32K Offset / 4K Allocation | D: 64K offset / 32K Allocation | E: 64K offset / 4K Allocation
The only volume that is optimal is our D: volume, the rest need some work…
Want to know more about alignment?
Go here >> http://www.vmware.com/pdf/esx3_partition_align.pdf and here >> http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.0.pdf
Here’s how to setup BGInfo…
- Download BGInfo from sysinsterals
- Create your *.bgi file (this file contains all the fields you want displayed, see below for the offset)
- Create a simple batch file to launch your *.bgi file on logon
- @echo off
- cd\
- CALL “C:\Files\bginfo.exe” “C:\Files\YOUR.bgi” /timer:0 /nolicprompt
-
Place all the files in a simple folder like c:\Files
-
Files Needed
- Bginfo.exe
- YOUR.bgi
- Startbgi.bat
-
- In Windows 2003 – Copy Startbgi.bat into “c:\Documents and Settings\All Users\Start Menu\Programs\Startup”
Every time you logon to the server, bginfo will launch and update your background with useful information..
Once you have created your basic *bgi file, then it’s time to add the WMI Query for Starting Offset and volume block size(aka allocation unit size)
- In BGInfo click on Custom, then New, Enter a name under Identifier, Choose WMI Query, then click on Browse
- Under WMI Class Choose Win32_DiskPatition and under Class Property choose StartingOffset
- Save it up and add it to your *.bgi file
Follow the same steps as above only this time…
Under WMI Class Choose Win32_Volume and under Class Property choose BlockSize (aka allocation unit size)
I hope this works for you as well as it did for me!
Enjoy!
Matt..
Working with the IOMega ix12-300r
I installed an IOMega ix12-300r for our ESX test lab and I must say it’s just as feature rich as my personal ix4 and ix2.
I enjoy working with this device for its simplicity and feature depth. It’s very easy to deploy and it’s a snap to integrate with ESX.
Here are some of the things I like about ix12 and a high level overview to enable it with esx.
Note: Keep in mind most of the
features below are available on the ix2 and ix4 line but not all..
See http://iomega.com/nas/us-nas-comp.html for more information about the ix line and their features…
The Drives…
Our ix12 (the ix## is the amount of possible drives in the unit, ie ix2 = 2 drives, ix4 = 4drives) is populated with 8 x 1TB drives.
By default the 8TB unit will come with 4 x 2TB drives, I opted to buy a 4TB unit and expand it by 4TB, giving us the 8 x 1TB drives.
The drives are Seagate Barracuda Green SATA 3Gb/s 1TB Hard Drive – ST31000520AS – SATA II (Rev 2.6 Drives) 5.9K RPM, they should perform nicely for our environment…
(Buts like most techies, I wish they were faster)
More information here about the drives and SATA 2.6 vs 3.x
http://www.serialata.org/documents/SATA-6-Gbs-The-Path-from-3gbs-to-6gbs.pdf
Storage Pools…
A storage pool is not a new concept but in a device this cost effective it’s unheard of.
Basically, I’m dividing up my 8 drives like this..
Storage Pool 0 (SP0) 4 Drives for basic file shares (CIFS)
Storage Pool 1 (SP1_NFS) 2 drives for ESX NFS Shares only
Storage Pool 2 (SP2_iSCSI) 2 drives dedicated for ESX iSCSI only
I could have placed all 8 drives into one Storage pool but…
One of our requirements was to have SP0 isolated from SP1 and SP2 for separation reasons…
NO Down time for RAID Expansion… Sweet…
Another great feature is NO down time to expand your RAID5 Set..
Simply edit the Storage pool, Choose your new drive, and click apply.
The Raid set will rebuild and you’re all done!
Note: the downside to this… If you decide to remove a drive from a RAID set, you’ll have to rebuild the entire set.
TIP: To check the status of your RAID reconstruction check on the Dashboard under status or the home page at the bottom.
Mine reconstructed the 3 Storage Pools or all 12 drives at the same time in about 4.5 hours…
Teaming your NIC’s!
The ix12 comes with 4 x 1gb NICS, these can be bonded together, stay separate, or a mix of both.
You can setup your bonded NICs as Adaptive Load Balancing, Link Aggregation (LG), or Failover modes.
In our case we bonded NIC 3 and 4 with LG for ESX NFS/iSCSI Traffic and set NIC 1 up for our CIFS traffic.
For the most part setting up the networking is simple and easy to do.
Simply enter your IP’s, choose to bond or not and click apply.
Note: Don’t uncheck DHCP from unused adapters, if you do you’ll get an invalid IP address error when you click apply.
Also, making changes to the network area, usually requires a reboot of the device.. Tip: Setup your Network First..
Adding the NFS Folder to your ESX server
Note: These steps assume you completed the Iomega installation (Enabled iSCSI, NFS, Files shares,etc), networking, and your ESX Environment…
From the ix12 web interface simply add a folder on the correct Storage pool.
In our case I choose the folder name of ESX_NFS and the SP1_NFS storage pool
Tip: ALL Folders are broadcasted on all networks and protocols… I haven’t found a way to isolate folders to specific networks or protocols.
If needed make sure your security is enabled… I plan to talk with IOMega about this…
In vCenter Server, Add NAS storage and point it to the ix12.
Note: use /nfs/[folder name] for the folder name…
Once it’s connected it will show up as a NFS Data store!
Adding iSCSI to your ESX Server..
Note: This assumes you setup your esx environment to support iSCSI with the ix12…
Add your shared storage as an iSCSI Drive, set your iSCSI Drive name, and Select the correct Storage Pool.
Next is to set the Size of the iSCSI device, in this case we have 922GB free, but can only allocate 921.5GB
After clicking on apply, you should see the information screen…
In vCenter Server ensure you can see the iSCSI drive..
Add the iSCSI disk…
Give this disk a name…
Choose the right block size…
Finally there she is… one 920GB iSCSI disk…
Summary…
From a price vs. performance stand point the IOMega line of NAS devices (ix2, ix4, and our ix12) simply ROCK.
It will be hard to find such a feature rich product that will cost you so little.
This post has merely scratched the features of these devices. It is really hard to believe that 10+ years ago Iomega was known only for ZIP and Jazz Drives…
There new logo is IOMega Kicks NAS, and from what I’ve seen they do!
Follow up posts…
Over the next couple of months I hope to performance test my VM’s against the ix12
I’d like to figure out their protocol multi tendency issue (CIFS, NFS, iSCSI broadcasting over all NICS)
I’ll post of the results as they come in..
