Test Lab – Day 2 CLI with the Xsigo!

Posted on Updated on

Yesterday I did about 90% of the hardware install. Today, Day 2, our Xsigo SE will be here to assist with the installation and configuration of the Xsigo to the ESX Hosts..

Today’s Goals..

  • Install 2nd Xsigo VP780
  • Install vmware ESXi 4.1 on 4 servers with Xsigo Drivers
  • Configure both Xsigo vp780’s

 

Install 2nd Xsigo VP780…

Day 2 started out with a gift from Mr. FedEx, it was the parts we needed to install the 2nd Xsigo. Only yesterday afternoon we discovered we were missing some power cords and mounting brackets. A couple quick calls to Xsigo and viola parts are on their way. Props to Xsigo for a VERY quick response to this issue!

Based on the lessons learned from Day 1 we mounted the 2nd Xsigo VP780 and it went much smoother. Notice the WE part of installing the VP780, these things are heavy & large and you’ll need some help or a giant with huge hands to install them into a rack. See their install manual for more information.

When we powered them up I was amazed by the amount of air they moved through the device >> Very NICE!

Keep in mind at this point all the test lab hardware including the Xsigo fiber modules (2 x 10gig Fiber modules per device), and networking is mounted and interconnected…

 

Install vmware ESXi 4.1 on 4 servers with Xsigo Drivers…

You’ll need the Xsigo Drivers installed for ESXi to recognize the infiniband cards and for proper communication.

There are two installation options…

  1. Install ESXi 4.1 and add the Xsigo Drivers after the install.
  2. Download the drivers and re-master the ESXi ISO yourself (This is a good option if your building / rebuilding lots of servers)

We chose to re-master the ESXi ISO with the Xsigo drivers.

Here is the link to master the ISO

I won’t bore you with the details of installing ESXi, however the only gotcha I ran into was the Dell R5400 SATA RAID controller.

I setup a SATA RAID group, during the ESXi install it recognized the RAID volume, and ESXi installed to it without issue.

However after the reboot of the host it would not boot to this volume.

I didn’t have time to troubleshoot, for now we just broke the RAID group, reinstalled, and it worked perfectly.

ESXi Management NICS’s..

Our test lab network will be isolated from production network traffic. However, one of our servers will need to be in the production environment. We setup one physical NIC (pNIC) on to our production environment. This will allow us to temporarily transfer VM’s from production to test, we’ll then disconnect this pNIC and setup ESXi to use the Xsigo NIC for management.

(More to come on this on Day 3)

 

Configure both Xsigo vp780’s…

Configuring the vp780 was very simple. We attached a laptop to the Xsigo and in about 20 commands our Xsigo was up and running..

These are the basic commands we used to setup our pair of Xsigo’s (A and B), the commands below reflect B only.

The commands would be the same for the A Xsigo simply change the appropriate parameters…

NOTE: I don’t recommend you execute these commands in your environment, keep in mind these are for my reference ONLY… I also recommend you contact your Xsigo representative for assistance.

 

Here are the commands we executed..

 

Getting into the Xsigo VP780…

We used a standard Xsigo provided rollover cable plugged into Serial1. (Serial2 is for Tech / Debug – Don’t use)

We connected to the console via Putty or Absolute Telnet (COM Settings are 115200,8,1,None,None)

Tip: All default passwords are in the CLI Config Guide by Xsigo

 

Setup the Xsigo via the Wizard…

Once the connected we used the XgOS config Wizard and entered in the following..

Welcome to XgOS

Copyright (c) 2007-2010 Xsigo Systems, Inc. All rights reserved.

 

Enter “help” for information on available commands.

 

Would you like to use the XgOS Configuration Wizard? [Y/n]

Hostname: xsigo-b

Domain: YOURDOMAIN.COM

Is this Director to be designated as the IB subnet manager (leave as Y unless using an external, non-Xsigo subnet manager) ? [Y/n]

Do you want this Director to send diagnostic data to Xsigo periodically? [Y/n]

Please input the ‘root’ password: ****

Please confirm the ‘root’ password: ****

Please input the ‘admin’ password: *****

Please confirm the ‘admin’ password: *****

Please input the ‘recovery-password’: ****

Please confirm the ‘recovery-password’: ****

IP Address [static/DHCP]: 555.555.555.555

IP Address [static/DHCP]:

Enter NTP Server 1: 555.555.555.555

Enter NTP Server 2:

Enter Timezone [<Tab><Tab> for the list of Timezones]: America_Phoenix

Welcome to XgOS

Copyright (c) 2007-2010 Xsigo Systems, Inc. All rights reserved.

 

Enter “help” for information on available commands.

admin@xsigo-b[xsigo]

 

Now it’s time to setup the Xsigo…

Place the Xsigo into Trunk Mode..

Port 10 and Port 11 are the 10gig Fibre Modules; this command places them in Trunk Mode

set ethernet-port 10/1 -mode=trunk << Port 10 will be used for our IP Network (Vlans for Guests, vmotion, hosts, etc)

set ethernet-port 11/1 -mode=trunk << Port 11 will be used for our NFS

Rear of VP780

Ensure Trunk Mode is activated..

Use the command ‘show ethernet-port ‘

admin@xsigo-b[xsigo] show ethernet-port

 

name type state descr mode flags lag access-vlan vnics vlans

——————————————————————————-

10/1 nwEthernet10GbPort up/up trunk -s— 1 0 none

11/1 nwEthernet10GbPort up/up trunk -s— 1 0 none

2 records displayed

 

Setup Phone Home for Support…

set system phone-home -customer-name=”YOUR COMPANY NAME HERE”

set system phone-home -contact-email-address=YOURNAME@YOURDOMAIN.COM

set system phone-home -contact-phone-numbers=”555-555-5555″

set system phone-home proxy [YOUR PROXY IP HERE] [PROXY PORT if needed, default is 3128]

Note: For this command the syntax is [PROXY IP Address] one space [PROXY PORT], don’t use ‘:’ to as the separator.

 

Once completed then check confirm your information…

Enter the command ‘show system phone-home’

admin@xsigo-b[xsigo] show system phone-home

——————————————————————————-

enabled true

freq weekly

next Fri Jan 14 12:44:52 MST 2011

notify no

strip yes

alarm yes

name COMPANYNAME

email EMAIL@EMAIL.com

phone 5555555555

copy

p-host 555.555.555.555:3128

p-user

——————————————————————————-

1 record displayed

admin@xsigo-b[xsigo]

 

Check on the Phone Home Log….

admin@xsigo-b[xsigo] showlog phonehome.log

Wed Jan 5 17:30:33 MST 2011: Phone home successful to http://phone-home.xsigo.com:6522

Wed Jan 5 18:04:14 MST 2011: Phone home successful to http://phone-home.xsigo.com:6522

Wed Jan 5 18:04:38 MST 2011: Phone home successful to http://phone-home.xsigo.com:6522

[Press CRTL-C to Exit]

admin@xsigo-b[xsigo]

Tip: your log might be empty until it has something to send

 

Ensure your Physical servers are attached…

As expected all 4 servers are attached to this Xsigo.. (If they don’t show up here it could be an interconnect or ESXi issue)

Enter the command ‘show physical-server’ to view your connected servers.

admin@xsigo-b[xsigo] show physical-server

——————————————————————————-

name localhost <<< This is the ESXi Hostname

guid 2c903000b4df5

descr

port xsigo-001397001:ServerPort2 << This is the Xsigo Port the Server is connected to

os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64 << This is the version of ESX & Xsigo Driver

version 2.7.0/3.0.0

server-profile << Notice this is blank, We configured it next

——————————————————————————-

name localhost

guid 2c903000b4ea5

descr

port xsigo-001397001:ServerPort3

os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64

version 2.7.0/3.0.0

server-profile

——————————————————————————-

name localhost

guid 2c903000b4ea9

descr

port xsigo-001397001:ServerPort4

os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64

version 2.7.0/3.0.0

server-profile

——————————————————————————-

name localhost

guid 2c903000b5095

descr

port xsigo-001397001:ServerPort1

os VMware/ESXi-4.1.0:xg-3.5.0-1-246491/x86_64

version 2.7.0/3.0.0

server-profile

——————————————————————————-

4 records displayed

 

Create Server Profiles…

Creating a server profile enables you to assign devices to your specific host.

In our case we used the ESX Hostname as the Xsigo Server Profile name.

This will help us to keep the profiles well organized.

Keep in mind YOURSERVERNAME# equals your ESX Hostname and it will become your Xsigo Server Profile Name…

Long way to create a Server Profile…

add server-profile [server profile name]

View the new server profile…

admin@xsigo-b[xsigo] show server-profile

name state descr connection def-gw vnics vhbas

——————————————————————————-

YOURSERVER1 up/unassigned 0 0

1 record displayed

 

Assign the server profile to a port on the Xsigo…

set server-profile YOURSERVER1 connect localhost@xsigo-001397001:ServerPort1

 

Short way to create a Server Profile…

add server-profile YOURSERVER2 localhost@xsigo-001397001:ServerPort2

add server-profile YOURSERVER1 localhost@xsigo-001397001:ServerPort3

add server-profile YOURSERVER1 localhost@xsigo-001397001:ServerPort4

 

Then use show server-profile to confirm your entries…

admin@xsigo-b[xsigo] show server-profile

name state descr connection def-gw vnics vhbas

——————————————————————————-

Yourserver3 up/up localhost@xsigo-001397001:ServerPort3 0 0

Yourserver4 up/up localhost@xsigo-001397001:ServerPort4 0 0

Yourserver1 up/up localhost@xsigo-001397001:ServerPort1 0 0

Yourserver2 up/up localhost@xsigo-001397001:ServerPort2 0 0

4 records displayed

admin@xsigo-b[xsigo]

 

 

Set Up and attach the virtual NICS to your server profile…

In this step we created our Xsigo vNICS, attached them to the appropriate server profiles, and the 10gig Modules.

When complete each of our ESXi servers will have 4 Xsigo vNICS.

(2 vNICs for IP Network, 2 vNICs for Storage network)

 

Decoding the command…

The command ‘add vnic xnb.yourservername1 10/1 -mode=trunk’ breaks down to…

add vnic << Add vNIC Command

xnb << The vNIC Name (xnb = Xsigo, IP Network, B Xsigo Device, Xsb = Xsigo, Storage Network, B Xsigo Device)

yourservername1 << Which profile to attach to

10/1 << Which Module on the Xsigo to attach to

-mode=trunk << What transport mode

These are the command we entered..

IP Network vNICS

admin@xsigo-b[xsigo] add vnic xnb.yourservername1 10/1 -mode=trunk

admin@xsigo-b[xsigo] add vnic xnb.yourservername2 10/1 -mode=trunk

admin@xsigo-b[xsigo] add vnic xnb.Yourservername3 10/1 -mode=trunk

admin@xsigo-b[xsigo] add vnic xnb.Yourservername4 10/1 -mode=trunk

 

Storage vNICS

admin@xsigo-b[xsigo] add vnic xsb.Yourservername1 11/1 -mode=trunk

admin@xsigo-b[xsigo] add vnic xsb.Yourservername2 11/1 -mode=trunk

admin@xsigo-b[xsigo] add vnic xsb.Yourservername3 11/1 -mode=trunk

admin@xsigo-b[xsigo] add vnic xsb.Yourservername4 11/1 -mode=trunk

 

Results from ESXi…

 

Other Information…

 

Set System back to factory Defaults…

If needed, you can set the System back to factory Defaults by the following command.

When complete you will need to access the system via Serial Cable.

Here are the steps:

set system factory-default

confirm << type in Confirm, my Putty will exited and the system will shutdown

NOTE: This command will erase the configuration from the Xsigo. Do it with caution

Tip: Note this will cause the system to shutdown, this means someone will have to manually power it back on.

 

Upgrade the XgOS via USB…

Download the GOS 2.8.5 to a USB Stick..

We inserted the stick into the USB Port on the VP780, then executed this command

system upgrade file://usb/xsigo-2.8.5.xpf

 

Other Handy commands…

show system status

show system

show system version

show system warnings

show serial

show system info

history

 

CLI Fun…

One thing I like about the CLI for Xsigo is TAB at the end of the command (most modern CLI’s have this and it sure is handy)

If I type in set system phone-home[Press TAB] it displays possible completions and qualifiers and then it displays the last command I typed in.

admin@ xsigo-b[xsigo] set system phone-home [Press TAB]

Possible completions:

disable Disable phone home

enable Enable phone home

noproxy Don’t use HTTP Proxy

proxy HTTP Proxy config

snooze Hit the snooze button

[Optional qualifiers]

-contact-email-address Email address for Xsigo technical support to contact when a problem is discovered. (or ‘none’)

-contact-phone-numbers Telephone number for Xsigo technical support to contact when a problem is discovered. (comma separated, or ‘none’)

-copy-url URL to send audit copy to

-customer-name Customer name (or ‘none’)

-frequency Phone home frequency (relative to when it is set)

-notify Will Xsigo notify you when problems are detected?

-send-alarms Send major alarms to Xsigo?

-strip-private Strip private information from phone-home data

Repeat ‘?’ for detailed help.

admin@xsigo-b[xsigo] set system phone-home

 

Day 2 Summary..

The pair of Xsigo’s were very easy to configure and install. I enjoyed working with Xsigo CLI, it is very well thought out, and I plan do to write additional blog about it alone.

Besides for the very few and sometime self-inflicted gotchas things went smooth.

It was nice to have a Xsigo SE on site to assist with the initial install and I’m looking forward to tomorrow when we spin up some VM’s and then test!

 

Still to do…

  • Copy vCenter Server & other VM’s from Production to this test environment
  • Test, Test, Test and more testing..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s