HP c7000 Mid-Plane Replacement

Posted on Updated on

 If you’re reading this blog, I’m hoping you never have to do this procedure. However the other day we had a c7000 Mid-plane go out.

Our chassis started to fail the entire Ethernet pass through module, it came back online and then a few weeks later just one port failed.  We tried all the troubleshooting steps and found it was the midplane.

I decided to take a few notes and photos about the event in hopes to alleviate any pain points next time (hopefully there isn’t a next time).

I worked with an HP certified technician and he had a punch list from HP to complete this repair.

I recommend a certified technician do the repair as you never know what you’re going to run into.

Quick Disclaimer – This is not a guide for repairing this device but merely my notes…

Fist thing.. Plan the outage…

7:00    Migrate VM’s to BC2 (BC = blade chassis) and shut down ESX hosts on BC1

8:00    Shutdown remaining blades in BC1

8:20    Start Repair on BC1

8:40    Test repair on BC1

8:50    bring up Blades on BC1

9:00    TEST applications

9:10    Migrate VM’s from BC2 to BC1

10:00    Finish

Here are the events and my notes…

7:00    Migrate VM’s to BC2 (BC = blade chassis) and shut down ESX hosts in BC1

  • No issues, vMotion worked without issues & shutdown 6 ESX hosts via vCenter Server

 

8:00    Shutdown remaining blades in BC1

  • No issues, shut down blade servers

 

8:20    Start Repair on BC1

Power down the unit, remove all the cards, power supplies and blades

Removal of the rear cards…  One thing we were able to do was remove the Ethernet & Fibre Pass through cards without having to disconnect the cables.

Second shot of the removal…  

 Remove the blades and Power supplies..

In the rear unscrew the four bolts holding in the mid-plane…

Pull it till it stops, then Push in the retaining clips, and remove it..  

Here is a shot of the retaining clips. They help to hold the mid-plane in…

It held us up a bit as the instructions failed to mention them.

Inside the blade chassis – Rear

After removing the mid-plane chassis there are several bolts that hold in this together.  One item you have to do is remove the foam tape in the screen shot below. 

This will allow you to separate the mid-plane board from the chassis. This was another gotcha that the instructions failed to mention this step.

Our package came with replacement tape, your may vary… 

 After replacing the mid-plane board we reassemble the chassis in this order..

Reinserted the Mid-Plane chassis

ilo Control Modules

Power Supplies

Powered on device, wait….

Inserted pass though modules, wait…

Inserted Blades and powered them on… 

9:00    Bring up Blades on BC1

  • No issues, everything is now working!

     

9:15     TEST applications

  • No issues, all blades and apps came right up

     

9:30    Migrate VM’s from BC2 to BC1

  • No Issues, all VM’s migrated perfectly

10:00    Finish

 

 

Gotchas / Notes

2 Items slowed us down…

  1. The retaining clips on the lower right and left hand sides were not noted in the documentation. We had lots of cables on the left and right side so it made it hard to clearly see where they were.
  2. The tape holding on the mid-plane board wasn’t documented. It took us a bit to figure out that a simple piece of foam tape could hold together, but it did.

 

Next time I mount a c7000 chassis I’ll remember to mount it about 4U from the base of a rack. The issue we were having was the cabling and PDU power cords were getting in the way of the mid-plane and this made it harder to remove.

Other than these minor issues, the repair went smooth, and it was fun to see the “guts” of a c7000.

2 thoughts on “HP c7000 Mid-Plane Replacement

    Tarek Itani said:
    March 13, 2012 at 10:30 am

    What was the reason for the failure, did you ever get to know?

    Thanks

    Like

      mattman555 said:
      March 15, 2012 at 4:45 am

      This c7000 had various issues, mainly the blades would have issues, like NICs would go missing or go down and we’d have to reboot the entire center to bring it back online. We worked with HP and our local partner to id the mid-plane and we repaired it from there.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s