Friday, August 13, 2010

How to Prevent Downtime Due to Human Error

Great post today over at Datacenter Knowledge, citing the fact that "70 percent of the problems that plague data centers" are caused by human error. Below are the best practices to avoid data center failure by human error:

1. Shielding Emergency OFF Buttons – Emergency Power Off (EPO) buttons are generally located near doorways in the data center. Often, these buttons are not covered or labeled, and are mistakenly shut off during an emergency, which shuts down power to the entire data center. Labeling and covering EPO buttons can prevent someone from accidentally pushing the button. See Averting Disaster with the EPO Button and Best Label Ever for an EPO Button for more on this topic.
2. Documented Method of Procedure - A documented step-by-step, task-oriented procedure mitigates or eliminates the risk associated with performing maintenance. Don’t limit the procedure to one vendor, and ensure back-up plans are included in case of unforeseen events.
3. Correct Component Labeling - To correctly and safely operate a power system, all switching devices must be labeled correctly, as well as the facility one-line diagram to ensure correct sequence of operation. Procedures should be in place to double check device labeling.
4. Consistent Operating Practices – Sometimes data center managers get too comfortable and don’t follow procedures, forget or skip steps, or perform the procedure from memory and inadvertently shut down the wrong equipment. It is critical to keep all operational procedures up to date and follow the instructions to operate the system.
5. Ongoing Personnel Training – Ensure all individuals with access to the data center, including IT, emergency, security and facility personnel, have basic knowledge of equipment so that it’s not shut down by mistake.
6. Secure Access Policies – Organizations without data center sign-in policies run the risk of security breaches. Having a sign-in policy that requires an escort for visitors, such as vendors, will enable data center managers to know who is entering and exiting the facility at all times.
7. Enforcing Food/Drinks Policies – Liquids pose the greatest risk for shorting out critical computer components. The best way to communicate your data center’s food/drink policy is to post a sign outside the door that states what the policy is, and how vigorously the policy is enforced.
8. Avoiding Contaminants – Poor indoor air quality can cause unwanted dust particles and debris to enter servers and other IT infrastructure. Much of the problem can be alleviated by having all personnel who access the data center wear antistatic booties, or by placing a mat outside the data center. This includes packing and unpacking equipment outside the data center. Moving equipment inside the data center increases the chances that fibers from boxes and skids will end up in server racks and other IT infrastructure.

3 comments:

  1. In a previous gig, I had a sign I would post in the computer lab that was taken from an episode of The Tick. It read:

    Do Not:
    - Eat in lab
    - Set lab on fire
    - Innovate unnecessarily

    Good advice for all.

    ReplyDelete
  2. Coach Outlet http://www.coachstoreoutlet.us.org

    Christian Louboutin Shoes http://www.christianlouboutinshoe.eu.com

    Valentino Shoes http://www.valentinoshoes.eu.com

    Michael Kors Outlet http://www.michaelkorsoutlet.us.org

    Coach Factory Outlet http://www.coachfactoryoutlet.us.org

    Coach Outlet Online http://www.coachoutletonlinestore.us.org

    Coach Purses http://www.coachpurse.us.com

    Kate Spade Outlet http://www.katespadefactoryoutlet.us.com

    Toms Shoes http://www.toms-shoes.name

    Hermes Belts http://www.hermesbelts.us.org

    Louis Vuitton http://www.louisvuitton-outletstores.us.com

    Fendi Handbags http://www.fendibelt.us.com

    Giuseppe Shoes http://www.giuseppezanotti.eu.com

    Michael Kors Outlet http://www.michael-kors-outlet.us.com

    Stephen Curry Shoes http://www.stephencurryshoes.us.com

    Salomon Shoes http://www.salomonshoes.eu.com

    North Face Outlet http://www.northfacesoutlet.in.net

    Coach Outlet http://www.outletcoach.in.net

    North Face Outlet http://www.northfacesoutlet.us.com

    Burberry Outlet http://www.burberryoutletstore.us.org

    North Face Outlet http://www.north-faceoutlets.in.net

    North Face Jackets http://www.outletnorthface.in.net

    Skechers Shoes http://www.skechersshoes.us.com

    Toms Outlethttp://www.tomsoutletstore.us.org

    North Face Outlet http://www.northfacestoreoutlet.us.org

    Nike Air Max http://www.nikeoutlet.us.org

    Nike Hoodies http://www.nikehoodies.us.com

    Marc Jacobs Handbags http://www.marcjacobshandbags.name

    Marc Jacobs Outlet http://www.marcjacobsoutletstore.us.com

    Jimmy Choo Shoes http://www.jimmy-chooshoes.us.com

    Jimmy Choos http://www.jimmychoos.in.net

    Burberry Belt http://www.burberrybelt.us.com

    Louis Vuitton Belt http://www.louisvuitton-belt.com

    Salvatore Ferragamo http://www.ferragamo-belt.us.com

    Marc Jacobs Handbags http://marcjacobshandbagsoutlet.blog.com

    Lululemon Outlet http://www.lululemonsoutlet.us.com

    True Religion Outlet http://www.truereligion-outlets.in.net

    Tommy Hilfiger http://www.tommyhilfigeroutlet.us.com

    Michael Kors Outlet http://www.outletmichaelkors.eu.com

    Coach Outlet http://www.outletcoach.eu.com

    Red Bottoms http://www.redbottom.in.net

    Kevin Durant Shoes http://www.kevindurantshoes.us.org

    New Balance Outlet http://www.newbalanceoutlet.in.net

    Adidas Outlet http://www.adidasoutlet.in.net

    Coach Outlet Online http://www.coachoutlet-online.eu.com

    Stephen Curry Jersey http://www.stephencurryjersey.us.com

    Vans Outlet http://www.vansoutlet.us.com

    Ralph Lauren Outlet http://www.ralphlauren-outletstore.in.net

    True Religion Outlet http://www.true-religionoutlets.in.net

    ED Hardy Outlet http://www.edhardyoutlet.in.net

    North Face Outlet http://www.northfaceoutlet.eu.com

    UGG Outlet http://www.uggoutlet.us.org

    UGG Outlet http://www.outletugg.in.net

    North Face Outlet http://www.northface--outlet.us.com

    Ugg Boots Sale http://www.uggbootssale.in.net

    UGGS For Women http://www.uggsforwomen.us.com

    Skechers Go Walk http://www.skechersgowalk.us.com

    Adidas Yeezy Boost http://www.adidasyeezyboost.in.net

    Adidas Yeezy http://www.adidasyeezy.us.com

    Adidas NMD http://www.adidas-nmd.us.org

    Coach Outlet http://www.coachoutletinc.us.com

    ReplyDelete