About the only thing harder than building a data center is dismantling one. The potential for disruption of business is much greater with data center decommissioning than with construction.
A case in point, the decommissioning of the Titan supercomputer at the Oak Ridge National Laboratory (ORNL), reveals just how complicated the process can be. More than 40 people were involved with the project, including staff from ORNL, supercomputer manufacturer Cray, and external subcontractors. Electricians were required to safely shut down the nine megawatt-capacity system. Cray staff was on hand to disassemble and recycle Titan’s electronics and its metal components and cabinets. A separate crew handled the cooling system. In the end, 350 tons of equipment and 10,800 pounds of refrigerant were removed from the site.
Trends in data center decommissioning
While most enterprise IT pros aren’t likely to face decommissioning a computer the size of Titan, they’ll likely be involved with dismantling smaller-scale data centers.
The pace of on-premise data center migration to leased colocation, cloud, or some combination (Hybrid IT) or closure will continue to accelerate in this decade, according to IDC Group Vice President for Worldwide Research, Rick Villars. “Every company we’ve spoken to is planning to close 10% to 50% of their data centers through 2025, and in some cases even 100%. No matter who you talk to, they absolutely have on the agenda they want to close data centers,” Villars says.
Successfully decommissioning a data center requires navigating many steps. Here’s how you can get started.
Inventory data-center assets
The first step is a complete inventory. Given the prevalence of zombie servers in IT environments, it’s evident that a significant amount of IT departments lack a firm grasp on data-center asset management.
“They need to know what they have. That’s the most basic. What equipment do you have? What apps live on what device? And what data lives on each device?” says Ralph Schwarzbach, a former security and decommissioning expert with Verisign and Symantec.
All that information should be in a configuration management database (CMDB) or data center infrastructure management application (DCIM). This type of tool serves as a repository for configuration data about physical and virtual IT assets. DCIM enables a data center to optimize space utilization, equipment usage and energy usage. It does this by providing a complete view of the entire facility performance. Whichever tool you use, “having the tool and processes in place to maintain data accuracy are two distinct things,” Schwarzbach says.
While CMDB or DCIM is a necessity for asset inventory, “any good management system is only as good as the data you put in it,” says Al DeRose, a senior IT director responsible for infrastructure design, implementation, and management at a large media firm. “If your asset management department is very good at entering data, your management tool is extremely valuable. [In] my experience, smaller companies will do a better job of assets. Larger companies, because of the breadth of their space, aren’t so good at knowing what their assets are, but they are getting better.”
Map dependencies among data center resources
Meanwhile, a successful decommission includes mapping out dependencies in the data center. The older a data center is, the more dependencies you are likely to find.
It’s important to segment what’s in the data center so that you can move things in orderly phases and limit the risk of something going wrong, says Andrew Wertkin, chief strategy officer with BlueCat Networks, a networking connectivity provider that helps companies migrate to the cloud. “Ask how can I break this into phases that are independent – meaning ‘I can’t move that app front-end because it depends on this database,’” Wertkin says.
For example, let’s consider a wide area network (WAN). Each connection point is often optimized. When you start to disassemble the network, you need to know who is getting what in terms of connections and optimized services. Otherwise, you risk SLA issues when you break the connection. Changing the IP addresses of well-known servers, even temporarily, also creates connection problems. The solution is to do it one step at a time.
Questions to ask decommissioning providers
Because of the intricacies and labor requirements of decommissioning a data center, it’s critical to hire a specialist.
Experience and track record are everything when it comes to selecting a vendor, says Mike Satter, vice president at OceanTech. There are a lot of small companies that say they can decommission a data center. They fail because they lack experience and credentials, he says. “I can’t tell you how many times we’ve come into a mess where we had to clean up what someone else did. There were servers all over the floor, hardware everywhere,” Satter says.
His advice? Ask a lot of questions.
“I love having a client who asks a lot of questions,” Satter says. “Don’t be shy to ask for references,” he adds. “If you are going to have someone do work on your house, you look up their references. You better know who the contractor will be. Maybe 10% of the time have I had people actually look into their contractor.”
Decommissioning processes
Among the processes you should ask about and conditions you should expect are:
- Have the vendor provide you with a detailed statement of work in every aspect of the data center decommissioning project.
- Ask the vendor to do a walkthrough prior to the project, showing how they will execute each step.
- Find out if the vendor outsources any aspect of data center decommissioning, including labor or data destruction.
- Inquire about responsible recycling (see more below).
- Ask for references for the last three data center decommissioning clients the vendor serviced.
- Will they be able to recover value from your retired IT hardware? If so, find out how much and when you could expect to receive the compensation.
- How will data destruction be handled? If the solution is software-based, find out the name of the software.
- Learn about the vendor’s security protocols around data destruction.
- Find out where the truck goes when it leaves with the gear and whether a tracking app is available.
- What is the disposal procedure for metals, hazardous materials and other components?
Recycle electronics responsibly
As gear is cleared out of the data center, it’s important to make sure it’s disposed of safely, from both a security and environmental standpoint.
When it comes to electronics recycling, the key certification to look for is the R2 Standard, Satter says. R2 – sometimes referred to as the responsible recycling certification – is a standard for electronics recyclers that requires certified companies to have a policy on managing used and end-of-life electronics equipment, components and materials for reuse, recovery and/or recycling.
But R2 does more than that; it offers a traceable chain of custody for all equipment, tracking who touched every piece and its ultimate fate. R2 certified providers “aren’t outsourced Craigslist tech people. These are people who do it every day,” Satter says. “There are techniques to remove that gear. They have a group to do data security on site, and a compliance project manager to make sure compliance is met and the chain of custody is met.”
“And don’t be cheap,” DeRose adds. “When I decommission a data center, I use a well-known company that does asset removal, asset destruction, chain of custody, provides certifications of destruction for hard drives, and proper disposal of toxic materials. All that needs to be very well documented not [only] for the environment’s protection but [also] for the company’s protection. You can’t wake up one morning and find your equipment was found dumped in a landfill or a rainforest,” DeRose says.
Documentation and liability
Documentation is critical when disposing of electronic waste, echoes Schwarzbach. “The process must capture and store info related to devices being decommissioned: What is the intent for the device, recycling, or new service life? What data resides on it? Who owns the data? And [what is] the category of data?”
In the end, it isn’t the liability of the disposal company if servers containing customer or medical information turn up at a used computer fair. It’s the fault of the owners. “The creator of e-waste is ultimately liable for the e-waste,” Schwarzbach says.
Control who’s coming into the data center
Shutting down a data center means one inevitability. You will have to bring in outside consultants to do the bulk of the work, as the ORNL example shows. Chances are, your typical data center doesn’t let anywhere near 40 people inside during normal operations. But during decommissioning, you will have a lot of people going in and out. Do not take this step lightly.
“In a normal scenario, the number of people allowed in the data center is selected. All of a sudden, you got a bunch of contractors coming in to pack and ship, and maybe there are another 50 people with access to your data center. It’s a process and security nightmare if all these people have access to your boxes and require a whole other level of vetting,” Wertkin says. His solution: Log people in and out and use video cameras.
Any company hired to do a decommissioning project needs to clearly identify the people involved, DeRose says. “You need to know who your company is sending, and they need to show ID.” People are to be escorted in and out and never given a keycard. Also, contractors should not be left to decommission any room on their own. He adds that there should always be someone on staff overseeing the process.
In short, a lot of outside, non-staff is being given access to your most sensitive systems. Vigilance is mandatory.
None of the steps involved in a data center decommissioning should be hands-off. Even when it requires outside experts. For the security and integrity of your data, the IT staff must be fully involved at all times. When millions of dollars (even depreciated) of server gear goes out the door in the hands of non-employees, your direct involvement is paramount.
