An example of Colocation vs Cloud for High-Performance Compute (HPC)
SEO Firm, Ahrefs, finds cloud to be cost-prohibitive for its high density computing needs. Ahrefs estimates it would cost $400 million over three years if it were to switch from its colocation environment to AWS.
In the fast-paced world of technology, where computational demands are escalating, the choice between colocation and cloud infrastructure is pivotal for companies that rely on high-performance computing (HPC) to power their operations. The debate over cost-effectiveness and performance optimization has been vividly highlighted by the case of Ahrefs, a prominent software firm based in Singapore specializing in search engine optimization tools. This case exemplifies how the nuances of a company’s computing environment can influence the feasibility of migrating from colocation to the cloud, especially in the context of high-density computing.
The Cost Dilemma: Colocation vs. Cloud
Ahrefs, a player in the SEO industry, found itself grappling with the decision of whether to transition its compute-intensive workload from its existing colocation setup to Amazon Web Services (AWS) cloud. In a revealing blog post authored by Efim Mirochnik, a data center operations executive at Ahrefs, the company estimated that the transition to AWS would incur a staggering cost of over $400 million over a three-year period. This eye-popping figure underscored the challenges of duplicating a high-density computing environment within a cloud framework.
At the heart of this cost analysis was a comparison between Ahrefs’ colocation environment and the equivalent configuration on AWS. In colocation, a company purchases its own IT equipment and outsources data center hosting to a colocation provider. This approach allows companies to manage their own IT systems while offloading tasks like power management and cooling. However, the allure of cloud services, often touted for their scalability and convenience, seemed to falter when confronted with the realities of high-density computing.
The Technical Complexities: Unique Hardware and Configuration
A crucial factor in Ahrefs’ decision was the distinct hardware configuration of its servers. The company’s hardware boasted high core-count CPUs, 2TB of RAM, 2x 100Gbps per server, and a whopping 16 15TB drives on average. Such specifications were notably absent from available cloud service providers, including AWS. As a result, Mirochnik’s analysis compared Ahrefs’ servers to Amazon Elastic Compute Cloud (EC2) instances. However, it was acknowledged that this comparison was not entirely equivalent due to the hardware disparities.
Additionally, Ahrefs’ storage architecture, including the use of NVMe drives, played a significant role in the cost breakdown. The AWS Elastic Block Storage (EBS) used for comparison, though cheaper, couldn’t match the speed of Ahrefs’ NVMe drives. The higher cost of AWS storage, coupled with the lack of an EC2 instance that truly replicated Ahrefs’ hardware, further contributed to the financial imbalance between the colocation and cloud options.
The Verdict: Colocation Triumphs for HPC
In a stark juxtaposition, Ahrefs concluded that a cloud migration would be economically unfeasible. The cost of running its own 850 servers in a colocation data center was estimated at $1.3 million per month, reaching $39.5 million over 30 months. In contrast, AWS EC2 instances with an equivalent computing capacity were projected to cost $14.9 million per month, totaling a staggering $447.7 million over the same period. This cost discrepancy was exacerbated by the fact that AWS couldn’t provide a comparable hardware setup and that storage solutions fell short of Ahrefs’ requirements.
The case of Ahrefs echoed the sentiment of industry experts who stressed that HPC environments might not be well-suited for cloud infrastructure. Ashish Nadkarni, a group vice president and general manager at IDC, emphasized that while public cloud services have their place, sustained high-performance computing is not their forte. The complexities of premium products and services, coupled with the specialized nature of HPC workloads, can render cloud environments cost-prohibitive.
This table provides a snapshot of the cost comparisons between Ahrefs’ colocation environment and an equivalent configuration on AWS for their high-performance computing (HPC) workload. It highlights the substantial cost disparity and underscores the challenges in replicating specialized hardware and performance characteristics in a cloud environment.
| Cost Comparison | Colocation | AWS Cloud |
|---|---|---|
| Cost per Server per Month | $1,550 | $17,557 |
| Ahrefs’ Hardware Specification | High core-count CPUs, 2TB RAM, 2x 100Gbps per server, 16x 15TB drives | N/A |
| Storage Equivalency | NVMe drives | AWS Elastic Block Storage (EBS) |
| Monthly EBS Cost | N/A | $11,486 |
| EC2 Instances Equivalent | N/A | 2x Amazon EC2 Instances |
| Monthly EC2 Instances Cost | N/A | $5,606 |
| Data Transfer Costs | N/A | $464 |
| Total Monthly Cost | $1.3 million | $14.9 million |
| Total Cost over 30 Months | $39.5 million | $447.7 million |
| Ahrefs’ Revenue (30 Months) | $257 million | N/A |
Balancing Performance and Economy
The Ahrefs case serves as a striking example of the intricate balance that companies must strike between performance and economy in the realm of HPC. While cloud computing offers undeniable advantages in scalability and flexibility, it may not always align with the needs of compute-intensive workloads. The key takeaway is that the decision between colocation and cloud should be rooted in a thorough understanding of a company’s unique hardware, workload demands, and operational characteristics. As Ahrefs has demonstrated, a one-size-fits-all approach does not suffice when dealing with the intricacies of high-density computing.
