Major Outage for Salesforce.com

Major Outage for Salesforce.com

• By: Rich Miller
  July 10th, 2012
•  
•  
•  
• inShare
• 

Salesforce.com is experiencing a significant outage today, leaving many customers unable to access the company’s services. The downtime was triggered by a power outage at anEquinix data center in Silicon Valley.

The disruption began just before 1 a.m. Pacific time, and Salesforce.com has been gradually restoring service throughout the day. The incident has affected at least six of the company’s regions, including three production instances in North America and at least three “sandbox”  instances used primarily for testing and development.

At 10:30 a.m. Pacific time, Salesforce reported that had resolved most of the login and dashboard access issues at its production instances. However, it said it was “working to fail the CS12 instance over to its DR datacenter,” suggesting a problem that can’t be quickly addressed at its current site. The company estimated service on that particular sandbox instance would be restored by Noon Pacific time.

Salesforce.com said the service disruption was due to “power issues.” In an update on its status dashboard, the company said it has “confirmed this has been fixed,” but customers continued to experience problems logging in and accessing data and services.

Equinix, which hosts most of Salesforce.com’s infrastructure,  acknowledged that it had a brief power failure.

“This morning Equinix experienced a power issue that lasted about one minute and affected approximately one quarter of one Silicon Valley data center,” the company said in a statement. “Although our recovery was swift, recovery time for individual customers in our data centers varies considerably, depending upon their specific deployment configurations. We deeply regret the impact and inconvenience this incident caused our customers.”

This is the second power loss at an Equinix facility this year. In January, a power failure at an Equinix data center in Silicon Valley caused downtime for Zoho. Such outages are rare for Equinix, which reports average uptime of “six nines” – 99.9999 percent – across its global platform of more than 100 data centers. Six nines works out to 31.5 seconds of downtime over the course of a year.

The outage was the second in the past month for Salesforce.com, which provides enterprise cloud and social media services to  more than 100,000 customers. The company had downtime on June 12 that was blamed on a fault in a storage tier, rather than power problems.

Salesforce.com has rapidly expanding its infrastructure to manage its growth, and plans asignificant expansion of its colocation footprint in the next several years. Salesforce.com currently operates eight data centers around the U.S. and Asia spanning about 100,000 square feet of space, with two data centers apiece in Chicago and northern Virginia, along with facilities in San Francisco, San Jose, Tokyo and Singapore. The company expects to add 100,000 square feet of colocation space this year.