Salesforce customers fume over CRM outage blunder

Salesforce – one of the biggest suppliers of on-demand marketing software in the world – has been hit by a barrage of customer complaints following a major outage of its CRM systems which started on Friday last week and has yet to be completely fixed.
The incident began at about 6pm BST on Friday after Salesforce deployed a database script that inadvertently granted all users full access to read and modify all of a company’s files.
The issue only directly affected users of Pardot, a marketing automation tool purchased in 2012, but Salesforce decided to pull the plug on access to more than 100 cloud servers that host Pardot users. Users in Europe and North America were the worst affected, Salesforce said.
By Saturday, the company claimed to have fixed the issue, but four days later customers continue to experience disruption to cloud-based services.
In a conference call on Monday, Salesforce senior vice-president of engineering Anmol Bhasin admitted the company was still dealing with “thousands” of trouble reports.
He said: “On the last customer update, I had communicated that the initial fix that we had put in place for restoring functionality for the pre-incident state – restoring permission sets in particular – which we believe should have restored functionality for all the affected organisations was not successful in doing so.”
He apologised for the disruption, and offered his assurance that Salesforce is focused on fixing things at the highest level of the company and has devoted all available engineering resources to resolving the issue.
In an email to The Register website, Blue Canvas chief technology officer Alex Brausewetter wrote: “It’s completely bonkers! From what we gather there are still hundreds if not thousands of customers affected. In one earlier call, they said they received thousands of complaints/support tickets after they ran the scripts that they thought would fix this issue. Salesforce has gone radio silent since yesterday night… there’s been no public communication.”