A Gartner analyst has posted “a few thoughts about Amazon and the enterprise”. She starts by blasting SAS 70 for weakness and then holds up yesterday’s Amazon ISO/IEC 27001 certification announcement as a totally different standard.

I too am a fan of the ISO process and have used it for many years with many organizations. It is good news that Amazon has chosen to certify to this new standard. I also am familiar with criticisms of SAS 70 (some of which have been addressed in the new standard, SSAE 16, as I have mentioned before).

Unfortunately, the analyst at Gartner makes some glaring logical errors in her analysis of the Amazon announcement and cloud compliance. She gets it wrong and is misleading readers. Note her criticism of SAS 70:

To start with, SAS 70 Is Not Proof of Security, Continuity or Privacy Compliance (Gartner clients only). As my security colleagues Jay Heiser and French Caldwell put it, “The SAS 70 auditing report is widely misused by service providers that find it convenient to mischaracterize the program as being a form of security certification. Gartner considers this to be a deceptive and harmful practice.”

The ISO runs the same risk. Here is why:

The ISO 27001 compliance certificate gives assurance only that a management system for information security (ISMS) is in place. It does not provide a report on information security controls within the organization. That is like saying it also may not give “Proof of Security, Continuity or Privacy Compliance”.

Note that she admits this risk when her blog post concludes

Getting something like ISO 27001, which is proscriptive [sic], hopefully offers some assurance that Amazon’s stuff constitutes effective, auditable controls.

ISO 27001 is not prescriptive in the way she is hoping. More to the point: “hopefully” is not an assurance for “Amazon’s stuff”.

Technical security controls such as firewalls or log management, for example, are not within the scope of a ISO 27001 certification audit; an organization is “hoped” to have the information security controls based only the fact that a management system is in place that satisfies ISO 27001 requirements.

That is why I say it is should not be said to be prescriptive by default or implied.

This is where it can be most misleading. Management determines the scope or the limits of an ISMS for certification, similar to primary criticism of a SAS 70. A business unit or location may be isolated to be certified, which ignores residual and real risk from remaining areas of the organization. An ISO 27001 certificate may exclude everything outside a scoped area; only the isolated area thus has an adequate approach to information security management. It looks something like this (blue boxes are examples of where scope can be limited and controls omitted):

Gartner has only added confusion by giving a misleading (“hopeful”) analysis and confusing ISO 27001 with ISO 27002. The standards are most effective when people do not oversell them (do not say that 27001 is prescriptive).

I am not certain why this analyst is criticizing the same thing she is practicing. I assume she just does not realize. Maybe she has seen the AWS Statement of Applicability (SoA) and certificate and believes them to be comprehensive and complete. That would be like saying, however, SAS 70 is a great standard because she found it was done comprehensively and completely at AWS.

It is important for AWS customers to realize that the ISO 27001 certificate is under NDA right now. Those who can review it in detail should have their audit or security staff look at exactly what area and controls are in scope. A good start would be to ask for their SoA (example). Only high-level information so far is available publicly.

The ISO 27001 certification includes AWS infrastructure, data centers and services, including Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3) and Amazon Virtual Private Cloud (Amazon VPC)

It is a step in the right direction, but achieving certification for an information security management system (in 27001 terms) is not necessarily prescriptive security for cloud compliance even if we “hope” that it is.

Those of you who are regular readers may know that I am an auditor. Experience in the field, as well as writing audit standards, definitely affects my perspective. The Gartner Vice President and analyst is neither an auditor nor a security professional:

Her prior roles have included product management, systems architecture, operations, deployment and product development.

That could also explain the difference in our views on this compliance announcement.

Edited to add: Also note how different the Amazon Web Services Blog sounds from the Gartner analysis:

SAS 70, a third party opinion on how well our controls are functioning, is often thought of as showing “depth” of security and controls because there’s a thorough investigation and testing of each defined control. ISO 27001, on the other hand, shows a lot of “breadth” because it covers a comprehensive range of well recognized information security objectives. Together, SAS 70 and ISO 27001 should give you a lot of confidence in the strength and maturity of our operating practices and procedures over information security.

They describe their SAS 70 in almost the same terms that Gartner used to describe the ISO 27001 as different. Then the two really diverge when Amazon goes so far as to say that, unlike the SAS 70, their ISO 27001 “on the other hand” is broad; it is not deep and not about each defined control. The Amazon announcement itself diffuses Gartner’s hopeful view.

UPDATE: A correction has been posted.