Sometimes we are asked about the failures of rating agencies and if a rating system could be a good approach for security evaluations. We have posted about it some time ago, but we think it is interesting to comment about the article titled "How Certification Systems Fail: Lessons from the Ware Report" (pdf), where Steven H. Murdoch, Mike Bond, and Ross Anderson give us a fantastic view of the reasons that make certification systems fail.
This article based on the report, "Security Controls for Computer Systems" (pdf) (commonly known as the Ware Report, after the chair of the task force - Willis H. Ware), summarizes the facts identified in that report from 1970 (!!!!) that explains the failures in certification systems.
Basically, there are three main reasons:
So, we agree with the authors of this article that "we should not expect certification to be a silver bullet" and that it should be used together with other security assessments systems, in this case, rating.
You can follow us on twitter.com/leet_security.