Sources of Attack
Even the sources of attack are problematic. The Australian survey notes that the rate of insider attacks has remained constant, but the Deloitte survey suggests that the rate is rising within its population of financial institutions. There is some convergence of findings, however. Viruses, Trojan horses, worms, and malicious code pose consistent and serious threats, and most business sectors fear insider attacks and abuse of access. Most studies indicate that phishing is a new and growing threat.
Many of the surveys capture information about effect as well as cause. A 2004 survey by ICSA Labs reports a 12 percent increase in "virus disasters" over 2003, but the time to recover lost or damaged data increased 25 percent. The cost of recovery exceeded $130,000 on average. By contrast, the Australian, Ernst and Young, and CSI/FBI surveys found a decrease in total damage from attacks. The nature of the losses varies, too; CSI/FBI reports that 25 percent of respondents experienced financial loss, and 56 percent experienced operational losses.
These differences may derive from the difficulty of detecting and measuring the direct and indirect effects of security breaches. There is no accepted definition of loss, and there are no standard methods for measuring it. Indeed, the ICSA 2004 study notes that "respondents in our survey historically underestimate costs by a factor of 7 to 10."
There is some consensus on the nature of the problems. Many surveys indicate that formal security policies and incident response plans are important. Lack of education and training appears to be a major obstacle to improvement. In general, a poor "security culture" (in terms of awareness and understanding of security issues and policies) is reported to be a problem. However, little quantitative evidence supports these views. Thus, in many ways, the surveys tell us more about what we do not know than about what we do know. Many organizations do not know how much they have invested in security protection, prevention, and mitigation. They do not have a clear strategy for making security investment decisions or evaluating the effectiveness of those decisions. The inputs required for good decision makingsuch as rates and severity of attacks, cost of damage and recovery, and cost of security measures of all typesare not known with any accuracy.
We can conclude only that these surveys are useful for anecdotal evidence. A security officer can point to a survey and observe that 62 percent of U.K. respondents reported a security incident at an average loss of £12,000. But management will rightly ask whether those figures are valid for other countries, what constitutes an incident, and whether its organization is vulnerable to those kinds of harm.
The convenience surveys are a good start, but for serious, useful analysis, we need statistically valid surveys administered to the same population over a period of time. In that way we can derive meaningful measures and trends. The surveys need to use common terminology and common ways to measure effect so that we can draw conclusions about past and likely losses. And ideally, comparable surveys will be administered in different countries to enable us to document geographical differences. Without these reliable data, economic modeling of cybersecurity is difficult.