Security Metrics - Measure What Matters - Part 3

Security Metrics - What Information Do We Need?

One of the right security metrics questions is, "What's the greatest threat to security at your organization?"  (My friend Joe, who works at a large bank, answers: "Management.")

Internal user behavior (accidental or intentional) results in nearly 80% (and higher by some estimates) of information security incidents.

Q: If this is true for your organization, then what should your security awareness metrics measure?

A: Internal user activities.

As much as possible, metrics should be related to a business function.

 

Q: When do managers care if employees know more about information security?

A: When it reduces the cost of operations.

The objective should be to measure user behaviors that are a part of normal business operations.

To do this, you'll need a baseline and a target.

Baseline

A baseline documents the level of awareness among employees:

Within your industry - a baseline may be available from surveys of your industry, such as the annual global security surveys performed by Deloitte Touche Tohmatsu and Ernst & Young for the financial industry. Surveys where organizations are asked about their: security awareness practices, security policies, education, training, compliance, threats, and acceptable behavior. 

In your organization - the baseline might document:

• How staff perceive security at your organization
• Specific behaviors that affect information security at your organization

Information for the baseline can be gathered from surveys, by observation, from software, from audits, from specific security tests, and from help desk reports.

How Staff Perceive Security

Questions that address staff perceptions of information security in your organization might include:

• Does security help people work by ensuring that assets are available when needed?
• Are the organization's security policies credible?
• Are good security behaviors rewarded?
• Are there real consequences for risky behaviors?

Specific Security Behaviors

Security behaviors that affect your organization include:

• Whether or not staff recognize specific security concerns. For example, given a number of scenarios, which will staff recognize as ones that should raise a red flag?

1. Leaving a workstation logged in to an application that contains personal information while going to another office to retrieve a fax.
2. Going on vacation without leaving the password to an application with the supervisor so that the temporary worker can get started right away while the worker is out of the office.
3. Responding to a call from someone at the help desk who says that as part of a network upgrade the employee's account log on information will be overwritten, and the help desk technician need's the employee's name and password so that she can re-enable the account after the upgrade.

• What staff will do in response to security scenarios

Once you've established a baseline of user perceptions and behaviors, track changes over time as your program progresses. This lets you know what's working and what needs to be changed.

Target (G*Q*M)

Victor Basili at University of Maryland developed an approach to metrics called GQM, or Goal, Question, Metric. He teaches these steps:

1. Start with a GOAL
2. Then find a QUESTION that will tell whether or not you're meeting the goal
3. Then, and only then, look for a METRIC that will support the goal.

The metric is developed last, not first.

For example, if your goal is: Decrease inappropriate Web site visits.

The question is: Are people continuing to visit Web sites that they shouldn't?

The Metric is: The number of attempts to access unauthorized Web site content (such as illegal or pornographic material).

This data can be extracted from Web filtering products. This is an automated metric - it's easy to collect, and it shows what the computer users are doing.

 

This is the third part of a four-part series of articles:
Security Awareness Metrics: Measure What Matters