Can you buy security? How can you make things better?

So I'd twittered a bit lately about the question of organizations "buying" security. What I really wanted to know was if an organization was taking unnecessary InfoSec risks ("insecure" in lay-man's parlance), then they could simply bypass the whole cultural change thing and just write big checks to get improve their risk management ("become secure").

I've been asking this question myself a lot lately. I've spent nearly a decade as a security consultant, trying to fix organizational security programs (both for profit and non-profits). I've also spent about that much time working inside companies as a either a network guy with security responsibilities or a pure-play security guy. In my current job, I am *the* security guy. The buck stops with me. Now, with that in mind:

When I was a security consultant, I noticed there were basically two types of customers:

• The organizations that were already practicing pretty good risk management but wanted to improve
• The organizations who were being forced to be "more secure"

Naturally, the first type of organization made the best kind of client. It was fun to come up with innovative solutions to push them from a B to an A. We just ate up those technically complex security challenges. And yes, we were very successful. Usually these were complex, highly-regulated organizations like hospitals, law-firms, banks and public utilities.

Now the second type of organization, the ones forced to be more secure. These are the folks who've failed an audit, experienced a breach, or have an important customer dissatisfied with their security. Many of these organizations produced online products or services, and they "simply didn't have time to worry about security."

Not surprisingly, these were often the least successful clients of my consulting career. Advice was ignored, reports were shelved, warnings were rationalized away, blame was shoveled about. And things rarely improved much beyond some cosmetic fixes. They wanted to "buy" a fix to their security problem. Sadly, a lot of money was often spent on either new hardware or expensive audit reports.

As Jerry Weinberg said "Things are the way they are, because they got that way." And that's very true in these organizations. Their risk management processes are seriously messed up. And hiring a bunch of consultants and buying a bunch of tools doesn't seem to make a dent in that.

Over time, I quickly sussed out some warning signs:

• Not invented here syndrome. When given a suggestion for a new process or tool, you are often told "That won't work, we're special." Hint: the only things that should be unique should be your cash cows. Usually wasn't true for IT operations or infosec.


• Ill-fitting and ignored policies. "The security policy says we can't do that. But we need to do that. So we just ignore the policy." And thus begins the precedent to ignore everything in the policy. Usually a soup-to-nuts rewrite of policy with copious re-education to regain the user's trust is needed here. Difficult to do, even more difficult to do as a consultant.


• Lack of defined process and/or roles for critical things. These are the folks who are surviving because they have a lot smart people thinking on their feet. No time for docs, no time for process. We can figure out as it comes at us. Unless you don't know anything about security.


• A culture of reactive fixes. Managing risk just becomes another reactive fix. After we're done patching that hole, we can get back to The Real Work. Right?

Of course, there's the obvious: evidence of breaches, failed audits, high turnover, etc

Compare this to the things I see in the organizations who are doing a decent job of managing risk:

• Change management, especially for critical services and components (hat tip, Gene Kim)
• A clear understanding of their environment, the risks present in that environment and an awareness of how well they're dealing with them.(hat tip, Alex Hutton)
• Actually looking at their logs and understanding them.(hat tip, Anton Chuvakin)
• Someone who's primary focus is security. And this person is internally credible and willing to learn more.
  
• Reliable infrastructure. May not be the best, the fastest, the latest or the most flexible; but they know how it behaves and they know where everything is

Conversely, many of the aforementioned "obvious" signs can mislead here: places with strong security can experience breaches, failed audits and high-turnover.

But what about those broken organizations? Now we circle back to the original question? Can they buy thier way out of the hole? Not in my experience. External consultants fail... "Like trying to throwing sod down on cement and hoping it'll grow" as a colleague used to say. Heck, even hiring in good security folks and having them try to turn the ship is mighty tough. Of course, everyone says you gotta have management buy-in if you want to effect cultural change. And that's usually the cognitive disconnect you have in these kinds of organizations.

I could go into another long list of the types of executive paralysis that I've seen. It usually starts with "We really care about security, do what you gotta do." And it ends with endless trickle of dying projects that go nowhere.

Not to be all doom and gloom, but I've had some success starting to fix these kinds of organizations. But not from the outside - meaning the answer to my original question is "no", you can't "buy" security. But I've made change from the inside with slow and steady grinding away at the old culture. Gaining trust, garnering political will and slowly building up the layers of paint.

The two things that have been helpful in selling security culturally have been:improving system reliability and increasing organizational agility. Reliability is easy... that's the A of the CIA triad of security. The second is more interesting. At some point, security's just another characteristic of the overall health of operations. Organizations that are doing a poor job of managing risk are not very agile. They can't to market changes without increasing their risk exposure (usually exponentially). They can't scale very well and they can't hire/replace people. Look back at the warning signs and the positive attributes. A lot of them can tie directly to agility. And I find it ironic selling security as an agility improver when security is often cited as something that "slows us down". It's also ironic that the organizations that need agility the most (the software product and service producers), have it the least.

That's the end of my ramble. I'm going to think on this more. Feel free to comment.