Threat or menace?

Sorry to be ranting again, but I am doing a survey that was sent via a certain large certifying organization.  One of the questions for "information security professionals" to answer was:


Thinking about your own organization, please rate the following potential security threats on the degree of concern you have for each.

• Trusted third parties                   
• Hacktivists                   
• Hackers                   
• Internal employees                   
• Contractors                   
• Mobile devices                   
• Cloud-based services                   
• Malware                   
• State sponsored acts                   
• Cyber terrorism                   
• Organized crime                   
• Application vulnerabilities    

Mobile devices are a threat?   Cloud-based services are a threat?    Really?  I think of those two things as technologies.  Neutral technologies.  Now these technologies may be full of vulnerabilities.  And there is a probability that these vulnerabilities may be exploited by threats... which will have impacts. 

Heck I'd even say malware isn't a threat.  Attackers using malware is.

And this from a survey targeting ccertified professionals who are supposedly tested in the basics of the risk equation.   But maybe different folks have a different way of thinking about threats?   Am I taking crazy pills?

So the question I put to all of you - What is a threat? 


 

Hofstadter's Corollary on Remediation

Any long-time readers of this blog know that when I get a pen-test or vulnerability scan report, I am usually displeased.  One of the things that get under my skin are the apparently wild-guess remediation estimates they put in for discovered problems.*



These estimates are usually done by a third-party (consultant, auditor, industry busy-body) who has not asked any detailed questions about my operational, development or business processes. They rarely even understand the business value of the service they're evaluating. They have no clue about the current workload or pipeline. Yet somehow they can pop off in a report that goes to my boss, my customers, and my regulators proclaiming that it shan't be any trouble 'tall to fix.

Now any of us in the security business for more than a year knows to take anything an outsider puts in a report with a grain of salt or two.  Unfortunately, our boss, our customers and our regulators tend to take these reports as gospel.

And then I hafta splain why it take so darned long to fix those things. I swear, sometimes I feel like Admiral Hopper explaining nanoseconds.

So here I present "Hofstadter's Corollary on Remediation".

Let's start with Hofstadter's law, which is:


Hofstadter's Corollary on Remediation states that remediation efforts are always longer than estimated by an outsider, even when they take into account Hofstadter's Corollary on Remediation.

But why is this? Let's unpack. Here's an example:

"Finally, let's estimate that each SQL injection vulnerability will require 40 developer hours x $100 per hour to fix, or about $4,000 total in labor costs." - Well Respected Industry Smart Person

Variable and fuzzy costs

Is that all it takes?  Well, no.  Let's put aside opportunity cost - which means you stall the development pipeline of requested customers features - a large but kinda fuzzy cost. 

And let's also put aside the flippant assumption that fixing a SQL injection vulnerability is just that simple and there are no underlying major software infrastructure components that need to be worked out. It happens but let's just leave that aside because it's also hard to quantify until you dig into the particulars.

And let's also not include the possibility that an organization has vulnerabilities because the whole development process is fubar - which actually is likely in places with lots of discovered vulnerabilities because of Boulding's Backward Basis - but that's also pretty variable for now so let's leave that aside too.

Known costs

So, let's focus on the tangible costs. For one, most organizations that care enough about security in their products usually also have very defined operational processes.

And some of us on highly regulated industries can't take advantage of new-fangled rapid deployment models. 

So any code change means it goes through release planning, functional requirements capture, use case development, specification development, specification review, test plan development, test case development, coding (what was estimated in quoted estimate above), code review, feature testing, regression testing, integration testing (sometimes against KLOCS of of code and third-party libraries), readiness review, release management, package development, documentation of release, site deployment to user acceptance testing, site deployment to production in an acceptable change window, site deployment to recovery sites.  Each one of these steps is at least a few hours of someone's time. And most painful of all, this whole wheel takes several months to turn... even for a seemingly minor code change.

In other words, remediation will slam right up into the immovable object of business requirements. 

A more simple example of this - You find a vulnerability on mail site - do you fix it immediately but in so doing shut off your CEO's email while he's at a customer site in tense negotiation?  Ummm. It's not an easy call and nor should it be.

But wait, there’s more

Suppose what you find something that appears minor, like a single XSS on a help screen.  Well fine and dandy, a quick fix. But wait, your developers just certified their product as 100% OWURST compliant with the Brand Spanking Impenetrable Cross-Site Scripting Defense Force-field.  So how did this XSS bug slip through?  You may have an endemic problem you weren't aware of.  And it's like that there are probably more of them to be found.   So now a minor remediation effort becomes a major bug hunt. Or it should if you're Doing The Right Thing(tm).  And some bugs, like XSS, are often part of the overall I/O engine of an application which may entail a major overhaul.



Now why would an outsider would stamp low remediation estimates on things they knew we have to hand craft a solution for?  I'll just slice it with Hanlon's razor.  But...

What about vendor-patched vulnerabilities?

So far I've just focused on the big and nasty vulnerabilities found in apps that you're responsible for fixing yourself. 

Suppose you have a minor vulnerability in your Pythia database server.  Well, you can just download the fix. Remediation effort: low.  Just apply the patch. 

Whoa boy, can't just do that.  You're talking about a production system on a high-volume financial transaction system.  We gots a procedure here. A procedure that's checked and audited with paperwork up the wazoo. Can't do squat without six manager's sign-off, two dbas to test queries to make sure nothing done broke (regardless of how banal the change actually is), at least a full month of regression testing, and then finally waiting for the appropriate change window which comes once a month at 3am on a Sunday.  When the moon is blue.

Remediation effort low, my ass.


* Remediation guestimation is the second most annoying thing in assessor reports.  This is the first.

The Vulnerability Assessor Risk Rating System revealed

Vulnerability Assessor Risk Rating System
Rating	Vulnerability
Low	Server has an IP address
Medium	Web server apparently is serving up web pages
High	Server running SSL instead of TLS
Super High	Directory content listing found in default Apache directory
Double-plus ungood	Self-signed certificate found on test web server
Critical	Correctly guessed login name is “admin”
Mega-bad	DoS vulnerability found in version number in banner grab
Fraught with peril	Developer email address found in HTML source
Horrendous	Password autocomplete not disabled on login form
Apocalyptic	Non-persistent cross-site scripting found
Singularity	SQL injection found on admin SQL database query tool

Where the security rubber hits the operational road

Where have I been? Absent from this blog, that's for sure. Mostly silent on Twitter as well. What up with that? As you mighta expected, life's been busy around here. But I thought I'd give a little more detail on what that meant.

Let's start with some of the problems that had come to a head this summer.

Gunnar had a great point when he spoke of the Top 5 security influencers. For past year or so, I'd already been working extensively with the Dev and QA team - now to the point where QA is taking bug directly from vulnerability reports from WhiteHatSec and
IOActive and developing QA testing procedures from them which is now spotting more holes, faster, and resolving them more deeply.

I already work alongside the DBAs and the Ops team, but I was really running up against a wall regarding just how much I could get done.  Basically, a lot of security projects were getting sidelined in favor of large infrastructure projects and I was starting to lose visibility into the whole process.  Worst, overlapping concerns between ops and infosec, like uptime and integrity, were losing ground to not well-planned horizontal expansion and vertical infrastructure structure issues.

The other big bugga-boo in my life was audit. Enough that I'd ranted a bunch about it at Source Seattle.

Well, I knew in the coming year that we'd be shifting some of our operational and access control models to a more expansive system to accommodate some new business directives. This was going to a problem as our existing set of control objectives needed to be redone from the ground up. And this change needed to come both from infosec as well as the operations team, but so far, they had not come to the table with any big ideas.  As mentioned before, they were too swamped fighting fires and keeping the lights on. This is an even bigger problem as auditing impacts operations as much as security, especially when you're doing SSAE-16 Type 2.  Without needed changes and strong leadership, Ops and Infosec were going to sink together come the auditors next year.

Note on SSAE-16/SAS-70: They're not all worthless. It's naive to assume they're a perfect measure of security and operational efficiency. But's also naive to assume they're worthless. As with anything, you need to do the work and actually read the report - does the scope match what you need to test? Was the audit conducted by observation and testing? Or just documentation review and attestation? A telling factor is how many individual control failures are noted. If the number is zero, then likely the report is scoped too tight to be useful or the auditors didn't do anything beyond interview people and leave.

Lastly, my Corporate Masters, were having problems getting a handle on operational projects.  As I mentioned in the lead, important work was being left on the table, while other lesser, more tactical projects were getting funded.  Operations needed help articulating and justifying critical infrastructure upgrades and aligning them to key business processes.  From my infosec seat, the solution was clear - risk to business objectives was not being defined with respect to operational problems.  Simplified example, conversations like "We need a better storage system because until we do, we can't take on any new customers" were not happening in the right places. Again, the team was too busy treading water to document, assess, analyze, strategize, and communicate. 

Enter me.

Our Chief of Operations wanted to inject a big does of risk management and clear process into the technical group.  Since infosec had been already been doing that in spades for the past four years, he asked me to step in and manage a chunk of the operations team. So about four months ago, I got promoted. In addition to security, I am now in charge of the Infrastructure team.

Believe it or not, I passed on some more lucrative opportunities to take on essentially a doubling of my workload. I'm silly that way.

Actually, it's more than doubled since it's been over a decade since I've done anything deep with infrastructure. For past few months, I've been playing catch up learning about large scale virtualization systems, storage area networks, operational resiliency and IT automation.

It's also more of a chance for me to bake security directly into the existing operational processes. Get my hands dirty and see what is going wrong.  For example, instead of auditing and dictating firewall policy, the team that directly manages network security reports to me.  I can directly see the effects of new security processes and technologies, both good and ill.  I'm on the front lines not just for security incidents (which also land in ops before they are identified and escalated to security anyway) but any other interruption events (which helps me design towards better integrity and availability).  So yes, it's good.

But not all good. Because, as I stated in the beginning, I've been treading water myself. In addition to getting to know the team and the technologies, I've also had to immerse myself in learning a bunch of new things.  Specifically:

• I'm getting to be a big fan of Limoncelli
  
• Been digging deep on project management
  
• Reading up on complexity and systems theory
  
• And good ole, Jerry Weinberg, who I keep coming back to for better understanding and wisdom.

I'll blog more in the future as learn interesting new things.

So, where have I been? A: Where the security rubber hits the operational road.

Compliance vs Security

Almost as exciting as a few other epic throwdowns, I am lecturing tonight for the University of Washington's infosec certificate program. A few quick highlights from my lecture notes, which is based on the Source talk I gave this summer.

Compliance-driven security forces you to make certain bets on the big enterprise roulette table - but I only have so many chips to play, so I prefer not to be constrained in my choices.

As a consultant I saw primarily two kinds of organizations:  Those practicing good risk management who wanted to get better and  Those forced to be more secure because of compliance or a breach.

Why is there such restrictive compliance regimens? Without repeatable, evidence-based, agreed-upon risk methodologies, you cannot rely on third-parties to make security decisions with your data that are aligned with your interests, instead of theirs.

Compliance is a multi-dimensional object... and lot more than three.  You've got width - the general rules f the standard plus a few specific new ones based on how the organization interprets it.  This is the easiest dimension.  Depth: As most compliance acceptance is based on auditor opinion, which is driven by the individuals experience.  Plus, if the standard is somewhat worthwhile, it includes the appropriateness of risk model (relevance) to your problem.   Then there's several dimensions of scope: Time (past events, present controls, future possible events) and then the general usual dimension of Physical, virtual, sofware, network... what’s constitutes a a barrier in those domains.    And of course, all of this is moving.

Security is also multi-dimensional but it has slightly different dimensions and moves differently than compliance.

 Best practices?  In other words, “This worked in our organization once upon a time, So it should work for you too.”

Where I live is the intersection of:

1. What the auditors demand we do,

2. What we need to do to keep from getting breached, and 

3. What we can afford to do.

And I'm mot going to get all of all three.

Stupid compliance failures:

- Why is the absence of a particular control is a risk? A high risk?

- How can I be 100% compliant with an open standard? With a product lifecycle of 12-18 months?

- Hey, that's a feature not a high-risk vulnerability - it all depends on your context

- Impact does not equal risk. You forgot probability. Dumbass.

I do it for the Lulz

I've always done for the lulz.   Security that is.  When I lecture to students up at UW, I try to warn them that don't do this job for money. 

Anyone going into infosec for the money, the prestige, or for the job security is seriously misguided.   I tell them the only reason I do it is for the lulz.  I warn them to prepare to face the humiliation of having a server hacked, the terror of knowing the bad guys will outspend and outlast you, the tedium of when nothing happens and the crunch when you need to justify every thing you've done to an auditor and the budget axe.  It's tough, it sucks, it's relentless, and I still love it.

What are my lulz?  The thrill of the chase. Heck, forget the chase, how about actually taking down some bad guys?

Besides the sexy stuff, I also get lulz making an organization safer… or even if it's just a friend of the family that needs some malware scraped off their machine.  Sure, it's tough work but it feels good to make the world a little safer, a little saner when you're done.  And knowing that you've deprived some creep one less victim.   Lulz.

I get my lulz designing new systems, making the strong, making them resilient, making them better than they were before.  And digging deep and figuring out where the holes are, where's the best place to fix things, and then working on presenting that to the people that care.  Even more fun than all the puzzles and video games in the world.


Technology, and especially information security has always been more than a job to me.  More than even a career.  It's a calling.  Don't tell my boss, but I'd do this even if they didn't pay me.  It's what I do.  I can't help it. 

And to those who say we're losing the war.  Whatever.  I've been hearing that for years.  The world hasn't ended.   I know that more systems than ever are online now and somehow we failures are still protecting a majority of them.   I know we'll be always outnumbered always outgunned.

That's what makes it a challenge.

Decompiling the week.

What an amazing week...

Fantastic time at Source Seattle.  If you didn't make it, you should really check out what you missed.

Great keynotes by Kris Herrin and Eric Cowperthwaite.  Nice getting the executive "big picture" on breaches and managing security.

Thoroughly enjoyed giving my talk and a lively audience as well.

Fascinating lunchtime discussion with Marcia Hofmann about privacy and the nature of social media.  Enough to make me re-up my membership in the EFF.  You should consider it too.

Not only a great demo by Ron Gula, but he spent time after the session doing a one-on-one with me giving an insider's tour of their software.  It was great to see a master at work.  How often do you get that kind of access to that caliber of talent?

If Source wasn't enough, I had to get me some Agora where Kirk B. pointed me at this fascinating paper on assuming a state of compromise.  Since that's what this blog is all about, you should check it out.

Now I need to sleep...