The Big Bang Approach to Vulnerability Management
February 18th, 2008 Drazen Drazic Posted in Risk Management, To cool, Vulnerability Management, governance |
An ongoing vulnerability assessment/management program is probably the most proactive tool-based measure an organisation can take to identify weaknesses in infrastructure, OS and mainstream applications, (with web application testing abilities of such systems still developing). It amazes still that many organisations still don’t do this but that’s another story.
The toughest part of VA as any organisation that has implemented VA will tell you is not in selection of a solution (QualysGuard is the standout choice :-)) nor implementing it…nor even the initial scanning - it is dealing with the deluge of vulnerabilities reported and where to start to fix them?! That first report is an eye-opener for most organisations! And, this is where 90% of organisations get bogged down! It’s here that many organisations stall and some stall big time!
We’ve been working with organisations on vulnerability assessment/management programs for years so I thought I would talk about the most effective approach that we have seen to implementing a program that works. The following is not for everyone, but if you can make it happen, it will make your life easier and your organisation more secure in the quickest time.
Background: Okay, we all know that key stakeholder buy-in is critical for any strategy. VA/VM is no different. If you don’t have that support eg; CIO and senior management buy-in, the whole strategy/program is always going to struggle. We see it all the time. If your VA/VM program is not supported by effective process, it’s going to struggle. If there’s no ongoing reporting and accountability, it’s going to struggle. Get the above right and you’re going to do okay! Unfortunately, most organisations take 12-24 months to get to a level where the VA/VM program is working like it should.
The Big Bang Approach Background: A few years ago, we were providing a managed VA service to a well known banking sector organisation. The scope of this part of our work for the organisation was running monthly VA scans and reporting the results back to the organisation. My team’s primary contacts were senior/middle managers and each month, we supported the reports with a one hour meeting on findings, potential and real impacts to the organisation and advice on remediation of risks. For 4 months, nothing changed apart from the number of vulns rising in the environment. Little to nothing was being done after each month’s reports. They were wasting their time and money on us, so I called a meeting with the CIO. I knew the guy quite well and had a lot of time for him and his ability. He was known as a hard nut so to speak and took no prisoners, so after I explained the situation to him and suggested his organisation was wasting their money with us, he looked at me, lifted his hand and said, “leave it with me”.
The Big Bang Approach: The fallout from the meeting was immediate with my team on-site that afternoon asking me what had happened - it seemed any and every stakeholder, including some new ones were all rushing around on a mission.
Then by the following month, all category 3-5 vulns, (1 being the lowest criticality) were fixed and the organisation was now in a position to be able to easily deal with anything new (at most 2-3 new vulns per month) that affected their environment. They were now on top of it! They were running one of the most successful VA/VM programs I had seen.
So what happened? The CIO went back to his office after our meeting and in no uncertain terms laid the law down to his management team. Amazingly, all the previous excuses as to why things could not happen were no longer issues when accountability was thrown at managers and other key stakeholders to deliver! It worked!
The Lessons: It will not work for everyone - that is my PC statement but why shouldn’t it?! If you’re making the investment and security is key to your organisation, why shouldn’t the big bang approach work?
February 18th, 2008 at 7:47 pm
Okay okay….I know…thank you emailers that know the story. Yes, I removed the fact that when CIO returned to the office and “spoke” to his management team, every second word was something not publishable. I should have added that for effect. True and well spoken by him….sometimes the best way to get the seriousness of the message across!
February 19th, 2008 at 12:08 am
Most orgs don’t care because it’s not tied into release and change management process. Security is too scary and is to hard. It is just another overhead.
February 20th, 2008 at 12:02 pm
I like your Big Bang theory, DD.
Some of us might call it ‘going straight to the top’.
Please forgive me for playing devil’s advocate for a moment..
In the scenario you described, and for your ultimate success, I reckon you had a few very important things in your favour;
1. You personally had a pre-existing and good relationship with the CIO.
2. The CIO had enough nous & the balls to rattle the cages.
3. You were external to the company, with no worry of a CLM (career limiting move).
I wonder for instance if the same result could have been achieved by say, an internal pleb/middle manager rocking up to the CIO and presenting him with the same information that you did ?
Its these guys at the coalface who typically have the best idea of whats really going on.
Forget your ‘chain of command’, reporting line, bullshit as a means of telling your CIO the facts. We all know where this stuff ends up.
Lets be fair-dinkum about this.
You’d either have to be a *very* brave person, or else expecting a very large redundancy payout to rock up to the CIO & present them with info thats not exactly a good news story.
The best hope for the theory’s success (and I give it full marks) is that someone *external* to the enterprise deliver the news to the CIO.
BG
February 23rd, 2008 at 12:55 am
It only matters when an external party delivers a report in my experience. Stupid hey, when we do the same thing ourselves and highlight the security issues. It does not matter in my org how bad the results are.
February 23rd, 2008 at 4:44 pm
Good article on the importance of VA/VM programs. Still amazes me that so many organisations don’t have such a common sense program in place but happy to spend big bucks on silver bullet mostly useless solutions.
http://www.darkreading.com/document.asp?doc_id=146041
February 23rd, 2008 at 8:48 pm
Muhuhuhuhuhuhhahahahahahah!
“Reports that say that something hasn’t happened are always
interesting to me, because as we know, there are known
knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some
things we do not know. But there are also unknown unknowns -
the ones we don’t know we don’t know.” Rumsfeld.
It still amazes me that individuals in organisations refuse to acknowledge where 0days come from. It still amazes me that they don’t realise it’s about enumeration. Enumeration of the positive *and* the negative. Without flows from host/network, auto discovery tools, extrusion detection and default routes pointing to sensors or /dev/null on your cores, you might as well give up and go home now. NAC is a waste of time and effort. Counter-productive.
As for VA/VM, it’s a great baseline and enumeration of “known” weaknesses and exposures i.e. it’s more beneficial in my mind, to know your weaknesses than your strengths. How do you defend your weaknesses if you don’t even know you have them?
February 23rd, 2008 at 8:59 pm
D2,
“As for VA/VM, it’s a great baseline and enumeration of “known” weaknesses and exposures i.e. it’s more beneficial in my mind, to know your weaknesses than your strengths. How do you defend your weaknesses if you don’t even know you have them?”
And thus the reason for the success of the ORMF (Ostrich Risk Management Framework) copyright BorB…yeah.:-)
http://beastorbuddha.com/2008/02/16/ostrich-risk-management-the-most-successful-it-risk-management-program-in-it/
How many organisations have you worked for that just did not want to know how insecure they really were and how compromised they already potentially were?
February 23rd, 2008 at 10:42 pm
Tooooooo many!
The other issue is that there are not many in an organisation that can visualize or use their spatial comprehension to comprehend the complexity and relationships between nodes and services… e.g. their organisational footprint.
One of my pet projects is something you let loose on segments of a network and in real time provides 3d imagery and telemetry of the IT footprint. Tough but needs to happen. No one is currently modeling their environments properly, if at all. Without a model, virtual or otherwise the challenge of change management and touching production systems remains. Think of it as a big distributed gaming system comrising a mixture of agents, dedicated nodes and data sets from existing infrastructure nodes and endpoints
Kinda like a mixture of Sourcefire RNA, Opnet Modeler and Quake engine, distributed security and monitoring