It is important to remember the key objectives of CyberStorm because success is not only determined by outcomes, but also from requirements.

The key CyberStorm outcomes are*:

* Examine organizations’ capability to prepare for, protect from, and respond to cyber attacks’ potential effects;
* Exercise strategic decision making and interagency coordination of incident response(s) in accordance with national level policy and procedures;
* Validate information sharing relationships and communications paths for collecting and disseminating cyber incident situational awareness, response and recovery information; and
* Examine means and processes through which to share sensitive information across boundaries and sectors without compromising proprietary or national security interests.

*http://www.dhs.gov/xabout/structure/editorial_0839.shtm

No where here does it say “make the internet secure” or “liaise with the private sector to fix most common system vulnerabilities” or anything of the sort.

When I was a Fireman, we used to perform routine exercises where by we would go through our response plan for various scenarios to to make sure when the real thing hit, we would all know what to do, and to see if there were any improvements that could be made to the plan.

At no point did we attempt to determine how easily a house could catch fire, or what we could do to stop houses from catching fire in the first place - while valid, these were a completely different exercise.

Things like equipment, communication and strategic thought were key and helped enormously when were called to response to a fire.

While they didn’t in themselves stop any fires from starting, these were completely successful and saved much property and many lives.

Its the same for CyberStorm - having worn incident response shoes before I know what that the hardest part is equipment, communication and strategy.

We don’t need to have a capture the flag style event across countries because we already know what will happen - in the same way that we don’t need to test nuclear bombs anymore because we know what happens - things blow up.

Should the results of cyberstorm be revealed so we can all determine for our selves whether or not it was a success and good use of our hard earned money ? Maybe. Or better yet how about releasing all the useful outcomes for us so we, as a community can be safer - well in a way they have.

Run your own CyberStorm and see how you go. List various scenarios and see how you would react - how you /could/ react.

Then update your risk register. Without attacking a single system you may well have solved the biggest problem in security that we all harp on about all the time.

Incorrect risk assessments.

Maybe CyberStorm was worth it after all.

“how does one engineer a process around a set of unknowns.”

Good point D2. How indeed - and that is precisely the point of CyberStorm.

If we head down the Australian Wheat Board investigation path into dealings with Saddam and decide due to politics and protection of the guilty that we create a scope – ie; called in cases like this, a “terms of references” - read: anything bad that could come out of this or anything that could implicate poor or negligent practice by “senior people” will not be considered in scope, then, with such “war games”, it doesn’t matter what the outcomes are or are not! We’ve done something that in the eyes of the majority, looks like we are on top of this and are acting in a manner that should be perceived as trying to “protect” and “serve” the interests of our country – individuals and business!

Waffle waffle waffle waffle….

Cyber-Storm I - Lessons learnt: ????? Nothing noticed nor measurable.
Cyber-Storm II – Lessons learnt: ????? Nothing seemingly released to business and community in general. Nothing noticed since then nor measurable.

Now maybe I am not close enough to this and that is a definite but surely the purpose of these exercises is to spread information and for that information to be turned into something that improves a position in a “war” scenario…ie; supposedly a war happening now? As an aside, where this differs to other “war games” is that we play these games while the real “war” is happening. Surely we don’t have the resources to play games while the actual events happen around us? That’s something to think about – play the game, analyse the results, plan what we should do, do it (yeah right) and then play another “war game” in 2 years time to see how we are going?!! WTF?

Genghis and the hordes have destroyed everything in their path. Their arrows are killing people through our (fire)walls but lets play games to assume/predict what they could do based upon the knowledge of a few “select” companies with real “bad-assed” hacker dudes (sarcasm extreme) to predict how bad it could get…..you get the gist……..

Bottom line is pretty much as D2 has put forward….we’re past the BS theoretical of it all and should be well past thinking about what we know and thinking about what we don’t know and how to build a capability (study) on approaches to dealing with that! A big macro-level exercise based upon what we already know does nothing….which then begs the question, are the people running this only getting to that level of knowledge that we in the industry have had for a long time?

Didn’t most of us go nuts at the TV screen every time Bush and Howard talked about how smart their approach to Iraq was and the work they were doing was for the good of the world? Flame on….Since when do we trust government to be doing the smart thing? With Cyber Storm exercises, I am yet to be convinced we have any positive and measurable outcomes!

@BG

>If Cyber Storm (in a teacup) were such a resounding success, why are we still all in the dark as to what went on ? Its the very reason we’re still arguing about it. Point is, no-one, outside the participants, really knows.

This is the key point - we don’t know.

Would it be fair to summarise the criticism to:

* CyberStorm is great but it does little to solve the deeper issues of Information Security.

* The lack of public disclosure inhibits the spreading of information back into the community which could well assist in the first point, and will help to shut up a whole heap of narky consultants who are frustrated because they genuinely want to make Australia a more secure place and see it as a lost opportunity.

On question I have is that other countries involved with the CyberStorm exercise engaged more specialist private firms give a more realistic focus to the attacks - but Australia did not.

So, I happily put up my hand to assist with CyberStorm III*

*Revenge of the Sith ?