Clouding Log Analysis - Anything New worth a Look?

April 17th, 2008 Drazen Drazic Posted in Firewalls, Forensics, IDS, IPS, Research, Risk Management, Vulnerability Management |

“The cloud…..so pretty!”….thanks Wade for pointing me to this one at loglogic. This opened up a bit of discussion between a group of us on this “security in the cloud” business. Thought some of the comments would be worth putting together.

Some of my thoughts were previously covered here also. Anyway, the following are some of our ramblings. Feel free to add your comments.

DD: It is pretty [the cloud]…..lets hide any complexity within it, make it look more complex and then come in and explain how only someone like a Verizon could possibly do this all for the client. Same old stuff, just prettied up under a different description. Once again, lets go to the contract/SLA and see what is actually going to be reported. Lets regurgitate Dec’s presentation at Kiwicon again as a measure of what needs to be understood and addressed. Surely the cloud will now stand up to what Declan says? Or have I really missed it and this is something so good?

Donal: Think loosely coupled, internet service bus for the little guys, not the big guys.
100% on the deliverables SLA’s/OLA’s etc… what is it that one is selling… correlation, utility cpu, utility storage, compliance… or are you selling SECURITY?….. hmmmmm. I know what I intend to sell from the cloud

Dec: http://www.youtube.com/watch?v=dPHtKarae2Q

It’s log! It’s log! Its big, its heavy, its wood !

The fundamental concept of it should be simple: context.

A log entry at one point of the system means something completely different to the same one at a different point. As an example, I don’t care that some place somewhere in the world has Slammer and is hitting my boarder FW. I very very much care if I see Slammer deep in my network.

The problem happens because in a large organisation the router guys, the firewall guys, the load balancer guys, the inner FW guys, the Server guys, the Apps guys, the Clients, the users - no one talks to each other. No one understands each other and no one has the time to care. The security people don’t have access to all this stuff, and if they do, they have to ask and go through offical channels which is a PITA at the best of times. Example: IDS guys see an attack in the outer part of the gateway: Did it get past the inner FWs? Did it hit the target? What did the target do?

Context is everything and when you are dealing with 1 000 000 000 events a day. You don’t have time to call and ask everyone what happened - especially when time may not be synced accross all these hosts (which may well be in different data centres, owned by different companies, across different time zones etc etc etc - AND NOT FSCKING TIME SYNC’D).

A system should take these logs, from every point in the network and give it a location rating for context. (as well as lots of other ratings along the way, i’ve simplified this a lot). Then it should be tracked through the network - there may be 10-15+ events that are from the same attack. So then you get one single alert built from all the diffent events throughout the system, contextually analysed so that if you see the attack through the router, the gateway etc two layers of IDS and the client firewall but then the host isn’t vulnerable.
- it gets downgraded and the analyst does not see it. It isn’t important.

You need to be able to follow the attacks, it is such a simple and brilliant idea!

Doing this, you reduce the number of investigations down to a more manageable level and you know, with some certainty what is and is not important.

So to the big question - why do we know this but only now are such “services” being promoted and sold?
Operational constraints then for organisations? ie; having to comply with certain standard builds and technologies that were not suitable, fast enough, stable or scalable. You get the idea. :-/

Wade: I’m with Draz, the ability to hide behind the cloud is really scary. I’ve seen organisations heading down this path this with the IDS’s they’ve got. Dumping the logs to vendor X.

I also wonder what the cloud provider’s doing with the logs. Are they like Arbor, publishing their research, or looking for more sales? “Oh you have so many alerts, you’ll need 3 full time staff to read them”. Try prove otherwise, it’s a cloud.

@Dec, sounds like you wrote one of the very first intelligent log parsers! Does CS MARS allow for that sort of stuff yet? Or is it still/just severity based?

Donal: NTP, DNS, Syslog/CEE, routing updates(multicast/unicast). Get these wrong and the rest is worthless.

Correlation/context. Context very important, agreed.. slowly taps/IDS/asset management are passively/actively identifying context of endpoints. Time and sliding window of services in an IT footprint
is always a problem thus context has to be garnered automagically.

Attributing value to endpoints, shared infrastructure and data/information is still the key. Metrics, what exactly are we measuring, value, risk, threat, compliance, progress, deltas, efficiency….. etc. etc.

Compliance or security. Debate already waged.

We eval’d Intellitactics before. MARS is quite good. ARcSight is interesting. Basically not having the ability to get all devices in footprint to provide for offbox logging is an organisational/build issue. Disparate groups need to start working together again.

There has to be an independent ability to verify assets in an asset database e.g. managed nodes and services and compliance to standards/templates. Cisco Network Compliance Manager deployment?

Most of the time IT is not even business aligned, and compounding issues is the business doesn’t know how to realise efficiencies from IT. How does one mitigate against a sentient attacker or future tech?

One must focus on auditability, surveillance and mean-time-to-repair…. or my new phrase… mean-time-to-return-integrity MTTRI

Also, most peeps forget to start with telemetry. Prevention always fails. Security needs to focus on temporality.

“[P]roving a negative is impossible except in the case where all possible alternatives are known and each is examined… “Prove a negative” in this context means to be able to show a skeptical party that such and such a thing did not happen… As a matter of science, to prove that something did not happen toy must have every place where it couldhappen under surveillance… ”

I REALLY like this post:
http://taosecurity.blogspot.com/2008/04/review-of-economics-and-strategies-of.html

@ Declan.. maybe it’s time to blow the dust off your product/service…I think it’s time has come! What A record should I point my logs to and I’ll plug my service in to yours?

Dec: The long and the short of it is that if done properly it’s good, if not, it’s no good. Whether or not you trust them [the vendors] is up the the client, really.