Interview with Marcus Ranum - Blunt Industry Assessment

Posted on November 19th, 2007 by Drazen Drazic

Marcus J. Ranum is a world-renowned expert on security system design and implementation. He is recognized as an early innovator in firewall technology, and the implementor of the first commercial firewall product. Since the late 1980’s, he has designed a number of groundbreaking security products including the DEC SEAL, the TIS firewall toolkit, the Gauntlet firewall, and NFR’s Network Flight Recorder intrusion detection system. He has been involved in every level of operations of a security product business, from developer, to founder and CEO of NFR. Marcus has served as a consultant to many FORTUNE 500 firms and national governments, as well as serving as a guest lecturer and instructor at numerous high-tech conferences. In 2001, he was awarded the TISC “Clue” award for service to the security community, and the ISSA Lifetime Achievement Award. Marcus is Chief Of Security for Tenable Security, Inc., where he is responsible for research in open source logging tools, and product training. He serves as a technology advisor to a number of start-ups, established concerns, and venture capital groups.
——————————————————————–
Marcus gave me some time today to ask him a few questions about his takes on the industry. You won’t die wondering as to what MjR’s true thoughts are:

BorB: Aside from the busy schedule of speaking engagements that you have booked up well into 2008, what else is keeping you busy at the moment?

MjR: I’ve usually got waaaaaaay too much stuff going on at any given time!

Right now, for example, I’m developing a class on using Nessus for compliance monitoring. At the same time, I’m working on reanimating some long-dormant code I wrote for ultra-fast log parsing, and I’m strategizing with a start-up about some marketing concepts.

Meanwhile, I’ve got a trailer-load of firewood I’m cutting, splitting, and stacking, and I’m rewiring my Russian-built 1974 tractor.

On the “fun” front, I’m running an art contest on deviantart and I’m trying to figure out the correct lighting for high-speed video capture of stuff getting hit by bullets. And there’s also some sleeping and eating I need to do someplace in the middle of all that.

BorB: Your “Ultimate Firewall” presentation on your website (http://www.ranum.com/) is the best one I have seen. Every week, one big vendor or another is releasing the ultimate security solution. Is there anything new you see in any of these products in recent times that you recommend people seriously investigate?

MjR: Honestly? No.

In security our problem is not a lack of new stuff - our problem is a lack of sensible application of existing technology. In other words, we should be spending less time doing stupid things rather than spending extra time doing new stuff. All these new products purport to solve problems that we wouldn’t have at all, if people would just stop doing dumb, dangerous things.

Take “data leakage” for example. The way to deal with data leakage is to know where your data is, track who’s got access to it, and minimize how many people get unrestricted access to full datasets. You don’t need technology to do that - doing that is a matter of NOT doing stupid things like letting your sales guys download the entire customer database to their laptop - because any idiot ought to know that it’s going to go with them to their next job.

This isn’t a technology problem - it’s a failure of technology management and leadership and it’s not going to get fixed by buying stuff.

A lot of what’s going on is very typical (if I may stereotype) of American society. Rather than do the hard but sensible things, (eat well, eat reasonable portions, exercise), we want a do-it-all pill (magic-o-zap fat burner) instead. It’s weird, because to me it’s just a denial of obvious reality: computer security is hard work that requires organized thinking and self-discipline.

The tools we need to build secure networks are all well-known and well-understood. What we need is good management and a serious attitude, not more new $25,000 doo-dads in 1u rackmount appliance configurations.

BorB: I’ll put you on the spot as I did with Matt Jonkman. Name some of the security vendors and/or products you trust.

MjR: I don’t establish trust on a vendor basis - that’s a really bad idea. How many vendors actually write their own stuff?

You’ve got to look at everything on a case-by-case basis, to see whether it fits the job - and you need to ask questions. There’s way too much trusting going on - people buy something called a “stateful firewall” and nobody asks, “what does ’stateful firewall’ mean?” If you do, you discover the answer is: not much!

Some of the huge vendors have entire security product lines that they bought from various start-ups and glued, (or didn’t glue) together. You’d be crazy to say, “well, I trust anything that big vendor X offers me.”

I first encountered this when I started selling firewalls when I worked at Digital. People would buy it because it was from a huge company - but they never seemed to understand that it was just 1 guy (and later 5 guys) within a company of thousands of employees.

So, stuff I trust? None.!

These days, if I am fielding something in an exposed and critical position, I’ve got to research it and make myself comfortable that it actually works as advertised and has a decent overall architecture.

BorB: What’s your take on the work being done in the Jericho Forum? They’ve been around for a few years now. Have they made any difference at all?

MjR: Standards efforts have no effect until after a market has reached the end of its financial play - by which time it is no longer interesting technologically; it’s commoditized.

Jericho Forum, specifically, is promoting the idea of “deperimeterization” which is just a bunch of hype and some fairly naive thinking about security.

I know that sounds like a pretty heavy accusation, but the claim is that end-points can self-secure - which is ridiculous in a networked environment. The Internet applications stack depends heavily on ARP and DNS and those protocols depend on a tamper-free network. It’s just silly to think your end-point can secure itself if the network fabric is untrustworthy! If the network is untrustworthy, it’s “game over, man!” as Private Hudson would say.

You can fix network level trust using encryption but only if you have a working trust infrastructure (like a PKI) sitting atop that. Anyhow, I’m sure the Jericho guys are great guys and they mean well but I just think they’re dealing in wishful thinking and security doesn’t work that way.

BorB: Many anti-badware vendors in recent times are coming out with optimistic statements that they’ve got the bad guys on the back-foot (eg; Sophos, Kaspersky), surely they’re kidding themselves?

MjR: They’re not kidding themselves and they’re not kidding the malware writers - so who are they trying to kid? Maybe us.

As I’ve written elsewhere, I think it’s amazing that the A/V industry happened at all - and it’s even more amazing that it’s survived so long. It’s a great example of how sloppily we think about system design and just how tolerant we are of mediocrity when it comes to operating systems.

BorB: You travel the world regularly. Do you see significant differences in how people in other countries approach IT security? Who in your opinion does it better than others, who doesn’t, who doesn’t care and does that make any difference?

MjR: It seems to me that a lot of countries outside the US look to the US.

Honestly, that doesn’t strike me as a particularly good idea! For example, the security of our government’s systems is largely very poor. I’ve found that when I travel, foreigners seem to think that the US DOD’s security is like “Mission: impossible”, but it’s more like “Monty Python”.

I’m deeply concerned that the degree to which government systems have been outsourced has allowed a tremendous dumbing-down on the part of those responsible for making decisions.

That will have a lasting impact, I feel, and will be a matter of national security.

On the corporate side, I think that most companies (US and non-US) have seen the kind of damage that you can suffer from an information leak. That has finally begun to sink in, which is good. Choicepoint and TJX and so forth have served as a good example of how bad it can get.

Meanwhile, on the home front - home users - it’s disaster everywhere. It’s going to take another decade or so for the various governments and corporations of the world to realize that millions of unsecure home users represent a public “Internet health” problem.

BorB: What’s your take on the state of the IT security industry today? Are things getting better or are we still very product focussed and neglecting the basics?

MjR: Most IT security professionals don’t even understand the basics well enough to neglect them.

BorB: While more and more breaches are being reported in the press, how bad do you really think it is? My take is that less than 1% of breaches actually come to light and most organisations would not even know is something has happened.

MjR: I think that the situation is definitely very bad, and I agree with you that a great many organizations would have no idea if they were, in fact, compromised. When you look at compromises like TJX or the Veteran’s Administration, I think it’s also pretty clear that organizations are unable to accurately assess the scope of a break-in - or are shockingly
willing to lie about it.

Now I’m going to sound like a paranoid for a minute; hand me my tin-foil hat, OK? The stuff that I really worry about is the potential for severe leakages of information regarding advanced technology, military deployments, and intelligence. I look at the US Government’s
security and how bad it is, and imagine it must be like a gigantic shopping mall for intelligence operatives from other powers. That’s a deeper threat than just “information leakage” - I’m worried about things like potential Sovereignty-ending events like losing an
economic or military war as a result of having exposed too much information.

Problems like that can take generations to mature and our society does not do a good job of thinking in terms of multi-generational threats or how to defend against them. But consider, for example, the FBI’s email system being hacked.
( http://www.gcn.com/online/vol1_no1/35019-1.html )

Do you think there might be anything interesting in there? What if 20 years from now we discover that we’re no longer a superpower because someone else had been reading the National Security Council’s Email for a decade? Sound implausible? So does the idea that the FBI’s Email would be hosted on AT&T’s public mail servers.

So, I think the situation is bad - but I’m vastly more concerned about the long-term threats. Those represent problems that we may never recognize specifically as coming from a particular exposure or source - but it won’t matter.

BorB: Any solutions to improving things or are we just heading down a path where things will continue to get worse?

Things certainly aren’t getting better.

You’ve no doubt seen charts like this?

And you’ve probably seen charts like this:

Well, that’s a classic negative synergy. The more we spend, the worse it gets!

I think there are interesting and serious questions that should be asked regarding how effectively our security dollars are being spent. Any time you’re spending lots of money on something and the problem is still getting worse, it means that you’re in trouble! One possible thing it means is that if you weren’t spending the money, the problem would have gone completely berserk - which is, I think, the case with security.

BorB: Most IT Security Managers I speak with all have the same problems – generally in a thankless role, battling to spread the word and continually fighting to make even a small difference in their organisation. Any tips you would give them?

MjR: Get used to losing.

I’ve never seen any sign that executive management is going to suddenly “get it” and every sign that the problem is going to get worse. Wait for Web 2.0 to really take off - then you’ll see what “bad” really feels like. We’re going to have executives asking for these complex mashed-up websites that do all kinds of literally incomprehensible stuff for the users.

“Incomprehensible” because nobody will actually know where the data actually goes, or what controls are on it, or where code is being executed - and it’ll all run atop a platform of
shovelware written by hobbyists.

BorB: A little while ago, you voiced some strong opinions on vulnerability disclosure
(http://www2.csoonline.com/exclusives/column.html?CID=28072) and got some divided opinions from the industry. Has you opinion changed at all since this article was published? You ended the story with “If you’re a customer or end user, you can see how well disclosure worked to improve your security over the last decade. Let me be frank: It’s up to you.” What does that mean?

MjR: There was a lot of vocal disagreement with my comments and nearly all of it came out of the camp of pro-disclosure practitioners that I was calling out.

Of course they were unhappy - I was publicly calling their actions naughty and selfish. I don’t get it; I’m a meanie and a bad guy for pointing out that these cheesballs have built their reputations finding flaws in people’s software so that they can make a living as consultants fixing flaws in people’s software? And, while they’re doing that, they’re exposing the users of the software to increased risk?

“You can see how well disclosure worked to improve your security over the last decade” - Well, the vulnerability pimps and full disclosure fans keep claiming that by finding and disclosing bugs they are helping make software better. If that were true, we’d be seeing a
massive down-turn in the number of systems being hacked into, right?

We’d be getting software that didn’t need to be constantly patched, right? Of course we’re not. Because vulnerability disclosure doesn’t help ANYTHING except it’s a good way for the vulnerability pimp to get his 15 minutes of fame. Or, nowadays, to sell his new Day-zero attack for $10,000.

Yeah - sure - these guys are doing it for the good of the community. Right.

So, I think customers need to educate themselves and look at the situation and decide who their friends are. The whole industry and a great deal of the dialog in computer security is being controlled and gamed by people that do not have the customer’s interests at heart. I feel bad for the customers because they’re getting played for suckers. Now, you’ve got ridiculous things like standards that mandate penetration tests. What a waste of time!

Penetration testing is not a substitute for having a good design to begin with - just like “bug hunt and patch” is not the way to get a stable and secure application. This stuff has to be coded in and built into the foundation, not added on reactively. But customers have been completely conditioned to live in this stimulus/response loop around bug disclosure followed by big patch, or penetration test followed by fire drill. The whole process is completely dysfunctional unless you’re a vulnerability pimp or a pen tester or a security consultant! Like me.

On that topic - these days I spend 95% of my time trying to talk clients out of bad ideas. I have concluded that not doing something stupid is vastly more cost-effective than any other security strategy, and it saves money. Needless to say, this approach usually does not work.

BorB: Do you think the bad guys positively contribute to the levels of uptime on perimeter systems for many organisations? Business is business and downtime is not good for anybody:-) (DD Note: This was a tongue in cheek question from me based upon our experience of so many organisations being clueless as to the state of their security systems sitting out there on the Internet)

MjR: One thing a lot of people refuse to see is that computer security has a moral dimension. I know it’s old-fashioned, but I think it’s important that people take responsibility for the consequences of their actions.

You also need to take some responsibility for the impact of your behaviors on the community at large. Basically, what we’ve got with security is the equivalent of soccer hooligans rampaging through quiet neighborhoods and saying, “we did this for your own good! look what a great job we’ve done of testing your security! now you can go fix everything better than it was before we came through!”

The thing everyone wants to ignore is that the “bad guys” share a moral responsibility for all the downtime they cause, all the extra system administration costs, all the lost sleep, all the extra software releases, etc, etc.

Certain people understandably are very upset by that argument, but its true. Without burglars, we would need no locks. Therefore the burglar victimizes us first by making us waste money on locks that would otherwise be unnecessary and again if he breaks in anyway.

What has happened as a consequence of hacking and cybercrime is that The Internet has reacted with massively expensive responses.

Our E-mail infrastructure today carries something like a 150 billion messages a day, 99.5% of which are spam. So the Internet community has spent millions and millions on excess capacity to carry all this junk.

Then, the Internet community spends millions and millions on anti-spam products. So the amount of time that is wasted (which also comes with a cost) is the tip of the iceberg when it comes to the total cost of a single form of online crime. How much bandwidth is being wasted by worms and bots and viruses? How many millions of dollars are being spent in wasted system administration? Antivirus is a multi-billion-dollar industry, just by itself.

Consider the total cost of all the system administration time wasted testing and installing security patches. If you were to somehow total up the costs we’re incurring as we try to keep the bad guys out of our systems, everyone would be out with the pitchforks and torches looking for these guys.

The end result of the current security industry has been the complete collapse of any kind of effective software management and system administration. I know system admins who are
now expected to install patches within 24 hours of their being released. Is that how to run a stable, reliable production system? Hell no! The way to build stable and reliable systems
is to install them, make them work, and leave them alone.

I know the head of security at a large software company that produces a very critical application that is widely used. Whenever someone finds a problem, they fix it and it triggers a new Q/A cycle and release cycle for multiple versions of code and dozens of different language releases. This costs them millions of dollars a year, which they pass on to their customers in the form of a 20% maintenance contract.

The way to produce mission critical software is not by having your release cycle get jacked around every time a vulnerability pimp feels its time for a little attention.

Hacking and the vulnerability pimping game have put us into this weird mode where our software development cycles and system administration practices are being driven by absolutely the wrong forces. All this does is drives up costs for everyone - except the bad guys.

BorB. Every man and his dog now has a security blog? Aside from Beast or Buddha :-), whose do you read?

MjR: I don’t read any security blogs regularly. That would be too depressing.

BorB: What other sites does MjR have bookmarked in his browser?

MjR:
ORAC’s blog: http://scienceblogs.com/insolence/
Pharyngula: http://scienceblogs.com/pharyngula/
JREF blog: http://www.randi.org/
Bad Science: http://www.badscience.net

Much of what I read these days is the science/skeptic-related blogs. I find the conspiracy theorist nutbars and alternative medicine cranks to be fascinating.

In a lot of ways, the current state of denial that the computing world is in regarding security is somewhat similar to some of the alternative medicine-think and vaccine denialists. It’s interesting to see places where people passionately and eagerly latch on to absolute nonsense.

Here’s a final point to ponder: the function of a Scientologist’s E-meter is better understood than what a vendor means when they say they are selling you a “stateful firewall”.

When I run across someone pushing pen-tests to improve network security, it’s not hard to imagine them as a homeopath or astrologer, fleecing their customer or selling them a placebo consisting of water and sugar.

BorB: Marcus, thanks for your time.