Is there really value in Vulnerability Management?

For almost the last 10 years now, I’ve worked for companies that sell Vulnerability Assessment and Management products as part of their portfolio. I’m very familiar with most of the VM products out there and what their capabilities are.

SC Magazine recently did reviews on most of the major VM products. This got me thinking about VM and it’s place in the customer’s network. Is there really value in VM? Is one VM product better than another? Should security administrators be using VM? My answer is yes to all of the above, but with certain conditions.

I remember about 5 or 6 years ago I got a call from a very well known book and music retailer, someone that most of you have probably used before. They were interested in evaluating our VM product up against a few competitors. I worked with this customer for the next 4 months learning about their network and current state of security, why they wanted a VM product, what they wanted out of a VM product, etc. After an evaluation period, the customer chose our product for purchase. This was going to be a huge rollout costing them quite a bit of money, therefore I was asked to take a meeting with some of the higher-level people in this company to explain the benefits and costs of using our VM product. I had been in plenty of these meetings before so I was expecting more of the same. What was about to happen shocked me.

I had my laptop connected to the projector and started off by introducing myself and the company I worked for, giving a little background on the work we had done over the past several months and what the goals of the meeting were. That took about 5 minutes. The next slide I had prepared was a graphical representation of the companies’ external customer network and the critical level vulnerabilities we had found on it. That slide was on the screen for about 1.8 seconds before the CTO literally reached over and yanked the projector cord from my laptop. Everyone just looked at him in shock. He jumped up and said “If we’re made aware of all of these issues on our customer network, we’re now responsible for fixing them!” His lead security guy said “Ummm, yeah. That’s why I was given this project in the first place, to make our customer network more secure.” The CTO stuck to his guns and told everyone else that they wouldn’t be spending ANY money on a product that exposed their liabilities and vulnerabilities. At this point he not so kindly asked me to leave.

Vulnerability Management is a tricky space to sell into and an even trickier product for a customer to get use out of. I can’t tell you how many times I’ve seen a phonebook sized vulnerability report printed out and sitting on some security admin’s desk collecting dust. You can run all the vulnerability scans you want, print out as many reports as possible, create nice executive level reports with tons of pretty graphics and charts, but unless you actually DO something with these results, they’re not helping you much. VM vendors have made pretty good strides in making results more actionable and readable, but they’re still pretty overwhelming for most short staffed companies.

So what is the best way to get value out of a VM product?

Prioritizing vulnerabilities and systems to be scanned is a great start. Don’t scan every system for every vulnerability known to man.

Task different people or groups with different vulnerability groups. For example, break up your scans into categories like “databases”, “web servers”, “desktops”,  “network devices”, then assign those scans and results to the corresponding owners.

Tie your results into a ticketing system in order to track results. If you have an existing ticketing system, make sure the VM products you’re looking at can feed in and out of those systems. If you don’t have a ticketing system, make sure the products you’re looking at include one.

The bottom line is that VM has its place in most networks and can add huge value, you just need to learn its place in your network.

…josh

Share and Enjoy:
  • Print
  • Digg
  • del.icio.us
  • Facebook
  • Twitter

This website uses IntenseDebate comments, but they are not currently loaded because either your browser doesn't support JavaScript, or they didn't load fast enough.

2 Comments

Dan CornellFebruary 19th, 2010 at 12:33 pm

You make a great point – too many folks simply don't want to know they have a problem. I suppose citing that knowing about a problem makes you more liable for the problem makes a certain sort of myopic sense, but that is amazingly short-term thinking.

One thing – vulnerability management is not just for networks. With attackers focusing on applications (most specifically web applications, web services, etc), vulnerability management needs to expand to better accommodate attacks against software systems. This can be a challenging change of mindset because whereas "scanning" is how most organizations identify network and infrastructure issues, more analysis is required to sufficiently review software for vulnerabilities – dynamic testing, static analysis, manual code review, and so on. Many network-centric vulnerability management systems do not yet support these new tools and practices.

Also, while IT Operations or Security Operations folks use ticketing systems, software development teams use defect (bug) tracking systems. These defect tracking systems serve the same purpose as ticketing systems, but are targeted toward a different audience. So instead of using Remedy, software developers use systems like Bugzilla, JIRA, and ClearQuest.

Many infrastructure or network level issues can be fixed with a configuration change or by applying a patch. However, software-level vulnerabilities typically require changes in code to address the underlying issue. Technical vulnerabilities such as SQL injection and Cross-Site Scripting (XSS) have pretty simple code fixes (Assuming your organization has a standard recommendation for fixes to these issues that has been communicated to developers. You do, don't you?) However logical vulnerabilities that deal with the business logic of the system in question can require more invasive changes requiring discussions between groups about changes in requirements that might have architecture and design implications. In any case, after code gets changed the application will need to undergo regression testing before the new code is actually pushed out to production.

I have been working with some of my team to put together an open source tool that looks to address vulnerability management for application and software security issues (cleverly named "Vulnerability Manager" http://vulnerabilitymanager.denimgroup.com/ ) and the HoneyNet folks are working on similar issues http://www.honeynet.com/ I think the current set of practices in industry for managing application-level vulnerabilities needs some serious improvement, and the tools used for vulnerability management are going to need to change and adapt as well.

–Dan
@danielcornell

Matthew HacklingMarch 2nd, 2010 at 2:48 pm

Hi, great thought provoking post!

I think the issue you encountered with the CTO flipping out was due to a bottom up only approach to information security. It takes a good deal of time to get executive buy-in into an information security program. The best way I have seen it done by one of my colleagues (CISO) at a major manufacturer was to meet with ALL of the executives (I’m talking General Managers of all of the divisions, the Chief Risk Officer, legal counsel etc. not just people in the IT group) explain his role and objectives and discuss information security in their context. He then created a security charter (aligned closely with the domains in ISO27002) aligned with the business requirements and then shopped it around again, revising it. The culmination of this year’s worth of work was formal approval of the charter by all the executives at a meeting. My colleague is now empowered by the business to write policy and procedure to support these objectives and is able to start projects that align with the policy and procedure.

Leave a comment

Your comment