© Open Security Foundation 2005 Vulnerability Databases: Everything is Vulnerable Brian Martin Jake Kouns.

© Open Security Foundation 2005

Vulnerability Databases:Vulnerability Databases: Everything is VulnerableEverything is Vulnerable

Brian MartinJake Kouns



•Overview•Inherent Problems•Important Issues •Major Players•Research and Rankings•Future



Overview


Vulnerability Database Vulnerability Database OverviewOverview

• What is a Vulnerability Database (VDB)?

• Database of information on security vulnerabilities. Simple!

• What about “dictionaries” (CVE) or “searchable indexes” VDB!

• Key is realizing VDBs will have their focus– Comprehensive Vulnerability Database – Focused Vulnerability Database – Vulnerability Notification Services– Value Added Services


Brief HistoryBrief History• First VDBs were private, mostly maintained

by hackers or budding security geeks (before security professionals were common)

• First public database?– Unix Known Problem List– Internal Sun Microsystems Bug List– Early CERT database

• VDBs abandoned (Fyodor), sold to corporations (BID), or home grown (X-Force)

• Additional VDBs continued to be launched to meet different demands (Secunia, OSVDB)


Basics of a VDBBasics of a VDB

• Vulnerability information gathered• Identification number/name assigned• Adherence to standard format• Ability to search and display dataOptional:• Mail lists (private or public)• Exports for integration• Other services


Purposes of a VDBPurposes of a VDB

• Provide accurate information on security vulnerabilities

• Provide historic reference on software bugs

• Provide information on solutions • Provide innovations to help

organizations deal with vulnerabilities

But are they?


WIIFM – What’s in it for WIIFM – What’s in it for me?me?

• Alerting/Notification– Information provided in timely fashion

• Detailed Content– Concise description, additional

analysis, references

• Organized Information– Vulnerability statistics– Trending – Historical context


Vulnerabilities TrendsVulnerabilities Trends

CERT CERT VulnerabilitVulnerability Countsy Counts((1995-2004)1995-2004)


Who uses a VDB?Who uses a VDB?

• Administrators• Auditors• Security Testers

– Penetration Testing– Vulnerability Assessments– Risk Management

• Criminals– Hackers, Crackers, Blackhats,

Greyhats, OH MY!


Legalities and LiabilityLegalities and Liability

• Issues with disclosure– Bug finder and irresponsible disclosure– Do VDBs have a responsibility to be ethical for bug

finders?

• Liability for providing information– Liability for including exploit code?

• Copyrights on information– Including unedited original source?– Re-branding or re-writing?

• Confusing lawsuits– Tegam vs. Guillaume Tena (France)– Sybase vs. NGSS? (US)– HP vs. NGSS? (US)


VDB SociologyVDB Sociology

• VDBs are taken for granted by users• Users need them but do not appreciate• Users rely on a VDB for 'thoroughness',

when they usually are not• Users quote VDB information as gospel,

as if VDBs confirm and validate every entry

• Users typically have favorite VDB, and only use that one



Inherent Problems


Inherent Problems with Inherent Problems with VDBsVDBs• Dependency

– If no entry for Product X, assumption it is secure

– Assume information is accurate, becomes gospel– Rely on VDB to alert you?

• Lack of Updates– Hard to update old entries (why don’t new

players care about old entries?)– Solutions not there or not fully updated– Workarounds not accurate or helpful

• Thoroughness– “multiple” entries– No digging for details– Ignoring obscure products


• Lack of standard– Naming conventions– “multiples” vs. breaking out entries– What deserves an entry at all

• Accuracy and Integrity– Who updates? What motivation to be accurate?– Myth/Fake– Why is the information inaccurate?

• Poorly written advisory, Lousy research• Poor vendor communication/verification

– Why do VDBs trust anything and everything they read?

• Number of database entries matter

Inherent Problems with Inherent Problems with VDBsVDBs


Inherent Problems with Inherent Problems with VDBsVDBs• Pros & Cons of adding entries

– Fast• No external references• Incomplete or inaccurate information

– Slow• Not timely like many people want

• Statistics & Metrics– Lack of classification (leads to problems)– Lack of severity (debate unto itself)

• Not only based on remote vs. local …• Availability of exploit• Impact of exploit• Installation base of software


Inherent Problems with Inherent Problems with VDBsVDBs• Relying on Bug finders

– Double edge sword, good bug hunters provide great information, many do not

– Vulns being reported although previously disclosed

– Not including versions or vendor site, and not easily Google'd

– Vague information, untested – Advisories without dates (big vendors

especially guilty.. MS, IBM, Novell, Sun, HP)

– People try to use bug finding as a way to advertise their security services


Inherent Problems with Inherent Problems with VDBsVDBs• What else?

– Many don’t make database easily available in full or not portable

– Don’t support third party utilities and use

– VDB snobs, refuse to reference certain other databases or sources

– Narrow focus on where to find vulnerability information (life outside Bugtraq)

– Often don’t give credit where due– […]



Important Issues


Important Issues for Important Issues for VDBsVDBs

• Most issues are easily overlooked• 7 key issues for a VDB to address

•User Dependency•Content Updates•Content Depth•Standards•Accuracy and Integrity•Statistics and Metrics• Integration Ability


User DependencyUser Dependency

•Can you rely on a VDB?•Do you verify the VDBs

statements?•Do you read into the

information and make assumptions?

•Rely on VDB to alert you?


Content UpdatesContent Updates

• Turnaround on new entries• Older entries need attention

– Updated external references– Updated solutions– Updated information on risk ratings

• Do all VDBs care about older entries?

• Corrections to entries


Content DepthContent Depth

• Number of entries– Catalogue all vulnerabilities or just major

issues• Vague information on vulnerabilities

– Often due to poor research or vendor not providing details (thus, external references are important)

• Effort to correlate or research– Weeding out duplicate entries

• Types of products cataloged– Not just about Windows and Unix anymore


StandardsStandards

• Definition of a Vulnerability• Naming Conventions• Dates• Write-ups• Risk Ratings• Solutions


Accuracy and IntegrityAccuracy and Integrity

• Who maintains the data • How are updates justified• Motivation for entries• Motivation for accuracy


Statistics and MetricsStatistics and Metrics

• How many entries exist?• How many entries are missing?• How do we know?• How many entries have solutions?• How many are critical?• How many vulns per month/year?• How many vulns per

vendor/product?


Integration AbilityIntegration Ability

• Can users change or ask for updates

• Is the data easy to obtain• Does the VDB support 3rd parties• Does the VDB reference all

information• Can users dynamically pull

information



Major Players


Major PlayersMajor Players

• Comprehensive VDBs– BID - http://www.securityfocus.com/bid– CVE - http://www.cve.mitre.org/– ISS X-Force - http://xforce.iss.net/– OSVDB – http://www.osvdb.org/– Secunia - http://www.secunia.com/– Security Tracker -

http://www.securitytracker.com/• Vulnerability Notification Services

– CERT - http://www.cert.org/– CIAC Advisory -

http://www.ciac.org/ciac/index.html• Value Added Services

– ICAT - http://icat.nist.gov/icat.cfm


BIDBID

• Started in 1999, acquired by SecurityFocus on 07/17/2002

• Full time dedicated resources• Free, 72 hour delayed information

(SF researched)


BID – Pros/ConsBID – Pros/Cons

• Pros– Brand awareness– Very detailed and technical information provided– Quick posting of new vulnerabilities due to

hosting of Bugtraq mail list

• Cons– Practices changed once acquired by corporation– Little response to feedback provided– Slow to load, banners ads a pain, 39 images per

entry– Product information based on erroneous

assumptions


CVE/ICATCVE/ICAT

• MITRE and NIST• Full time dedicated resources,

federal funding• CVE started in 1999, ICAT ~2000• Both claim not to be a VDB• ICAT adds vulnerability classification

and statistics to a predominantly CVE based database

• Free


CVE/ICAT – Pros/ConsCVE/ICAT – Pros/Cons

• Pros– Detailed statistics and classification scheme– Easy ability to download entire database– Widely adopted, heavily integrated into security

products

• Cons– Heavy use of CVE for vulnerability information– CVE “candidate” process slow and backlogged– Limited external references (ICAT)


ISS X-ForceISS X-Force

• Run by Internet Security System (ISS)

• Full time resources dedicated• Started around Aug, 1997• VDB is free and public• Heavily used and referenced in ISS

security products• Fast and courteous reply to emails

with questions or errors


ISS X-Force – Pros/ConsISS X-Force – Pros/Cons

• Pros– Very detailed, very thorough, historical entries– Fairly standard naming conventions– Very thorough external references

• Cons– Disclosure Issues– Many entries related to IDS events, not classic

vulnerabilities– No easy export, can’t easily integrate


OSVDBOSVDB

• Open Security Foundation, 501(c)3 non-profit organization

• 3 project leaders, over 200 volunteers since inception

• First started on 08/30/2002• Free security information• Security community driven• Vendor dictionary, ethical disclosure

service, active integration


OSVDB – Pros/ConsOSVDB – Pros/Cons

• Pros– Vendor Neutral, Un-biased– Integration with open source products– Broad source for data importation (sources,

dates)– Very thorough, attention to detail, historical

entries

• Cons– Slow updates on new vulnerabilities– Relies on community for resources– Currently no long term funding


SecuniaSecunia

• Corporation located in Denmark• Full time staff• Launched 03/26/2003• Focus on timely vulnerability alerts• Free mailing list of new vulns

mailed daily


Secunia – Pros/ConsSecunia – Pros/Cons

• Pros– Free mailing list– Very strong on monitoring vendor advisories and

updates– Attempt to work with open source community

• Cons– Lack of standards/confusing standards

• Issues lumped into “multiple” entries• Same vulnerability assigned a dozen entries, one per linux

vendor

– Only focuses on new vulnerabilities– Some solutions not practical or helpful


Security TrackerSecurity Tracker

• Corporation in MD, USA• Full time resources dedicated• Started in 2002• Free weekly summary of

vulnerabilities, fee for instant alerts


Security Tracker – Security Tracker – Pros/ConsPros/Cons

• Pros– Maintain their own standards, uniform

entries– Includes data source for vulnerability– Good data importation, monitor broad

source of information

• Cons– No statistics– Limited external references


CERTCERT

• Carnegie Mellon, funded by US government

• Full time staff dedicated • Started in 1988, after Morris worm• Advisories for important issues• Maintains CERT-VU/KB Database• National Cyber Alert System


CERT – Pros/ConsCERT – Pros/Cons

• Pros– US Federally funded and supported– Providing reports to technical and non-technical– Statistics provided

• Cons– Limited vulnerabilities tracked– Provide early information for exorbitant fee– Not always willing to coordinate with security

community– Serious questions about statistics, efficiency of

staff/funds– Overlap with CIAC and others


CIACCIAC

• US funded and supported, DOE• Full time dedicated resources• Started in 1989• Advisories for major issues• Free service


CIAC – Pros/ConsCIAC – Pros/Cons

• Pros– Stability, around since 1989– Updated regularly

• Cons– Limited vulnerabilities covered– Limited external references– Many advisories reprinted, no value added– Overlap with CERT


Additional ResourcesAdditional Resources

• Vulnerability Sources Not Included:– COOP =

https://cirdb.cerias.purdue.edu/coopvdb/public/– Dragonsoft - http://vdb.dragonsoft.com/– FrSIRT -

http://www.frsirt.com/english/index.php– Securiteam - http://www.securiteam.com/– Sec Watch- http://www.secwatch.org/

• Focused Vulnerability Database– Nikto, Nessus– Sun, HP, IBM, Oracle, Microsoft, etc

• Vulnerability Sharing Clubs– http://www.idefense.com/– http://www.immunitysec.com


Government FundedGovernment Funded

• CERT– The CERT/CC is funded primarily by the U.S. Department of Defense and

the Department of Homeland Security, along with a number of other federal civil agencies. Other funding comes from the private sector. As part of the Software Engineering Institute, we receive some funds from the primary sponsor of the SEI, the Office of the Under Secretary of Defense for Acquisition and Technology.

• CIAC– U.S. Department of Energy (DOE) funded

• CVE– CVE is sponsored by the National Cyber Security Division (NCSD) at the

U.S. Department of Homeland Security. US-CERT is the operational arm of the NCSD.

• ICAT– ICAT is maintained by the National Institute of Standards and

Technology.• US-CERT

– US-CERT is part of the Department of Homeland Security

• Little overlap? Consolidation? Oversight and audit?



Research and Rankings


Data HarvestingData Harvesting

• Where is information usually gathered?– Mail lists (Bugtraq, Full-disclosure, Vulnwatch,

Ntbugtraq)– Vendors (advisories)

• Where else should information be gathered?– Mail lists (Freshmeat, Vuln-dev, Dailydave, Pen-

test, other specialty security focused lists)– Vendors (Changelogs, Knowledge bases, Vendor

forums)– Exploit archives


VDB IncestVDB Incest

• Who references who? Who refuses?– CVE: ISS, BID, Secunia, SecurityTracker, OSVDB– BID: CVE, Bugtraq, ISS, Secunia,

SecurityTracker, OSVDB– ISS: CVE, BID, Secunia, SecurityTracker, OSVDB– Secunia: CVE, OSVDB– SecurityTracker: CVE, OSVDB, Nessus– Nessus: CVE, BID, OSVDB– OSVDB: CVE, BID, Secunia, SecurityTracker,

ISS, Nessus, Snort, more

• Red denotes an apparent refusal to reference, even if the original point of disclosure or only available source.


VDB RatingsVDB Ratings

• Based on important issues identified• Score of 1-10 provided for each of the

7 key performance areas• 1 = lowest, 10 = highest• Ratings given for each issue per VDB• Provides baseline for expectations for

each service• Identifies areas of improvements


VDB Individual RankingsVDB Individual Rankings

• Ratings For Each Category• Top 3 VDBs• Top 3 Areas for VDB Improvement

See research posted at:• http://

www.opensecurityfoundation.org



Future


Future of VDBsFuture of VDBs

• Long way to go• Hope to improve existing resources

– Better search interfaces– Better upkeep of older entries

• More services available to more people

• Further integration into products• Better statistics and trending


Standardization of Standardization of DefinitionsDefinitions

• Risk ratings• Vulnerability Classifications

– Local vs. Remote (Remote Local)– Impact assessment (CIA)– Exploit availability– Access required to exploit

(Dependencies)

• Vulnerability definitions and terminology


VDBs Suck - Expect MoreVDBs Suck - Expect More

• 20 years since inception, Limited improvements• Same mechanism for updating/verifying info• Very few classify or assign risk• Still no standardized classification for the few who do• Still no standardized risk value for the few who do • Still offer limited search ability overall • Many don't follow their own standards consistently• Most still very weak on external references • Barely any new services or ways to use information • Many don't seem to care about the vuln disclosure

process (why did it take 20 years for a vendor dict to emerge?)

• Bottom line, VDBs need to drastically improve


Open Security Open Security FoundationFoundation


Brian Martin – [email protected] Kouns – [email protected]

© Open Security Foundation 2005 Vulnerability Databases: Everything is Vulnerable Brian Martin Jake Kouns.

Documents

open security foundation

vdbs slide

security services

security vulnerabilities

security professionals

vdb vulnerability information

vdb information

database of information