Data Dictionary and Schema Overview¶

Database Scale & Statistics¶

pie title CVE-Product/Package Pairs Distribution
    "Red Hat (9M)" : 9000000
    "Cisco (385K)" : 385000
    "GitHub (297K)" : 297000
    "Microsoft (185K)" : 185000
    "MoreFixes (35K)" : 35276

Table Overview & Relationships¶

graph TB
    subgraph "Core Vulnerability Data"
        CVE[cve_main<br/>Primary CVE Repository]
        EXPLOITS[exploits<br/>ExploitDB Public Exploits]
    end

    subgraph "Commercial Vendor Patches"
        MSRC[msrc_patches<br/>Microsoft: 185K pairs]
        CISCO[cisco_patches<br/>Cisco: 385K pairs]
        REDHAT[redhat_patches<br/>Red Hat: 9M pairs]
    end

    subgraph "Open Source Intelligence"
        GITHUB[github_advisories<br/>GitHub: 297K pairs]
        MOREFIXES[morefixes_*<br/>Academic: 35K commits]
    end

    subgraph "Reference Data"
        CWE[cwe_ref<br/>MITRE CWE Catalog]
        CAPEC[capec_ref<br/>MITRE CAPEC Patterns]
    end

    CVE --> EXPLOITS
    CVE --> MSRC
    CVE --> CISCO
    CVE --> REDHAT
    CVE --> GITHUB
    CVE --> MOREFIXES
    CVE --> CWE
    CVE --> CAPEC

    style CVE fill:#e1f5fe
    style EXPLOITS fill:#fff3e0
    style MSRC fill:#e8f5e8
    style CISCO fill:#e8f5e8
    style REDHAT fill:#e8f5e8
    style GITHUB fill:#f3e5f5
    style MOREFIXES fill:#f3e5f5

Core Tables¶

cve_main - Primary CVE Repository¶

exploits - Public Exploit Repository¶

Commercial Vendor Patches¶

msrc_patches - Microsoft Security Response Center¶

redhat_patches - Red Hat Security Advisories¶

cisco_patches - Cisco Product Security¶

Open Source Intelligence¶

github_advisories - GitHub Security Database¶

Academic Research Data¶

MoreFixes Dataset Family¶

graph TB
    subgraph "MoreFixes Schema"
        CVE_MF[morefixes_cve<br/>CVE Metadata]
        FIXES[morefixes_fixes<br/>CVE-Commit Links]
        COMMITS[morefixes_commits<br/>Commit Details]
        REPOS[morefixes_repository<br/>Repository Info]
        FILES[morefixes_file_change<br/>File-level Changes]
        METHODS[morefixes_method_change<br/>Method-level Changes]
        CWE_CLASS[morefixes_cwe_classification<br/>CVE-CWE Mapping]
        CWE_MF[morefixes_cwe<br/>CWE Reference]
    end

    CVE_MF --> FIXES
    FIXES --> COMMITS
    COMMITS --> REPOS
    COMMITS --> FILES
    FILES --> METHODS
    CVE_MF --> CWE_CLASS
    CWE_CLASS --> CWE_MF

    style CVE_MF fill:#e3f2fd
    style FIXES fill:#f3e5f5
    style COMMITS fill:#e8f5e8
    style FILES fill:#fff3e0

Reference Tables¶

cwe_ref - Common Weakness Enumeration¶

capec_ref - Common Attack Pattern Enumeration¶

Data Quality & Processing Notes¶

Important Data Considerations¶

WHERE (
    LOWER(product_name) REGEXP '(rh|red.hat|red-hat|rhel|enterprise.linux|baseos|appstream|openshift)'
    OR LOWER(product_id) REGEXP '(rh|red.hat|red-hat|rhel|enterprise.linux|baseos|appstream|openshift)'
)

Record Duplication Patterns¶

CVE-Product/Package Pairs

All vendor patch tables contain intentional duplication where a single CVE affects multiple products:

graph LR
    CVE[CVE-2023-12345] --> P1[Product A v1.0]
    CVE --> P2[Product A v2.0]
    CVE --> P3[Product B v1.5]
    CVE --> P4[Product C v3.2]

    style CVE fill:#ffebee
    style P1 fill:#e8f5e8
    style P2 fill:#e8f5e8
    style P3 fill:#e8f5e8
    style P4 fill:#e8f5e8

Examples of legitimate duplication: - Microsoft: Single advisory covering multiple Windows versions - Red Hat: CVE affecting multiple RHEL releases and packages - Cisco: Network vulnerability across multiple router models - GitHub: Package vulnerability affecting multiple ecosystem versions

Dataset Comparison Guidelines¶

Query Performance & Optimization¶

-- Efficient temporal analysis with proper indexing
SELECT 
    EXTRACT(YEAR FROM date_published) as year,
    COUNT(*) as cve_count,
    AVG(cvss_v3_score) as avg_severity
FROM cve_main 
WHERE date_published >= '2020-01-01'
    AND cvss_v3_score > 0
GROUP BY 1
ORDER BY 1;

-- Multi-vendor patch comparison with proper filtering
WITH vendor_patches AS (
    SELECT cve_id, release_date, 'Microsoft' as vendor 
    FROM msrc_patches
    UNION ALL
    SELECT cve_id, current_release_date, 'Cisco' 
    FROM cisco_patches
    UNION ALL
    SELECT cve_id, current_release_date, 'RedHat'
    FROM redhat_patches 
    WHERE LOWER(product_name) REGEXP 'rhel|enterprise.linux'
)
SELECT vendor, COUNT(*) as patch_count
FROM vendor_patches
GROUP BY vendor;

Research Applications & Use Cases¶

mindmap
  root((Data Applications))
    Academic Research
      Vulnerability Lifecycle
        Time-to-Exploit Analysis
        Patch Response Metrics
        Cross-Vendor Comparison
      Economic Analysis
        Resource Allocation
        Cost-Benefit Models
        Risk Assessment
      Predictive Modeling
        ML Feature Engineering
        Temporal Validation
        Cross-Ecosystem Training
    Industry Intelligence
      Threat Assessment
        Exploitation Prediction
        Priority Ranking
        Risk Scoring
      Patch Management
        Response Planning
        Resource Optimization
        Vendor Evaluation
      Security Operations
        Threat Hunting
        Incident Response
        Vulnerability Triage
    Policy Research
      Disclosure Coordination
        Vendor Response Analysis
        Timeline Optimization
        Best Practice Development
      Regulatory Impact
        Compliance Metrics
        Policy Effectiveness
        Industry Standards

Data Validation & Quality Assurance¶