Static Analysis Tool Comparison Report Generator

WHAT IT DOES

Static Analysis Tool Comparison Report Generator ingests scanner output files from multiple tools and produces a consolidated comparison report. It reads JSON or plaintext findings from local configuration or pipeline input, optionally supplementing with contract source code and compiler metadata from the filesystem for context. The tool normalizes findings across different scanner schemas, groups them by rule or category, and highlights overlaps, unique detections, and matches against user-supplied comparison criteria. It generates both summary statistics and detailed per-finding tables showing which tools flagged each issue. The output is designed for security engineers reviewing scan results, helping them identify consensus findings versus tool-specific noise without manually cross-referencing multiple reports.

HOW TO USE

Provide one or more scanner output files via the web interface or command-line input. The tool accepts JSON formats from Slither, Mythril, Oyente, Securify, SmartCheck, and generic plaintext listings. Optionally, paste a list of comparison criteria (one rule name per line) to filter and highlight specific categories of interest. Click "Generate comparison report" to produce a visual table showing all findings, their frequency across tools, and which criteria they satisfy. The report displays total findings, unique categories, tool count, and a dedicated section for criteria matches. All processing is local—no data leaves your browser or environment. For pipeline integration, the tool can also read from stdin and output JSON or markdown reports.

TECHNICAL MECHANISM: SCHEMA NORMALIZATION AND MULTI-AXIS GROUPING

Unlike simple diff tools that compare raw text, this tool builds a normalized finding taxonomy by extracting rule identifiers, severity levels, and contract locations from each scanner's proprietary schema. It uses a fuzzy matcher that strips prefixes, suffixes, and common variations (e.g., "reentrancy" vs "reentrancy-eth" vs "REENTRANCY") to group semantically equivalent findings across tools. The comparator maintains a multi-dimensional index mapping (rule, contract, line_range) to identify exact matches, partial overlaps, and unique detections. For user-provided criteria, it performs case-insensitive substring and regex matching against normalized rule names. The entire report is rendered client-side with O(n log n) sorting and aggregation, scaling to 10,000+ findings across five tools without noticeable latency. The source code parser extracts compiler version and optimization settings from Solidity metadata, adding context to findings without executing any analysis itself.

WHAT IT CANNOT SEE

Cannot execute scanners or generate raw findings—it only processes already-generated outputs. Cannot compare semantic meanings of findings across tools with different rule taxonomies beyond text-based normalization; a "reentrancy" finding in one tool may have different severity criteria than in another. Cannot determine which tool is "correct" when findings differ, as there is no ground truth. Cannot assess severity or exploitability; severity fields are displayed as reported but not validated. Relies entirely on user-provided scanner outputs being current and correctly formatted—stale or malformed inputs produce misleading comparisons. Cannot identify false negatives (findings that no scanner caught) because it only sees reported issues. Cannot validate that comparison criteria are comprehensive or unbiased—users must supply appropriate rules for their review context. Does not perform any static analysis itself, does not compile contracts, and does not access on-chain state.

PLEASE NOTE

EVM chains only. This tool processes Solidity and Vyper source metadata but does not execute analysis. All processing is read-only and local.