Objective

Create tools to analyze Slang's GitHub issues and pull requests to systematically identify gaps, bug-prone components, and areas needing improvement. Combine analysis results with test coverage data for test planning.

Proposed Solution

Add a suite of Python scripts in tools/issue-analysis/ to:

1. Fetch GitHub data - Download issues and PRs with full historical data, file changes, and relationships
2. Analyze bug patterns - Identify bug reports, categorize by type and severity, map to compiler components
3. Identify hotspots - Find files and components most frequently involved in bug fixes
4. Track metrics - Measure bug rates, test coverage, time to fix, and trends over time
5. Compare analysis results with coverage data - Identify need for new tests based on coverage data and issue analysis results

Key Features

• Incremental updates (fetch only new data)
• Component-level analysis (IR passes, semantic checker, emitters, etc.)
• Critical issue detection (crashes, ICEs, validation errors)
• Test coverage tracking by bug type
• Historical trend analysis

Key Benefits

Aim is to create analysis tools for:

• Data-driven identification of quality gaps
• Prioritization of testing and refactoring efforts
• Baseline for measuring quality improvements
• Better onboarding for new contributors
• Risk assessment for releases
• Output of the scripts can be used for planning improvements

Implementation

Pure Python scripts with no build dependencies. Tools can be run locally or integrated into CI for periodic reporting.