|
| 1 | +# Bijectors.jl Development Instructions |
| 2 | + |
| 3 | +**ALWAYS** follow these instructions and only fallback to additional search and context gathering if the information here is incomplete or found to be in error. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +Bijectors.jl is a Julia package for transforming probability distributions, implementing both an interface for transforming distributions from Distributions.jl and many transformations needed in this context. Used heavily in the probabilistic programming language Turing.jl. |
| 8 | + |
| 9 | +## Working Effectively |
| 10 | + |
| 11 | +### Quick Setup and Build |
| 12 | + |
| 13 | +```bash |
| 14 | +# Install dependencies (NEVER CANCEL: takes ~30 seconds) |
| 15 | +julia --project=. -e "using Pkg; Pkg.instantiate()" |
| 16 | + |
| 17 | +# Test package loading |
| 18 | +julia --project=. -e "using Bijectors; println(\"Package loaded successfully\")" |
| 19 | +``` |
| 20 | + |
| 21 | +### Running Tests |
| 22 | + |
| 23 | +```bash |
| 24 | +# Interface tests only (NEVER CANCEL: takes ~7.5 minutes, timeout 30+ minutes) |
| 25 | +julia --project=. -e "ENV[\"GROUP\"] = \"Interface\"; using Pkg; Pkg.test()" |
| 26 | + |
| 27 | +# AD tests with specific backend (NEVER CANCEL: takes ~1.5 minutes, timeout 10+ minutes) |
| 28 | +julia --project=. -e "ENV[\"GROUP\"] = \"AD\"; ENV[\"AD\"] = \"ForwardDiff\"; using Pkg; Pkg.test()" |
| 29 | + |
| 30 | +# Full test suite (NEVER CANCEL: may take 20+ minutes and some Enzyme tests may fail - this is expected) |
| 31 | +julia --project=. -e "using Pkg; Pkg.test()" |
| 32 | +``` |
| 33 | + |
| 34 | +**CRITICAL**: |
| 35 | + |
| 36 | + - **NEVER CANCEL** any test commands - they can take significant time to complete |
| 37 | + - Set timeouts to 30+ minutes for Interface tests, 60+ minutes for full test suite |
| 38 | + - Enzyme tests may fail due to system compatibility issues - this is expected and not a blocker |
| 39 | + |
| 40 | +### Documentation |
| 41 | + |
| 42 | +```bash |
| 43 | +# Build documentation (NEVER CANCEL: takes ~47 seconds, timeout 5+ minutes) |
| 44 | +julia --project=docs -e "using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate(); include(\"docs/make.jl\")" |
| 45 | +``` |
| 46 | + |
| 47 | +### Code Formatting |
| 48 | + |
| 49 | +```bash |
| 50 | +# Install formatter (one-time setup, takes ~45 seconds) |
| 51 | +julia --project=. -e "using Pkg; Pkg.add(\"JuliaFormatter\")" |
| 52 | + |
| 53 | +# Format code |
| 54 | +julia --project=. -e "using JuliaFormatter; format(\".\")" |
| 55 | +``` |
| 56 | + |
| 57 | +## Validation Scenarios |
| 58 | + |
| 59 | +**ALWAYS** test basic functionality after making changes: |
| 60 | + |
| 61 | +```julia |
| 62 | +# Test basic bijector operations |
| 63 | +using Bijectors, Distributions |
| 64 | + |
| 65 | +# Test 1: Basic bijector for LogNormal |
| 66 | +d = LogNormal() |
| 67 | +b = bijector(d) |
| 68 | +x = 1.0 |
| 69 | +y = b(x) # Should return 0.0 |
| 70 | + |
| 71 | +# Test 2: Inverse transformation |
| 72 | +x_reconstructed = inverse(b)(y) # Should return 1.0 |
| 73 | + |
| 74 | +# Test 3: Log absolute determinant of Jacobian |
| 75 | +logjac = logabsdetjac(b, x) |
| 76 | + |
| 77 | +# Test 4: Combined transformation |
| 78 | +z, logabsdet = with_logabsdet_jacobian(b, x) |
| 79 | +``` |
| 80 | + |
| 81 | +## Test Groups and Timing Expectations |
| 82 | + |
| 83 | + - **Interface tests**: ~7.5 minutes - Core functionality tests |
| 84 | + - **AD tests (single backend)**: ~1.5 minutes - Automatic differentiation tests |
| 85 | + - **Full test suite**: 20+ minutes - All tests including multiple AD backends |
| 86 | + - **Documentation build**: ~47 seconds - Generate documentation |
| 87 | + - **Package installation**: ~27 seconds - Install dependencies |
| 88 | + |
| 89 | +## CI and Development Workflow |
| 90 | + |
| 91 | +### Before Committing Changes |
| 92 | + |
| 93 | + 1. **ALWAYS** run Interface tests: `ENV["GROUP"] = "Interface"; Pkg.test()` |
| 94 | + 2. **ALWAYS** run formatting: `using JuliaFormatter; format(".")` |
| 95 | + 3. Test basic functionality scenarios listed above |
| 96 | + 4. If changing AD-related code, run specific AD backend tests |
| 97 | + |
| 98 | +### CI Workflows |
| 99 | + |
| 100 | + - **Interface tests**: Core functionality (runs on PRs) |
| 101 | + - **AD tests**: Multiple AD backends including ForwardDiff, ReverseDiff, Tracker, Enzyme, Mooncake |
| 102 | + - **Format check**: Uses TuringLang/actions/Format |
| 103 | + - **Documentation**: Builds and deploys docs |
| 104 | + |
| 105 | +## Common Issues and Workarounds |
| 106 | + |
| 107 | + - **Enzyme tests failing**: Expected on some systems - not a blocker for development |
| 108 | + - **Network issues during package installation**: May see "could not download" warnings but installation continues |
| 109 | + - **Documentation git warnings**: Expected in sandboxed environments |
| 110 | + - **Deprecation warnings**: Some warnings about MatrixReshaped are expected |
| 111 | + |
| 112 | +## Key Repository Structure |
| 113 | + |
| 114 | +``` |
| 115 | +. |
| 116 | +├── Project.toml # Package definition and dependencies |
| 117 | +├── src/ # Main source code |
| 118 | +│ ├── Bijectors.jl # Main module |
| 119 | +│ ├── interface.jl # Core interface definitions |
| 120 | +│ ├── chainrules.jl # AD integration |
| 121 | +│ └── bijectors/ # Individual bijector implementations |
| 122 | +├── test/ # Test suite |
| 123 | +│ ├── runtests.jl # Main test runner |
| 124 | +│ ├── ad/ # AD-specific tests |
| 125 | +│ └── bijectors/ # Bijector-specific tests |
| 126 | +├── docs/ # Documentation |
| 127 | +├── ext/ # Package extensions for AD backends |
| 128 | +└── .github/workflows/ # CI configuration |
| 129 | +``` |
| 130 | + |
| 131 | +## Dependencies and Extensions |
| 132 | + |
| 133 | +Core dependencies include Distributions.jl, ChainRulesCore, and various math packages. Extensions provide integration with AD backends: |
| 134 | + |
| 135 | + - ForwardDiff, ReverseDiff, Tracker, Enzyme, Mooncake |
| 136 | + - DistributionsAD, LazyArrays |
| 137 | + |
| 138 | +## Manual Validation Requirements |
| 139 | + |
| 140 | +After any changes: |
| 141 | + |
| 142 | + 1. Build the package successfully |
| 143 | + 2. Run Interface tests (minimum requirement) |
| 144 | + 3. Test at least one complete bijector transformation scenario |
| 145 | + 4. Verify inverse transformations work correctly |
| 146 | + 5. Check log absolute determinant Jacobian calculations |
| 147 | + 6. Run formatter before committing |
| 148 | + |
| 149 | +The package should load without errors and basic transformations should work as demonstrated in the validation scenarios above. |
0 commit comments