Open benchmark library · version 1.0.0
Bushfire decisions, benchmarked on public data.
AUFireBench turns 34 seasons of mapped fire perimeters, the national Census and the road network into optimisation problems that any solver, heuristic or AI method can attack. One independent checker scores every answer the same way.
National insights from the public layer
Where exposure, isolation and vulnerability meet
The same tables that feed the benchmark answer screening questions for agencies and councils: which localities are threatened often, which lose every egress route, and where help is far away.
Fire seasons by state
Localities with a mapped perimeter within 10 km, 1990-91 to 2023-24
Share of expected exposure
Dwellings times share of seasons threatened, by state
Locality ranking
Sort by the question you are asking; the bar shows the selected measure
| # | Locality | State | Dwellings | Threatened | Exits cut | Station min | Selected measure |
|---|
Threatened and cut off
Each dot a locality; size shows dwellings
Screening measures from public data, not risk ratings. A locality is threatened in a season when a mapped perimeter of at least 50 hectares lies within 10 km; an exit route is crossed when a perimeter buffered by 100 m meets it, a geometric proxy not validated against recorded closures. Seasons run July to June; 2023-24 is partial. Localities are ABS Urban Centres and Localities outside Major Urban areas, so most residents in peri-urban mesh blocks are not represented.
AI assistant · grounded in this dataset
Ask the data
Answers come only from the AUFireBench tables on this page and follow the state filter above.
Data assistant
Filter: All statesTry a question
Each one is sent straight away.
Screening measures from public data, not risk ratings. Answers are AI-generated from the tables on this page; check figures against the charts.
How it works
From public records to a verified decision
Every step is scripted and checksummed, so a result in Melbourne can be reproduced in Munich. The checker never reads a solver's own report; it recomputes the answer from the tables.
Public records
Census, fire perimeters, roads and stations, each listed with publisher, licence and SHA-256.
Instance
CSV tables plus a solver-neutral
model.mps; small cases also ship as QUBO.Any method
Commercial or open solvers, heuristics, learned policies, annealers.
Independent check
aufirebench checkaccepts or rejects and recomputes the value, with no solver call.Public ledger
Verified improvements enter a dated best-known ledger through a pull request.
Five problem classes
Planning decisions that recur every fire season
Each class is a familiar supply-chain pattern applied to a hazard: siting, network hardening, fleet dispatch under closures, and pooling a scarce workforce.
Equal-hardware solver comparison
Four solvers, one machine, one thread
Each solver read the same 57 model files on an Apple M3 with one thread and a 30-minute limit. Every returned answer passed the checker, and proven values agree to one part in a million.
Proven within 1,800 seconds. Gurobi 13 is commercial; HiGHS 1.15 and SCIP 10 are open source; CP-SAT 9.15 is the constraint-programming solver of Google OR-Tools. The best of all solvers proves 53 instances, and four bus-staging instances are still open.
Instance explorer
Every instance, its best known answer and how hard it is
Open instances are invitations: beat the best known value, pass the checker, and your result enters the ledger. Select a row for details.
| Instance | Class | Columns | Best known | Status | Solve time, log scale |
|---|
Instance space
Where the hard problems live
Seven structural features of each model, projected to two axes. The horizontal axis tracks model size; the vertical axis tracks the time-expanded structure of bus staging.
Open-source solvers fail in two regions: the bus-staging cluster at the top and the largest crew models at the right. Hollow markers are 19 instances that were not used to fit the map, and they land beside their own class.
Why the numbers can be trusted
A checker that was tested to fail
We built 1,863 deliberately altered answers of 89 kinds, from budget overruns to tampered tables. A separately written oracle labelled each one before the checker saw it.
infeasible answers rejected, including negative or fractional counts, buses leaving after a road closed, and unstaffable crews.
feasible answers accepted with their value recomputed to a relative 10-9.
automated tests on Python 3.10 to 3.12; make check re-verifies every stored solution in about 20 seconds.
# install, build a Tasmanian road-protection instance, solve it, verify it pip install -e . aufirebench generate P2 --region TAS --budget_km 50 --out inst aufirebench solve inst/p2_TAS_K3_B50_S34 --time 60 aufirebench check inst/p2_TAS_K3_B50_S34 inst/p2_TAS_K3_B50_S34/solution.json # proven optimum 102.100382, 49.05 of 50 km protected
Get started
Install the library, or plug it into an AI assistant
Python 3.10 to 3.12. The instances ship with the code, so nothing else is downloaded to reproduce a result. The code is shared with reviewers now and will be public on release.
# 1. get the code and create an environment git clone https://github.com/<owner>/aufirebench.git && cd aufirebench python -m venv .venv && . .venv/bin/activate # 2. install with HiGHS; add SCIP and Gurobi interfaces if you have them pip install -e . pip install -e ".[solvers]" # 3. confirm every released instance and stored solution make check
# build, solve and verify a Tasmanian road-protection instance aufirebench generate P2 --region TAS --budget_km 50 --out inst aufirebench solve inst/p2_TAS_K3_B50_S34 --time 60 --out /tmp/sol.json aufirebench check inst/p2_TAS_K3_B50_S34 /tmp/sol.json --bks echo $? # 0 valid, 20 below best known, 21 infeasible, 10 invalid
# install the optional assistant interface (Model Context Protocol) pip install -e ".[mcp]" # claude_desktop_config.json (or any MCP client that starts a stdio server) { "mcpServers": { "aufirebench": { "command": "/path/to/aufirebench/.venv/bin/aufirebench-mcp", "env": { "AUFIREBENCH_DATA": "/path/to/aufirebench/data/instances", "AUFIREBENCH_LEDGER": "/path/to/aufirebench/results/bks_ledger.csv" } } } }
What the assistant can do
Seven tools over a local connection. Only the checker reports a value, and every answer carries hashes of the instance and the decision.
list_instancesbrowse by class or tier
describe_classdecision, data and verifier test
load_instancetables and checksums
solveHiGHS, at most 60 s, marked unverified
check_decisionthe only source of an objective
compare_to_bksgap to the best known value
server_infoversions and stamps
Try asking: "Solve the smallest P1 instance, verify it and report the gap to the best known value."
Who it is for
One library, four kinds of users
The instances screen options on public data. They are not operational advice, and no agency has adopted them.
Screen a region in an afternoon
Change a budget, a reach or a hub count, regenerate the instance for your own state, and compare plans on the same footing.
Test on real structure
Hard, stress and QUBO tiers with reference values that do not depend on your own solver.
Publish reproducible results
Submit a decision file; the checker and the ledger do the rest.
Teach with real data
Covering, two-stage stochastic and staffing models that run in one command, already used in a class lab.
Version 1.1 · in progress
Checking the release against new public records
Version 1.0.0 stays the released and cited version: its 57 instances, tiers, ledger and paper do not change. Version 1.1 adds validation and calibration data only. Results below are first runs of 4 November 2026.
E80 · road failure rule
recall of the 100 m rule against observed bushfire closures in Victoria, season 2025-26 (14 of 25 closed route edges; 58 fires of at least 50 ha). Precision 0.07 (14 of 198 predicted edges) is a lower bound: the feed does not report every closed road. Across buffers of 50 to 250 m and windows of 7 to 30 days: recall 0.52 to 0.68, precision 0.06 to 0.10. 75% of the season's 406 closure points lie within 1 km of a mapped perimeter. Season 2024-25 is not testable (0 observed on-route closures).
E81 · NSW fire stations
Geoscience Australia rural stations in NSW that match a NSW Rural Fire Service facility within 500 m (median offset 3.4 m). 773 of the 2,287 NSW RFS facilities have no GA station within 500 m, so the crew classes see fewer NSW stations than exist.
E84 · NSW union instances
stations in the rebuilt NSW crew instances: 1,558 GA stations plus 430 RFS brigade stations without a GA match. The same 50,520 volunteers spread thinner (median per station 13.0 to 10.5). Release folders untouched.
E83 · Ordinary traffic already uses road capacity
Victorian AADT 2019, 14,439 directional sections matched to the road graph. Design-hour volume (k = 0.10, peak direction 0.60) as a share of the capacity the traffic simulator assumes. Dot: median section; ring: 90th percentile.
The simulator assumes empty roads, so its clearance times are optimistic on trunk (0.31 at the median) and primary roads (0.47). Motorway ratios above 1 mostly reflect urban sections with more lanes than assumed. National Harmonised Traffic Counts are downloaded; their bins are totals over each counting period and are converted to daily volumes before a national table is published.
E84 · Objective against the release ledger
NSW crew instances (H = 60) re-solved on the union of 1,988 stations, Gurobi, 2 threads, limit 30 min, against the release best known value (gap 1e-6).
| Instance | release | union | change |
|---|---|---|---|
| p4_NSW_H60_r20_t60_a25_s1 | 62.486 | 62.485 (time limit, gap 0.002%) | -0.00% |
| p5_NSW_H60_r20_t60_a15_s1 | 62.460 | 62.358 | -0.16% |
| p5_NSW_H60_r20_t60_a25_s1 | 62.486 | 62.486 | +0.00% |
| p5_NSW_H60_r20_t60_a40_s1 | 62.486 | 62.494 | +0.01% |
E82 · Disaster funding activations
The DRFA activation history has no NSW or Victorian bushfire activation for season 2019-20, so the file is not a complete record of earlier events. It is reported but not used as validation.
New sources
| Publisher | Dataset | Use | Licence | Redistributed |
|---|---|---|---|---|
| National Freight Data Hub | Harmonised road closures, fire records | E80 check of the road failure rule | none stated | no, aggregates only |
| National Freight Data Hub | Harmonised Traffic Counts | E83 capacity check | none stated | no, aggregates only |
| Dept of Transport and Planning, Victoria | Traffic Volume (AADT) 2019 | E83 capacity check | CC BY 4.0 | yes, with attribution |
| DEECA, Victoria | Fire history scars, season 2025-26 | E80 perimeters | CC BY 4.0 | yes, with attribution |
| Spatial Services NSW | Features of Interest, NSW RFS facilities | E81 station check, E84 | not stated | no, aggregates only |
| NEMA (data.gov.au) | DRFA activation history by LGA | E82 state season check | check dataset page | no, aggregates only |
| National Aerial Firefighting Centre | ITT AFS 2021 Table of Services | reference only | no open licence | no |
Raw files stay with the author; this page shows aggregates only. Details: docs/v11_validation_data.md in the repository.
National roadmap
From bushfire to national freight, resilience and sustainability
Eight public datasets planned for the next versions, each opening a new problem class on the same checker and ledger. Licences marked "to verify" are checked before any data is redistributed.
Showcase and explorer
- Instance explorer and best-known ledger online
- Licence and attribution page for every source
- Automatic rebuild after each verified submission
Freight and flood classes
- Road and rail freight network protection on the Key Freight Routes
- Flood closure scenarios from satellite water observations
- Interactive maps of instances and solutions
Check it in the browser
- The same checker running in the browser for small instances
- Online submission of decisions
- Coastal, heat, emissions and energy classes
About this tool
The concept
A benchmark library: recurring bushfire planning decisions (siting, network hardening, dispatch under road closures, pooling a scarce workforce) written as optimisation instances built from public fire perimeters, Census and road data.
How it works
Each instance ships as CSV tables plus a solver-neutral model file. Any method can attack it, and one independent checker recomputes every answer from the tables without calling a solver. Verified improvements enter a dated best-known ledger.
Winning strategy
Trust only checked numbers. Start from the instance explorer, beat the best known value on an instance that matters to your decision, and let the checker, not the solver's own report, confirm it.