16 minutes read

v0.1 validation

Offing v0.1 was validated on 2026-10-14 against 20 public repositories pinned to immutable commits: ten Node.js repositories, nine Python repositories, and one Node.js/GitHub Actions repository. The complete corpus and expected finding locations live in benchmarks/public-repositories/; the executable harness is benchmarks/public-repositories.ts.

Results

Moved to Validation corpus: Results.

GitHub Actions corpus additions

Moved to Validation corpus: GitHub Actions corpus additions.

Method

Moved to Validation corpus: Method.

The 50-repository corpus

Moved to Validation corpus: The 50-repository corpus.

Kubernetes corpus

Moved to Ecosystem corpora: Kubernetes corpus.

Moved to Ecosystem corpora: Kubernetes blind selection.

Java and Kotlin corpus

Moved to Ecosystem corpora: Java and Kotlin corpus.

Moved to Ecosystem corpora: Java and Kotlin blind selection.

Scala and sbt corpus

Moved to Ecosystem corpora: Scala and sbt corpus.

Moved to Ecosystem corpora: Scala and sbt blind selection.

Ruby corpus

Moved to Ecosystem corpora: Ruby corpus.

PHP corpus

Moved to Ecosystem corpora: PHP corpus.

Swift corpus

Moved to Ecosystem corpora: Swift corpus.

Moved to Ecosystem corpora: Swift blind selection.

Serverless corpus

Moved to Ecosystem corpora: Serverless corpus.

Moved to Ecosystem corpora: Serverless blind selection.

Moved to Ecosystem corpora: .NET blind selection.

CI runner images corpus

Moved to Ecosystem corpora: CI runner images corpus.

C and C++ corpus

Issue #1897 promotes the C/C++ detector to stable. The corpus already held twelve C/C++ ecosystem entries (four targeted migration pins and eight blind pins from the 300-repository pass). The gap for Supported was zero-finding controls: only catchorg/Catch2 had an empty expected list.

The blind zero-finding rule, fixed on 2026-11-07 before Offing ran on these additions: for each of cmake_minimum_required(VERSION 3.20, cmake_minimum_required(VERSION 3.22, and cmake_minimum_required(VERSION 3.25, run gh search code <query> --filename CMakeLists.txt --limit 50, walk the results in order, and keep unique repositories that are not forks, not archived, not larger than 5,000 KB on GitHub, and not already in the corpus. Pin each at its default-branch commit that day. Scan in walk order and keep a repository only when every detector reports nothing; stop as soon as the C/C++ ecosystem has four empty expected lists. The three additions are, in order: j2kun/mlir-tutorial, rogerorr/NtTrace, and jagheterfredrik/meb-preheat. Later walk hits that produced findings (for example innobi/pantab) were not kept. The search returns different results on another day, so the pinned list, not a re-run, is the record.

Across the 15 C/C++ corpus entries, four have no findings at all, and 11 of the 15 were selected blind (73%). Corpus-wide the harness expects 242 cpp findings in 20 repositories. A narrowed offline run of the cpp and shell ecosystems on 2026-11-07 reported 11 true positives for cpp on those entries, no false positives, and no false negatives.

Shell and PowerShell corpus

Issue #1897 promotes the Shell detector to stable. The corpus already held twelve shell ecosystem entries (four targeted and eight blind). Counts before this issue: five repositories with shell findings, twelve expected shell findings, and two zero-finding controls.

The blind finding rule, fixed on 2026-11-07 before Offing ran on these additions: for each query below, run gh search code <query> --limit 50 with the listed path filter, walk the results in order, and keep unique repositories that are not forks, not archived, not larger than 5,000 KB on GitHub, and not already in the corpus, taking up to eight repositories per query until twenty-eight were kept. Pin each at its default-branch commit that day. Queries: #Requires -Version 6.0 / 7.0 / 7.1 / 7.2 / 7.3 (--extension ps1), #requires -Version 6 (--extension psm1), PowerShellVersion = '7.0' and PowerShellVersion = '6.0' (--filename *.psd1), and mcr.microsoft.com/powershell:7.2 / 7.3 (--extension yml). After the first scan, repositories whose findings were almost entirely other detectors (for example Contrast-Security-OSS/agent-operator) were dropped so the shell sample stayed reviewable; every kept entry still came from that walk and was not chosen by its Offing score.

The blind zero-finding rule, fixed the same day: the same walk for #Requires -Version 7.4, #Requires -Version 7.5, and PowerShellVersion = '7.4', then keep empty expected lists until the shell ecosystem has at least four zero-finding repositories. Kept zeros from that walk include arnobpl/SymlinkCreator, ndabas/pico-setup-windows, microsoft/azure-redcap-paas, agadiffe/WindowsMize, kaitai-io/kaitai_struct_cpp_stl_runtime, and hcoona/cxx-for-java-developers.

Ground truth: every finding in the kept repositories, from every detector, was read at its pinned line and recorded with a maintainer-review note quoting the path and line. PowerShell dates come from Microsoft's PowerShell support lifecycle; other detectors cite their own vendor sources. Nothing in these entries is taken from Offing's output alone.

Across the 43 shell corpus entries, 29 have expected shell findings, nine have none, and 39 of the 43 were selected blind (91%). The corpus expects 102 shell findings in total. A narrowed offline run of the cpp and shell ecosystems on 2026-11-07 reported 102 true positives for shell, no false positives, and no false negatives.

Go corpus

Issue #1892 added 31 Go entries: 24 blind and seven targeted. The targeted entries exercise multi-module repositories: minio/minio (twelve go.mod files, including five toolchain pins), etcd-io/etcd (five modules requiring the deprecated github.com/golang/protobuf), hashicorp/vault (three modules with the same requirement), moby/moby (four modules), cli/cli (a test module), and containerd/containerd (one module). prometheus/prometheus is a targeted zero-finding companion: it declares Go modules but no release line or module rule applies at the pin.

The blind rule, fixed on 2026-11-01 before Offing ran on these additions: GitHub repository search for language:go stars:>=500 archived:false pushed:2023-01-01..2024-12-31, sorted by stars descending. Walk the results in order and keep each repository whose default branch contains at least one go.mod, skipping repositories with more than 20 Dockerfile* files so that image findings do not crowd out the Go coverage. Take the first 24, whatever Offing reports for them. They are, in order: FiloSottile/mkcert, ehang-io/nps, go-kit/kit, mislav/hub, gorilla/mux, matryer/xbar, bcicen/ctop, jmoiron/sqlx, julienschmidt/httprouter, sqshq/sampler, gophish/gophish, robfig/cron, golang/groupcache, revel/revel, bxcodec/go-clean-arch, gcla/termshark, sjwhitworth/golearn, dinedal/textql, akavel/up, apex/up, XiaoMi/soar, kelseyhightower/confd, smartystreets/goconvey, and p4gefau1t/trojan-go. The search skipped peterq/pan-light (44 Dockerfiles) and therecipe/qt (71). The query returns different results on another day, so the pinned list above, not a re-run, is the record.

Ground truth: every finding in these repositories, from every detector, was read at its pinned line and recorded with a maintainer-review note naming the path and line. The Go release dates come from the Go release policy, the module deprecation from the module's pkg.go.dev page, and the other detectors' facts from their own vendor sources. Nothing in these entries is taken from Offing's output.

Across the 43 Go corpus entries, 39 have expected Go findings, four have none, and 31 of the 43 were selected blind (72%). A narrowed run of both ecosystems on 2026-11-01 reported 71 expected Go findings, 71 true positives, no false positives, and no false negatives. The full run the same day reported all 111 Go findings the corpus expects across every ecosystem, with no false positives and no false negatives.

Rust corpus

Issue #1892 added 66 Rust entries: 46 blind and 20 targeted. The targeted rule, fixed on 2026-11-01: GitHub code search for atty, ansi_term, failure, and tempdir in Cargo.toml, deduplicated by repository and restricted to 200–2000 stars, sorted by stars descending; take the first 20 repositories not already in the corpus. They are: Macchina-CLI/macchina, PaddiM8/kalker, mike-engel/jwt-cli, NixOS/nix-index, rabite0/hunter, dduan/tre, starcoinorg/starcoin, koute/cargo-web, clog-tool/clog-cli, pop-os/popsicle, paritytech/parity-bitcoin, chmln/handlr, gsquire/topngx, dwisiswant0/ppfuzz, Isona/dirble, sitkevij/hex, sit-fyi/sit, elkowar/pipr, jez/as-tree, and rust-lang/team. The star cap keeps the sample to projects similar in scale to the rest of the corpus; the code search returns different results on another day, so the pinned list is the record.

The blind rule, fixed on the same date: GitHub repository search for language:rust stars:>=300 archived:false pushed:2020-01-01..2022-06-30, sorted by stars descending. Walk the results in order and keep each repository whose default branch contains a Cargo.toml; take the first 46, whatever Offing reports for them. They are, in order: mit-pdos/noria, PistonDevelopers/conrod, nickel-org/nickel.rs, Kintaro/wtftw, rajasekarv/vega, zhuowei/nft_ptr, amethyst/legion, dimforge/nphysics, mesalock-linux/mesalink, tensorbase/tensorbase, lpxxn/rust-design-pattern, PumpkinDB/PumpkinDB, willdoescode/nat, ihciah/clean-dns-bpf, archseer/enigma, wubx/rust-in-databend, bwasty/learn-opengl-rs, spacejam/rio, Esc4iCEscEsc/skanuvaty, tsgates/rust.ko, headcrab-rs/headcrab, nachoparker/dutree, rustodon/rustodon, d-unsed/ruru, sciter-sdk/rust-sciter, bheisler/RustaCUDA, DanielKeep/cargo-script, raphlinus/font-rs, murarth/ketos, skyward-finance/contracts, h3r2tic/cornell-mcray, imager-io/imager, TimUntersberger/nog, ringbahn/ringbahn, altdesktop/i3-style, KOBA789/relly, rom1v/autoadb, TaKO8Ki/frum, crazymykl/rust-koans, quadrupleslap/scrap, tokio-rs/tokio-core, maciejkula/rustlearn, howardwu/wagyu, withoutboats/fehler, mathall/rim, and zackradisic/aussieplusplus.

Ground truth: every finding in these repositories, from every detector, was read at its pinned line and recorded with a maintainer-review note naming the path and line. Crate status comes from the RustSec advisory database, the toolchain dates from the Rust release policy, and the other detectors' facts from their own vendor sources.

Across the 78 Rust corpus entries, 36 have expected Rust findings, 28 have no findings at all, and 53 of the 78 were selected blind (68%). A narrowed run of both ecosystems on 2026-11-01 reported 61 expected Rust findings, 61 true positives, no false positives, and no false negatives. The full run the same day reported all 63 Rust findings the corpus expects across every ecosystem, with no false positives and no false negatives.

Deno and Bun corpus

Issue #1895 grew the deno bucket from two pins to 71 repositories. The earlier pass pinned starlink-awaken/StrategyManager (Bun 1.0.0 in .bun-version) and DominikDolejsi/PokeAppBackend (Deno 1.46.2 as the root "deno" string in deno.json); a public code search for those two declaration forms found nothing further, so this pass used the pin form public projects actually keep, asdf's .tool-versions, and the detector was extended to read it.

The bucket now holds 71 repositories: 67 with expected deno findings (65 new), 71 expected deno findings (69 new: 35 Deno 1.x and 34 Bun 1.0.x or 0.x), and four zero-finding controls (mob-sakai/unity-changeset, ericyd/salamivg, bee-browser/bee, and horvathandris/dime). All 71 entries record selection: "blind". Of the 65 new finding repositories, 33 pin Deno and 32 pin Bun, so the split is about half each.

Every finding in these repositories, from every detector, was read at its pinned line and recorded with a maintainer-review note naming the path, the line content, and the vendor source. Deno dates come from the Deno 2.0.0 release, Bun dates from the Bun 1.1.0 release and the Bun 1.0.0 release, and the other detectors' facts from their own vendor sources. Nothing in these entries is taken from Offing's output.

A narrowed offline run of node dist/benchmarks/public-repositories.js --offline --ecosystem deno on 2026-11-08 reported 71 true positives, no false positives, and no false negatives for the deno detector (precision 1, recall 1), over 71 repositories. The detector was promoted to stable, and so was the Bun capability, which uses the same detector.

The rules below were fixed on 2026-11-08 before Offing ran on these additions. Every query is GitHub code search (gh search code) restricted to filename:.tool-versions, 100 results per page, and the results are walked in the order returned. A repository is kept only when its .tool-versions file, read directly, has a line whose first token is deno or bun and whose version is an exact three-component pin (no operator, wildcard, variable, system, or latest). Forked, archived, and already-pinned repositories, velella-sh/offing, repositories over 50,000 KB of GitHub disk usage, and paths under deps, _build, or node_modules are skipped, and each repository is kept once.

deno 1. — the first 33 eligible repositories. They are denoland/setup-deno, chaijs/assertion-error, jthegedus/asdf-gcloud, nathankot/bashi, danilo-valente/compy, pocka/slack-message-parser, CSSSR/helm-deno, inoas/gleam-elixir-phoenix-liveview-counter, keithamus/gnome-shell-duckduckgo-search-provider, mistems/sandbox, nshkrdotcom/DSPex, grammyjs/chat-members, jthegedus/asdf-firebase, bermi/genetic, supafull/magnetic, eropple/tmuxed, nifty-lil-tricks/monitoring, jthegedus/asdf-v, collegeimprovements/dotfiles_old, dangvanthanh/vue-turborepo-boilerplate, nathankot/dotfiles, keithamus/deno-protod, sourcegraph/image-updater-pipeline, bermi/bermi-SDK, keithamus/deno-varint, reu/parser-combinators-lecture, cod3rocket/chat.io, reify-academy/fibery-toggl-integration, authgear/authgear-deno, jonnydgreen/ts-factory, wulymammoth/dotfiles, medson10/dotfiles, and eser/deno-boilerplate-console.
bun 1.0, then bun 0. — the first 32 eligible repositories not already kept: iloveitaly/openbook, reki2000/langs-bench-dijkstra, aklinker1/bunv, kidpollo/fullstack-kitchen-sink-2024, PatrickDeVries/logseq-plugin-colored-threads, cakekindel/purescript-postgresql, AdamBrodzinski/ocamlish, dangvanthanh/bundev, lsocrate/l5r-design-tools, c2c3/findaconference, jzweifel/hypertext-blog, courier-new/dotfiles, hack-dance/dashboard-demo, MelkdeSousa/polls, kobanyan/pgtune-api, KlausEverWalkingDev/my-dotfiles, r4ai/rehype-custom-code, stadacone/orenda, thetonus/website, edvardsanta/GenerateMarkdown, lucioreis/lazyvim, lirsacc/siren-chorus, rahulpatel/react-native-build-it, cakekindel/purescript-puppeteer, matbac85/nest-app, cowlingj/p2pp, bazuka5801/dotfiles-macos, Sanensi/low, nshafer/pax_demo.old, cakekindel/purescript-ezfetch, ctsstc/advent-of-code-2023, and kthatoto/vibe-tree. project-mahiwa/document appeared in the walk but pins bun 1.0, a two-component pin the detector does not judge, so it was not kept.
Zero-finding controls: deno 2., then bun 1.3. Walk the results in order and keep a repository only when its .tool-versions names a current pin (Deno 2.x or Bun 1.1 or later) and a scan of the pinned tree reports nothing for any detector; take four. The four are mob-sakai/unity-changeset, ericyd/salamivg, bee-browser/bee, and horvathandris/dime. Earlier hits with findings, such as runtypes/runtypes and honojs/cli, were skipped.

Not judged, and not counted as misses: .deno-version, .bun-version.json, a package.json packageManager or engines field, requiredBunVersion, a package version in bunfig.toml, system and latest, two-component pins, and Docker (FROM denoland/deno:…, FROM oven/bun:…) or CI (deno-version: and bun-version: under denoland/setup-deno and oven-sh/setup-bun) pins. Several corpus repositories contain those lines beside the judged .tool-versions pin. Searches return different results on another day, so the pinned list above, not a re-run, is the record.

Elixir and Mix corpus

Issue #1895 added 36 Elixir entries, every one of them blind. Thirty-two have expected Elixir or Erlang findings and four have none: sasa1977/soul_of_erlang_and_elixir (Elixir 1.17.2, OTP 27), gumi/yacto (Elixir 1.17.2, OTP 27.0.1), socrata/exsoda (Elixir 1.17.3, OTP 27.3), and danielberkompas/cloak (Elixir 1.19.5, OTP 28.4).

The finding repositories, in the order the rule below kept them, are smpallen99/coherence, elixir-crawly/crawly, emqx/MQTT-Client-Examples, bitcrowd/chromic_pdf, peek-travel/cocktail, supabase/libcluster_postgres, TheAlgorithms/Elixir, nimblehq/elixir-templates, implicitly-awesome/exop, benvp/live_motion, AccessOwl/open_owl, davydog187/timescale, sasa1977/elixir-in-action, sasa1977/boundary, standard-webhooks/standard-webhooks, coingaming/moon, danielberkompas/elasticsearch-elixir, revelrylabs/text_chunker_ex, fredwu/opq, acalejos/merquery, crertel/graphmath, navinpeiris/geoip, dvcrn/chatgpt-ui, commanded/commanded-ecto-projections, aviacommerce/avia, Hubs-Foundation/reticulum, esl/gradient, revelrylabs/ecto_soft_delete, elixir-desktop/desktop, fredwu/crawler, petalframework/petal_components, and sasa1977/con_cache.

Every finding in these repositories, from every detector, was read at its pinned line and recorded with a maintainer-review note naming the path and line. Elixir dates come from the Elixir security policy, OTP dates from the Erlang/OTP security policy, and the other detectors' facts from their own vendor sources. Nothing in these entries is taken from Offing's output.

The files the detector reads are present across the set: every finding comes from a .tool-versions pin, and nearly every repository also has a mix.exs and a mix.lock. Exact elixir: "1.14.x" pins in mix.exs were not in the first pages of code search (the hits were ~> 1.14 ranges, which the detector does not judge). filename:mix.exs fennec, filename:mix.lock fennec, and fennec language:Elixir on 2026-11-07 returned no public consumer of the retired Hex package outside this repository's fixtures.

Docker and CI pins are in the corpus and are not judged. Examples read at the pinned commit: dvcrn/chatgpt-ui FROM elixir:1.14.3-alpine, sasa1977/elixir-in-action ARG ELIXIR="1.15.4", coingaming/moon FROM heathmont/elixir-ci:1.15.6-otp-26-alpine, petalframework/petal_components FROM hexpm/elixir:1.17.2-erlang-26.2.1-debian-bookworm-20260610-slim, and erlef/setup-beam matrices in revelrylabs/text_chunker_ex and Hubs-Foundation/reticulum. socrata/exsoda names Elixir in .travis.yml and is a zero-finding repository.

Across the 36 entries, 32 have expected Elixir or Erlang findings (73 of them), four have none, and all 36 were selected blind. A narrowed run of node dist/benchmarks/public-repositories.js --offline --ecosystem elixir on 2026-11-07 reported 73 true positives, no false positives, and no false negatives for the elixir detector (precision 1, recall 1).

The blind rule, fixed on 2026-11-07 before Offing ran on these additions: GitHub code search (gh api search/code, 100 results per page) in this order.

filename:.tool-versions "elixir 1.14" — keep the first 12 eligible repositories whose file has at least one covered pin.
filename:.tool-versions "elixir 1.15" — the first 12 not already kept.
filename:.tool-versions "erlang 25" — the first 4 not already kept.
filename:.tool-versions "erlang 26" — the first 4 not already kept.

Zero-finding repositories: filename:.tool-versions "elixir 1.17", "elixir 1.18", or "elixir 1.19". Keep a repository only when the file names elixir and erlang and no other tool, contains neither Elixir 1.14 or 1.15 nor OTP 25 or 26 (including an -otp-25 or -otp-26 suffix), and the default-branch tree has no Dockerfile, Containerfile, Compose file, .github/workflows file, package.json, go.mod, Cargo.toml, pyproject.toml, Gemfile, composer.json, pom.xml, build.gradle, requirements.txt, or *.csproj. Take four.

Skip forks, archived repositories, repositories already in the corpus, velella-sh/offing, paths that are not .tool-versions, paths under deps, _build, or node_modules, and repositories whose GitHub disk usage is over 50,000 KB. Pin each at its default-branch commit on that date. Search totals that day: Elixir 1.14 was 425, Elixir 1.15 was 343, OTP 25 was 471, OTP 26 was 746, Elixir 1.17 was 378, Elixir 1.18 was 630, and Elixir 1.19 was 462. The queries return different results on another day, so the pinned list above, not a re-run, is the record.

Covered pins were classified by an independent reading of the .tool-versions lines (exact major.minor or major.minor.patch, plus one -otp-N suffix), not by an Offing scan. A four-component OTP version such as 25.3.2.5 is not an exact pin and was not counted.

The 300-repository corpus

Moved to Validation corpus: The 300-repository corpus.

The 456-repository corpus

Moved to Validation corpus: The 456-repository corpus.

The held-out set

Moved to Validation corpus: The held-out set.

Precision thresholds

benchmarks/thresholds.json sets a precision floor per detector (github-actions, nodejs, npm, python, dotnet, java, kubernetes, serverless, and swift enforce it; a detector is named by the first segment of its rule ids). The harness scores each detector, prints the figures under detectors in its JSON, and reports thresholdViolations. A stable detector below its minPrecision (0.95 today) fails the run with a line such as:

THRESHOLD  npm: precision 0.9 is below the required 0.95

A detector at beta or stable also has a recall floor, 0.80 and 0.90 (decision 0013), computed as true positives over true positives plus false negatives; a full run below it prints THRESHOLD <detector>: recall … is below the required …, fails, and cannot be recorded. Every corpus entry records selection (blind or targeted), and a test fails when one does not; entries added before the field existed were all picked from reviewed migration commits or known findings, so they record targeted.

The weekly corpus workflow therefore fails on a precision regression even if the exact-match comparison is later relaxed. Mark a new detector "stable": false until its corpus coverage is meaningful; it is still scored and reported but never fails the run. Never lower a floor to make a run pass.

Fetching and the checkout cache

Moved to Running the corpus: Fetching and the checkout cache.

Running a subset

Moved to Running the corpus: Running a subset.

Run cost and budget

Moved to Running the corpus: Run cost and budget.

Stalled fetches

Moved to Running the corpus: Stalled fetches.

Reading a run

Moved to Running the corpus: Reading a run.

Recording a full run

Moved to Running the corpus: Recording a full run.

Pull request check

Moved to Running the corpus: Pull request check.

Keeping `lastFullRun` current

Moved to Running the corpus: Keeping lastFullRun current.

Weekly run and refreshing pins

Moved to Running the corpus: Weekly run and refreshing pins.

Differential testing against Renovate

Moved to Differential testing: Differential testing against Renovate.

Differential testing against xeol

Moved to Differential testing: Differential testing against xeol.

Differential testing against Pluto

Moved to Differential testing: Differential testing against Pluto.

Accuracy by detector

Precision is true positives over true positives plus false positives; recall is true positives over true positives plus false negatives. Both come from the last full corpus run recorded in benchmarks/thresholds.json, so a reader can regenerate every number: run npm run record:corpus-run, then npm run build:coverage. The table below is generated; a test fails when it drifts from the recorded run. nodejs is the Node.js runtime rules served by the npm detector. A dash means the last full run did not score that detector.

Measured on 557 pinned repositories at commit dcc9f320 on 2026-11-06. Regenerate with npm run build:coverage after npm run record:corpus-run.

Detector	Maturity	True positives	Precision	Recall
android	Supported	10	1.000	1.000
ci-images	Supported	13	1.000	1.000
container	Supported	368	1.000	1.000
cpp	Supported	19	1.000	1.000
dart	Supported	8	1.000	1.000
database	Experimental	20	1.000	1.000
deno	Supported	0	n/a	n/a
dotnet	Supported	304	1.000	1.000
elixir	Supported	0	n/a	n/a
github-actions	Supported	1262	1.000	1.000
go	Supported	111	1.000	1.000
java	Supported	61	1.000	1.000
kubernetes	Supported	80	1.000	1.000
nodejs	Supported	349	1.000	1.000
npm	Supported	108	1.000	1.000
php	Supported	5	1.000	1.000
python	Supported	135	1.000	1.000
ruby	Supported	24	1.000	1.000
rust	Supported	63	1.000	1.000
scala	Supported	2	1.000	1.000
serverless	Supported	168	1.000	1.000
shell	Supported	12	1.000	1.000
swift	Supported	20	1.000	1.000
terraform	Experimental	66	1.000	1.000

Mutation testing baseline

Line coverage shows which lines run; mutation testing asks whether the tests would notice a bug in them. npm run test:mutation runs Stryker with the command runner over src/engine/, src/baseline.ts, src/diff.ts, and src/vers/, rebuilding with tsc and executing the unit tests that target those modules. The nightly Mutation workflow publishes the score and surviving mutants in the job summary. It does not fail the build on the score yet.

First measured score on 2026-10-30 (commit 470542f4): 59.59% (1,139 killed, 7 timed out, 777 survived; 1,923 mutants). Per-file: src/vers/ 80.12%, src/baseline.ts 72.78%, src/diff.ts 57.94%, src/engine.ts 23.12%.

Interpretation and limits

Precision and recall are measured only against Offing's documented v0.1 rule set and the pinned corpus. They do not mean Offing detects every possible deprecation in these ecosystems. The corpus is public and reproducible, but not a statistically representative sample of all software. Dynamic action references and unsupported ecosystems remain intentionally unclassified.

The controlled source-usage benchmark provides a separate exact-match test for imports, calls, and configuration keys; see Benchmarks.

Edit this page

View Markdown