The Emscripten Oracle

The Oracle is the stage that typically moves the biggest block of coverage in a single step. Emscripten, musl, dlmalloc/emmalloc, libc++, and the whole WebAssembly runtime are open source. WARDEN exploits that: it compiles the exact runtime a target was built with, across the build-flag matrix that affects codegen, retains the name section, fingerprints every labeled function into a portable store, and then runs that store against the stripped target. Functions that match at or above the acceptance threshold get the real upstream name attached immediately, with provenance oracle.

Alpha status. The matching engine, signature store, MinHash-LSH index, and the matrix corpus farm (the build_matrix.sh matrix builder, its manifest.json, and the harvest_directory harvest step) are all implemented and working, along with the robustness work covered below: deterministic identify tie-breaking, calibrated version-inference confidence, SignatureStore.stats() introspection, and the evaluate_identification precision/recall harness. The seed store that ships with a fresh clone is intentionally empty. You build your own. The large multi-thousand-signature corpus that would let the Oracle identify most of a real target still needs the full emsdk matrix run (Docker plus emsdk); the precision/recall numbers you can produce today are calibration on the labeled modules you have, not a published accuracy figure. A shared community corpus is on the roadmap.

Compile your own ground truth

No other FLIRT- or BinDiff-style workflow auto-builds its corpus from the toolchain’s own source across the version/flag matrix. The critical ingredient is --profiling-funcs. Without it, an optimized Emscripten build strips the wasm name section. The corpus builder, extract_signatures, looks up each function’s name from:

module.names.function_names: the wasm name section (primary source).
func.export_names[0]: used if the function is exported and the name section is absent.

If neither yields a name, the function is silently skipped. Retaining names is not optional: it is the entire source of ground truth. Use --profiling-funcs (or -g / -gsource-map) on every corpus build. This flag forces Emscripten to emit the name section even at -O2 and -Oz where it would otherwise be stripped.

The signature store

`Signature`

Defined in src/warden/oracle/signatures.py. One Signature is a labeled fingerprint of a single function extracted from a corpus build.

Field	Type	Meaning
`name`	`str`	The real upstream name, from the wasm name section or export.
`library`	`str`	Which library: `musl`, `musl-pthread`, `libc++`, `libc++abi`, `compiler-rt`, `dlmalloc`, `emscripten`, `wasi-libc`, or the `--library` default.
`emscripten_version`	`str \| None`	The emsdk version these were built with.
`opt_level`	`str \| None`	The `-O` level, e.g. `-O2`.
`source_ref`	`str \| None`	Optional upstream source link.
`structural_hash`	`str`	Blake2b of the normalized control-flow/opcode skeleton.
`exact_hash`	`str`	SHA-256 of the raw function body.
`minhash`	`list[int]`	32-element MinHash over instruction 4-grams (fuzzy matching).
`histogram`	`dict[str, int]`	Opcode-class frequency histogram.
`call_targets`	`list[str]`	Import call-neighborhood (`module.field` strings).
`type_signature`	`str`	WASM type signature string.
`instruction_count`	`int`	Total instruction count.
`local_calls`	`int`	Count of calls to locally-defined functions.

Signature.fingerprint() reconstructs a FunctionFingerprint so the same similarity() function used everywhere in WARDEN can compare a corpus entry against a live target function. Library classification is automatic, based on name prefixes. classify_library walks an ordered prefix table and returns the first library whose prefix matches the name. Anything that matches no prefix keeps the fallback --library value (default musl). The full table:

Prefix	Assigned library
`emscripten_`, `__em_`	`emscripten`
`pthread_`, `__pthread`	`musl-pthread`
`_ZNSt`, `_ZSt`, `_ZN`	`libc++`
`__cxa_`	`libc++abi`
`__muldi3`, `__divdi3`, `__udivdi3`, `__fixunsdfdi`, `__floatunsidf`	`compiler-rt`
`dlmalloc`, `dlfree`	`dlmalloc`
`__wasi`, `wasi_`	`wasi-libc`
(no match)	whatever `--library` you passed

The compiler-rt row covers the soft-float and wide-integer builtins Emscripten links from compiler-rt. These show up in almost every non-trivial module and used to land in the musl fallback bucket, which made the per-library breakdown misleading.

`SignatureStore`

A SignatureStore is an ordered list of Signature objects with JSON persistence. The on-disk format is plain JSON (portable, git-diffable, and shareable without a database).

store = SignatureStore.load("oracle.json")  # returns empty store if file is missing
store.add(sig)
store.extend(sigs)  # bulk add, deduplicated (see below)
store.save("oracle.json")
len(store)          # number of signatures
store.libraries()   # dict[library_name, count]
store.stats()       # full introspection summary (see below)

Introspecting a store with `stats()`

store.stats() returns one deterministic summary dict for a store, so you can audit what a corpus actually covers without loading the JSON by hand. It rolls up the counts that matter when you are deciding whether a corpus is broad enough to identify a given target:

Key	Type	Meaning
`total`	`int`	Total number of signatures (same as `len(store)`).
`by_library`	`dict[str, int]`	Per-library signature count (same as `libraries()`).
`by_emscripten_version`	`dict[str, int]`	Per-version signature count; a signature with no version is counted under `"?"`.
`distinct_names`	`int`	How many distinct upstream names are covered.

Every dict is ordered for stable output: keys are sorted so the same store always prints the same summary, which keeps it safe to snapshot in a test or diff across runs, and the result does not depend on the order signatures were added. A store whose by_emscripten_version covers only one version will identify fewer functions in a target built differently. stats() makes that gap visible before you run identify and wonder why coverage is low.

Signature dedup on `extend`

The matrix farm compiles the same reference programs across many versions and opt levels, so the same function body recurs constantly. extend deduplicates as it adds: a new signature is dropped when an existing entry already shares the same identity key. The key is the tuple of (name, library, emscripten_version, opt_level, exact_hash), so two builds of memcpy that produced a byte-identical body at the same version and -O level collapse to one entry, while a genuinely different -Oz body is kept. This keeps oracle.json from ballooning when you re-run the farm or harvest the same directory twice, and it makes the build idempotent: harvesting the same tree again does not grow the store.

Dedup happens on extend only, the path every corpus and harvest build takes. add is the raw single-signature append and does not deduplicate, so a manual add loop can still create duplicates if you bypass extend.

{
  "version": 1,
  "count": 214,
  "signatures": [ { "name": "memcpy", "library": "musl", ... }, ... ]
}

The shipped seed store is empty

src/warden/oracle/seed_signatures.json ships with "count": 0 and an empty signatures array. This is deliberate: a pre-built corpus would encode assumptions about which emsdk versions and flag combinations matter for your target. You build the store that matches your target and commit it alongside your project.

Identifying functions: `identify` and `ORACLE_THRESHOLD`

identify is defined in src/warden/oracle/match.py. Given a KnowledgeBase, a version_id, and a SignatureStore, it:

Fetch all defined functions for the version

Imports are excluded. They already have names from the JS glue.

Reconstruct a fingerprint from the KB row

Uses fingerprint_from_record. No re-parsing of the original .wasm is required.

Score every corpus signature against the target fingerprint

The similarity() function combines four signals:

Signal	Weight	Method
Exact body equality	shortcircuit → 1.0	SHA-256 match
Fuzzy instruction similarity	0.45	MinHash Jaccard over 4-grams
Opcode-class histogram	0.25	cosine similarity
Call-neighborhood overlap	0.20	import Jaccard
Structural skeleton match	+0.10	hash equality bonus

Accept or reject

The best-scoring signature wins. If score >= threshold, it is an Oracle match.

Deterministic tie-breaking

Two signatures can score identically against one target function. The matrix farm makes this common: the same memcpy body compiled at the same -O level under two different emscripten versions produces two signatures that are byte-identical apart from their version label, so both score 1.0. Picking whichever happened to come first in the store would make the result depend on insertion order, and the same target could be labeled 3.1.50 on one run and 3.1.55 on the next. To keep identify deterministic, ties are broken by a fixed, total ordering rather than by store position. Among signatures that tie on score, the Oracle prefers, in order:

the higher score (the primary key, already decided above),
then the lexicographically smaller name,
then the lexicographically smaller source_ref (a missing source_ref is treated as the empty string, so it sorts first).

Because every field in that chain is part of the signature itself and the comparison is total, identical inputs always yield the same OracleMatch regardless of the order signatures were added to the store. identify_indexed applies the same tie-break over its candidate pool, so the indexed path returns the same match as the linear scan. The acceptance threshold is:

ORACLE_THRESHOLD = 0.82   # src/warden/oracle/match.py

When write=True (the default), every accepted match is written back to the KB with:

Provenance "oracle" and confidence = score.
A Symbol entry with the matched name, type signature, and a summary noting library and version.
An evidence trail: {"kind": "oracle", "detail": "<library> <emver> @<opt> score=<N>"}.
A row in oracle_matches keyed to the internal function ID.

Human-set names (provenance = "human", locked = True) are sovereign. The Oracle cannot overwrite them, no matter the score.

The --threshold flag overrides ORACLE_THRESHOLD at the CLI. Lower values catch more functions at the cost of false positives; higher values are conservative. The default of 0.82 is a reasonable starting point for functions of moderate size, but you should audit matches after the first run.

The `OracleMatch` return type

identify returns list[OracleMatch]:

@dataclass
class OracleMatch:
    func_index: int
    stable_id: str
    matched_name: str
    library: str
    emscripten_version: str | None
    opt_level: str | None
    score: float
    source_ref: str | None

Scaling: the MinHash-LSH index

The default identify scan is O(targets × signatures). For a corpus of a few hundred signatures that is negligible, but once the matrix farm has accumulated thousands of entries across many versions and opt levels the linear pass becomes a bottleneck. warden.oracle.index provides SignatureIndex, a banded MinHash + structural-hash index that reduces each function’s search space to a small set of candidates before any scoring happens.

How it works

SignatureIndex.build(store, *, bands=None) splits each signature’s 32-element MinHash into bands equal slices and hashes each slice into a bucket. Two signatures that agree on any whole band land in the same bucket. The probability of that collision grows with their true Jaccard similarity, so near-matches across -O levels still surface. Signatures are also indexed by their exact structural_hash, so structurally identical functions are always found regardless of their MinHash values.

Automatic band selection

bands controls the recall/precision tradeoff of the index. Few wide bands cast a wide net (high recall, larger candidate sets); many narrow bands are selective (smaller candidate sets, slightly lower recall). The right number depends on how many signatures are in the store, so you do not have to guess: when you pass bands=None (the default), build calls recommended_bands(len(store)) to pick a value scaled to the store size.

Store size	`recommended_bands`
empty or tiny (`< 64`)	`4`
small (`64` to `999`)	`8`
medium (`1_000` to `9_999`)	`16`
large (`>= 10_000`)	`32`

The function returns a divisor of 32 (the MinHash width) so the bands partition the signature evenly with no remainder. As the matrix farm grows the corpus from a few hundred to tens of thousands of signatures, the index narrows its bands automatically and the candidate sets stay small. Pass an explicit integer to bands to override the heuristic. index.candidates(fp) returns the deduplicated union of every signature sharing a band-bucket or the structural hash with fp. This is a pure dictionary lookup on the hot path. identify_indexed is the LSH-accelerated mirror of identify. When the index yields no candidates for a function it falls back to the full store, so no function is ever silently dropped. The result is identical to the linear scan at ORACLE_THRESHOLD = 0.82.

from warden.oracle.index import SignatureIndex, identify_indexed
from warden.oracle import SignatureStore
from warden.kb import KnowledgeBase

store = SignatureStore.load("oracle.json")
index = SignatureIndex.build(store)   # bands=None -> recommended_bands(len(store))

# Query candidates for one fingerprint
candidates = index.candidates(fp)   # list[Signature], usually << len(store)

# Full pass: same matches as identify(), faster on large stores
with KnowledgeBase("project.db") as kb:
    v = kb.get_version("v1")
    matches = identify_indexed(kb, v.id, store, threshold=0.82, write=True)

Leave bands at its None default and let recommended_bands scale it to the store. Only pass an explicit value when you are tuning: raise it to narrow candidates further, lower it to catch more edge cases at the cost of a larger candidate set.

Version inference

After identify returns, call infer_version to infer which Emscripten version the target was built with:

def infer_version(matches: list[OracleMatch]) -> VersionInference:
    ...

@dataclass
class VersionInference:
    emscripten_version: str | None   # e.g. "3.1.55"
    confidence: float                # calibrated, see below
    histogram: dict[str, int]        # full vote distribution

The implementation tallies emscripten_version across all matches that carry one. The version with the highest vote count (the plurality) wins. Ties on vote count are broken by the smaller version string, so the winner is deterministic when two versions draw. Matches with no emscripten_version are ignored, and the histogram reports only the version-bearing votes.

Calibrated confidence

Raw vote share alone overstates certainty. Two matches that both vote 3.1.55 give a vote share of 1.0, but two matches is thin evidence, and a target whose runtime spans several versions can still produce a clear plurality winner that does not deserve full confidence. The reported confidence is therefore the plurality share calibrated down by sample size:

Plurality share. The base figure is the winner’s votes over all version-bearing votes. A clean sweep approaches 1.0; a split field scores proportionally lower (a 2-2 tie gives the winner a share of 0.5).
Sample size damping. A handful of matches cannot pin a version as firmly as hundreds can, so the share is multiplied by total / (total + 1) over the version-bearing match count. That factor is 0.5 for a single vote and rises toward 1.0 as evidence accumulates, so a lone match is deliberately modest even though its raw share is 1.0.

The result stays in [0.0, 1.0] and is fully deterministic: identical matches give an identical confidence, with no clock or random input. When 75% of a few hundred Oracle matches are tagged 3.1.55, the calibrated confidence is high; when only two matches agree, it is held down by the damping factor. This version pin matters downstream:

It is stored on the ModuleVersion row and shown by warden versions.
The diff engine uses it to classify toolchain churn: if v1 infers 3.1.50 and v2 infers 3.1.61, functions that changed can be re-examined against the Oracle and reported as “changed due to Emscripten upgrade” rather than “application change”.
It sharpens dynCall/elem-table convention assumptions for any subsequent JS glue analysis.

The accuracy of the inference is bounded by corpus coverage. A wider flag matrix and more reference programs produce more matches and a more confident vote.

Measuring identification quality: `evaluate_identification`

A threshold of 0.82 is a starting point, not a proof. Before you trust the Oracle on a real target you want numbers: how many of its matches are correct, and how many real runtime functions does it miss. evaluate_identification is a precision/recall harness for exactly that, and it runs against a labeled module without needing emsdk or Docker on the machine. The idea is to evaluate against ground truth you already have. A corpus build is itself a labeled module: every function in it carries its real name from the wasm name section. So you can build a store from one set of labeled modules, run identify against a held-out labeled module whose true names are known, and compare the Oracle’s answers to those names. Because the held-out module still has its name section, the harness knows the correct answer for every function and can score the run.

from warden.oracle import evaluate_identification, SignatureStore
from warden.kb import KnowledgeBase

store = SignatureStore.load("oracle.json")
with KnowledgeBase("heldout.db") as kb:
    v = kb.get_version("v1")           # an ingested module that still has its true names
    report = evaluate_identification(kb, v.id, store, threshold=0.82)

print(report["precision"], report["recall"])

It returns a JSON-friendly dict. For each labeled function (one whose raw_name survived ingest from the wasm name section or an export) the harness compares the Oracle’s predicted name against that ground truth and bins the outcome. Functions with no raw_name carry no label, so they contribute nothing to either metric.

Key	Definition
`checked`	Labeled functions in the version (those with a `raw_name` ground truth).
`true_positive`	Matches whose predicted name equals the ground truth.
`false_positive`	Matches that landed on a labeled function but disagreed with the truth.
`precision`	`tp / (tp + fp)`: of the matches checked against a label, the fraction correct. Falls back to `0.0` when nothing matched.
`recall`	`tp / checked`: of the labeled functions, the fraction the Oracle named correctly. Falls back to `0.0` when nothing was labeled.

A match on an unlabeled function is ignored: there is no ground truth to confirm or deny it, so it never inflates the false-positive count. The whole evaluation is read-only and deterministic: it never writes back to the KB (it scores the matches identify would produce), and the same store, target, and threshold always yield the same report. That makes it a clean way to sweep the threshold. Run it at several values, watch precision rise and recall fall as you raise the bar, and pick the operating point that fits your tolerance for false positives.

This harness measures quality on a held-out labeled module, which you can produce on any machine. It does not replace a broad corpus. A genuinely confident precision/recall number, and a corpus large enough to identify most of a real target, still needs the full multi-thousand-signature emsdk matrix run described below, which requires Docker and emsdk. Until that run exists, treat these numbers as a calibration tool on the modules you have, not as a published accuracy figure.

CLI usage

Step 1: Build a signature store

warden oracle build <wasm> [<wasm> ...] \
    --out oracle.json \
    --emver 3.1.55 \
    --opt -O2 \
    [--library musl]

<wasm>: one or more .wasm files with a name section (--profiling-funcs at compile time).
--out / -o: the output (or existing) oracle.json. If the file already exists, new signatures are appended; the CLI reports +N new.
--emver: the Emscripten version string to tag these signatures with.
--opt: the optimization level (e.g. -O2, -Oz).
--library: fallback label for names that do not match any known prefix (default musl).

After building, the CLI prints the total signature count and the per-library breakdown:

Wrote 214 signatures (+214) to oracle.json
Libraries: dlmalloc=8, emscripten=12, libc++=34, musl=160

Inspect a store before you trust it

warden oracle inspect prints what a store covers, so you can audit a corpus without opening the JSON. It is the CLI surface over SignatureStore.stats():

warden oracle inspect oracle.json

Store: oracle.json
Signatures:     214
Distinct names: 187
Libraries:      dlmalloc=8, emscripten=12, libc++=34, musl=160
Versions:       3.1.55=214

The Versions line maps to by_emscripten_version and is the one to read closely. A store that covers only one version (as above) will identify fewer functions in a target built with a different toolchain. A broad corpus from the matrix farm shows several versions here. Every line is sorted, so the same store always prints the same summary.

warden oracle inspect <store> is the CLI surface over SignatureStore.stats(). It is being wired into the CLI separately; stats() is the stable library entry point in the meantime. Both read the same oracle.json and report the same counts.

Step 2: Identify runtime functions in a target

warden oracle identify v1 \
    --store oracle.json \
    [--threshold 0.82] \
    [--indexed] \
    [--db warden.db]

v1: a version label already ingested with warden ingest.
--store / -s: path to the oracle.json built above (default oracle.json).
--threshold: override ORACLE_THRESHOLD.
--indexed: use the MinHash-LSH index (identify_indexed) instead of the linear scan. Produces identical matches but is significantly faster once the store grows large. The index is built in memory at the start of each run. No separate build step is required.

The CLI prints a match table and, when infer_version succeeds, the toolchain line:

 idx  name              library   score  emver
 ───  ────────────────  ────────  ─────  ──────
  12  memcpy            musl      1.00   3.1.55
  14  memset            musl      1.00   3.1.55
  19  __cxa_throw       libc++abi 0.91   3.1.55
  ...

Inferred Emscripten version: 3.1.55 (confidence 0.91)

The confidence on that line is the calibrated value described under version inference, not the raw vote share, so it stays modest when only a handful of matches agree. All matches are written into the KB immediately. The match table itself is deterministic: ties on score are broken by name then source_ref, so repeated runs print the rows in the same order with the same labels.

Step 3: Check coverage

warden coverage v1

defined functions   184
named               112  (60%)
  by oracle          98
  by human            0
  by agent           14

The emsdk matrix corpus farm

Building a corpus by hand for each version and flag combination is tedious. The scripts in scripts/corpus/ provide a containerized, reproducible alternative. The only host dependencies are Docker and the warden CLI. The farm runs in two stages. First build_matrix.sh compiles the reference programs across the emscripten-version times opt-level matrix and writes a directory of labeled .wasm files plus a manifest.json that records how each file was built. Then a single harvest pass turns that whole directory into one oracle.json. Splitting compile from harvest means the expensive Docker builds happen once and you can re-harvest (after a classifier change, say) without recompiling.

cd scripts/corpus

# Stage 1: compile the matrix into a labeled artifact directory + manifest.
./build_matrix.sh \
  --versions "3.1.50 3.1.55 3.1.61" \
  --opt "-O0 -O2 -Oz" \
  --out ./corpus-build

# Stage 2: harvest the whole directory into one signature store.
warden oracle harvest ./corpus-build --out ../../oracle.json

Stage 1: `build_matrix.sh`

For each (emscripten_version, opt_level) pair:

Pull the official emsdk image

emscripten/emsdk:<version> is pulled from Docker Hub. Emscripten never touches your host.

Compile reference programs inside the container

Every .c file under scripts/corpus/reference/ is compiled with emcc, passing --profiling-funcs and -sEXPORT_ALL=1. --profiling-funcs forces the name section to be emitted even at high -O levels. Without it, the wasm has no labels and extract_signatures extracts nothing useful.

Write the labeled .wasm and a manifest row

Each artifact is written to the output directory under a tag built from the version and opt level (for example 3.1.55_O2/strings.wasm). The container is discarded; only the compiled artifacts and the manifest remain. A row describing the build flags is appended to manifest.json next to the .wasm files.

The output directory is self-describing. You can commit it, copy it to another machine, or hand it to a collaborator, and the harvest step reconstructs the same store because every label lives in the manifest rather than in the shell command that produced it.

The manifest

build_matrix.sh writes one manifest.json at the root of the output directory. It is the contract between the compile stage and the harvest stage: it records, for every artifact, the labels the harvester needs to tag the extracted signatures (the emscripten version, the opt level, the extra flags, and the relative path to the .wasm). Because the labels travel with the files, the harvester never has to re-derive them.

{
  "version": 1,
  "entries": [
    {
      "wasm": "3.1.55_O2/strings.wasm",
      "emscripten_version": "3.1.55",
      "opt_level": "-O2",
      "flags": "",
      "source": "strings.c"
    },
    {
      "wasm": "3.1.55_Oz/strings.wasm",
      "emscripten_version": "3.1.55",
      "opt_level": "-Oz",
      "flags": "",
      "source": "strings.c"
    }
  ]
}

Manifest entry fields:

Field	Meaning
`wasm`	Path to the labeled `.wasm`, relative to the manifest.
`emscripten_version`	The emsdk version the image used, passed through to each `Signature`.
`opt_level`	The `-O` level, for example `-O2` or `-Oz`.
`flags`	Any extra matrix flags for this build (`-pthread`, `-fexceptions`, LTO). Empty for the base matrix.
`source`	The reference program the artifact came from, useful when auditing a match.

Stage 2: harvesting the directory

A single harvest pass walks the output directory, reads the manifest, fingerprints every named function in every listed .wasm, and folds the results into one store. The library does this with harvest_directory:

from warden.oracle import harvest_directory

# Read manifest.json under root, fingerprint every listed .wasm,
# return one deduplicated SignatureStore.
store = harvest_directory("./corpus-build")
store.save("oracle.json")

harvest_directory(root) loads <root>/manifest.json, and for each entry it parses the .wasm, runs extract_signatures with the entry’s emscripten_version, opt_level, and flags, and extends them into the store. Because extend deduplicates, byte-identical functions that recur across the matrix collapse to a single signature, so harvesting a large matrix does not produce a bloated store. The whole pass is deterministic: the same directory always yields the same oracle.json.

warden oracle harvest ./corpus-build --out oracle.json

from warden.oracle import harvest_directory

store = harvest_directory("./corpus-build")
store.save("oracle.json")

warden oracle harvest <dir> is the CLI surface over harvest_directory. It is being wired into the CLI separately; the library function is the stable entry point in the meantime. Both read the same manifest.json and write the same oracle.json.

Why the matrix matters

Emscripten codegen varies across optimization levels in ways that affect fingerprints. A function compiled at -O0 has a different structural skeleton than the same function at -Oz after inlining and dead-code elimination. A corpus that covers only one opt level will miss matches in targets compiled at a different level. The MinHash fuzzy similarity (ORACLE_THRESHOLD = 0.82) tolerates some variation, but breadth of coverage in the corpus is the primary lever for match rate.

Reference programs

scripts/corpus/reference/ currently contains strings.c, a program that exercises musl string functions and the Emscripten runtime. The design calls for more: programs exercising pthreads, exceptions, different allocator configurations, and ideally portions of the Emscripten runtime itself. To widen the Oracle’s reach, add .c files to reference/ and extend the FLAGS array in build_matrix.sh to cover -pthread, -sPROXY_TO_PTHREAD, -fexceptions, and LTO. Each extra flag set becomes its own matrix entry, with its flags recorded in the manifest flags field, so the harvest step labels those signatures with the build that produced them.

The corpus accumulates into a single oracle.json. Because the format is plain JSON, it can be committed to your project repository and shared with collaborators. Everyone runs against the same ground truth without rebuilding.

Using the Oracle as a library

from warden.oracle import (
    SignatureStore,
    build_corpus_from_files,
    evaluate_identification,
    harvest_directory,
    identify,
    infer_version,
    ORACLE_THRESHOLD,
    load_seed_store,
)
from warden.kb import KnowledgeBase

# Build a store from a handful of labeled modules by hand
store = build_corpus_from_files(
    ["runtime_debug.wasm"],
    library="musl",
    emscripten_version="3.1.55",
    opt_level="-O2",
)
store.save("oracle.json")

# Or harvest a whole matrix-farm output directory in one pass
store = harvest_directory("./corpus-build")  # reads manifest.json, deduplicates
store.save("oracle.json")

# Audit what the store covers before running it
print(store.stats())  # total, by_library, by_emscripten_version, distinct_names

# Identify against the KB
with KnowledgeBase("project.db") as kb:
    v = kb.get_version("v1")
    matches = identify(kb, v.id, store, threshold=ORACLE_THRESHOLD)
    inferred = infer_version(matches)
    print(inferred.emscripten_version, inferred.confidence)

# Measure precision/recall on a held-out labeled module (no emsdk needed)
with KnowledgeBase("heldout.db") as kb:
    v = kb.get_version("v1")
    report = evaluate_identification(kb, v.id, store, threshold=ORACLE_THRESHOLD)
    print(report["precision"], report["recall"])

load_seed_store() loads src/warden/oracle/seed_signatures.json (the empty store that ships with the package). It is exported from warden.oracle for completeness but has no signatures until you build them.

Ingestion

What happens before the Oracle: parsing, fingerprinting, seeding from exports and the name section.

The agent crew

The Oracle handles runtime code; the agent crew names what remains.

Diff and carry-over

How the inferred Emscripten version feeds into toolchain-churn suppression during diffing.

CLI reference

Full flag listing for warden oracle build, warden oracle harvest, warden oracle inspect, and warden oracle identify.

Getting started

The pipeline

Reference

Project

The Emscripten Oracle

Compile your own ground truth

The signature store

`Signature`

`SignatureStore`

Introspecting a store with `stats()`

Signature dedup on `extend`

The shipped seed store is empty

Identifying functions: `identify` and `ORACLE_THRESHOLD`

Deterministic tie-breaking

The `OracleMatch` return type

Scaling: the MinHash-LSH index

How it works

Automatic band selection

Version inference

Calibrated confidence

Measuring identification quality: `evaluate_identification`

CLI usage

Step 1: Build a signature store

Inspect a store before you trust it

Step 2: Identify runtime functions in a target

Step 3: Check coverage

The emsdk matrix corpus farm

Stage 1: `build_matrix.sh`

The manifest

Stage 2: harvesting the directory

Why the matrix matters

Reference programs

Using the Oracle as a library

See also

Ingestion

The agent crew

Diff and carry-over

CLI reference

​Compile your own ground truth

​The signature store

​Signature

​SignatureStore

​Introspecting a store with stats()

​Signature dedup on extend

​The shipped seed store is empty

​Identifying functions: identify and ORACLE_THRESHOLD

​Deterministic tie-breaking

​The OracleMatch return type

​Scaling: the MinHash-LSH index

​How it works

​Automatic band selection

​Version inference

​Calibrated confidence

​Measuring identification quality: evaluate_identification

​CLI usage

​Step 1: Build a signature store

​Inspect a store before you trust it

​Step 2: Identify runtime functions in a target

​Step 3: Check coverage

​The emsdk matrix corpus farm

​Stage 1: build_matrix.sh

​The manifest

​Stage 2: harvesting the directory

​Why the matrix matters

​Reference programs

​Using the Oracle as a library

​See also

Ingestion

The agent crew

Diff and carry-over

CLI reference

Compile your own ground truth

The signature store

`Signature`

`SignatureStore`

Introspecting a store with `stats()`

Signature dedup on `extend`

The shipped seed store is empty

Identifying functions: `identify` and `ORACLE_THRESHOLD`

Deterministic tie-breaking

The `OracleMatch` return type

Scaling: the MinHash-LSH index

How it works

Automatic band selection

Version inference

Calibrated confidence

Measuring identification quality: `evaluate_identification`

CLI usage

Step 1: Build a signature store

Inspect a store before you trust it

Step 2: Identify runtime functions in a target

Step 3: Check coverage

The emsdk matrix corpus farm

Stage 1: `build_matrix.sh`

The manifest

Stage 2: harvesting the directory

Why the matrix matters

Reference programs

Using the Oracle as a library

See also