sqlite3, json, hashlib, and
struct (the [mcp], [openai], [anthropic], and [agents] extras add optional layers on top).
WARDEN is alpha. The public API described here is stable enough to build on, but a few
higher-level helpers (thread-model analysis, struct inference) are scaffolded and may change
shape before 1.0.
Package layout
Opening a knowledge base
KnowledgeBase is a context manager. It opens (or creates) a project database, applies the
schema if it does not exist, and commits on __exit__.
KnowledgeBase.__init__ accepts a str or pathlib.Path. The database file is created if it
does not exist; PRAGMA foreign_keys = ON is always set.
Ingesting a module version
ingest_into_kb is the workhorse. It parses the .wasm, fingerprints every function, records
a per-version appearance log, and seeds the durable symbol layer from free facts (name section,
exports, imports), giving fresh modules partial coverage before the Oracle or agents run.
ingest_into_kb signature:
IngestResult is a plain dataclass. You can safely pass it to dataclasses.asdict() or log it.
Querying the knowledge base
Module versions
ModuleVersion fields: id, label, wasm_sha256, emscripten_version, num_functions,
num_imported, shared_memory, ingested_at, notes.
Functions
id, version_id, func_index, stable_id, exact_hash,
structural_hash, minhash (list of ints), histogram (dict), call_targets (list of strings),
local_calls, type_signature, instruction_count, body_size, is_import, raw_name.
Symbols
Writing symbols
Always write throughupsert_symbol. This is what enforces the provenance/confidence economy
described in core concepts. Direct INSERT into the symbols table bypasses the
economy and risks clobbering verified work.
upsert_symbol returns (bool, str). The first value is True if the write was accepted, or False if rejected. A rejection is not an error; it means a higher-authority annotation already exists.
Symbol fields:
Diffing two versions
diff_versions matches functions between two ingested versions, classifies each as unchanged,
moved, modified, new, or deleted, and (by default) carries annotations forward via fuzzy
match with a confidence penalty.
diff_versions also calls kb.store_diff so the result is cached and retrievable with
kb.get_diff(from_version_id, to_version_id).
DiffReport fields: from_label, to_label, changes: list[Change], carried_symbols: int.
Change fields:
render_changelog returns a plain markdown string. Pipe it to a file or display it in a TUI.
Fingerprinting functions
fingerprint_function produces the four fingerprints + stable identity for a single function.
You usually call ingest_into_kb and let it handle this, but you can also call it directly for
ad-hoc analysis or to build custom corpora.
Comparing two functions
Low-level: MinHash Jaccard only
If you already have two MinHash signatures (e.g. reconstructed from KB rows viafingerprint_from_record), you can call minhash_jaccard directly:
fingerprint_from_record reconstructs a full FunctionFingerprint from a stored KB function
dict. This lets you run the similarity engine against already-ingested functions without
re-parsing the original .wasm.
Parsing a module directly
For scripting that only needs the parsed model (not the KB), import fromwarden.ingest:
End-to-end example
A complete two-version ingest, diff, and symbol write in one script:Audit log
Every symbol write (accepted or rejected) is recorded. You can read the last n entries:created, updated, or rejected.
Key constants
Built-in lifter (warden.lift)
lift_function and lift_module turn a parsed Module into readable pseudo-C with no external
toolchain. The lifter is a symbolic stack evaluator: it walks func.instructions, folds the
stack-machine operations back into infix expressions, and renders a deterministic text output that
can be diffed across binary versions. Unmodeled opcodes degrade to /* mnemonic */ comments
instead of raising, so every function always lifts to something.
Structured control flow. The lifter now reconstructs structured control
flow, not just straight-line code:
- Real
if/else. Result-typed ifs assign each branch into a temporary and use the temp as the expression value. whileloops withbreak/continue. The common WebAssemblyblock+loopidiom renders as a cleanwhile (1) { ... if (cond) break; ... }. A labeledgotois the fallback for control flow that does not fit the innermost-loop/break pattern, so output is always correct.switchforbr_tabletargets.
abs_demo lifts to an if/else that returns. sum_to_n lifts to a while loop
with break. Both are available via samples.control_flow().
lift_function is also the backend for warden export --format pseudo: pseudocode exports now
emit real structured pseudo-C rather than a mnemonic dump.
Round-trip symbol bridge (warden.bridge)
warden.bridge is the round-trip symbol bridge: export a version’s annotations to a neutral
CSV or JSON file, edit them in Ghidra, IDA, or by hand, then import them back. Imports go through
the provenance/confidence economy, so an import never clobbers higher-authority work.
Exporting
fmt accepts "csv" or "json". The output is deterministic. CSV columns are: index,
stable_id, name, type, provenance, confidence. JSON is a list of the same dicts.
Importing
ImportResult is a plain dataclass:
Stable identity keying
Resolution usesstable_id first (robust across builds where the function index can shift),
then falls back to index. A name recovered against one build lands on the same logical function
in a different build as long as the function body is identical or near-identical.
CLI
The bridge is exposed via two CLI commands:warden export --format csv|json is added alongside the existing headers, pseudo, kb-text,
and ghidra formats. The existing Ghidra rename script (warden export --format ghidra) is the
push side. The CSV/JSON formats complete the round trip.
Short round-trip example
Mini interpreter (warden.interp)
execute_function runs a function body on concrete integer inputs using a pure-Python, zero-dependency
interpreter for the i32 subset. differential_execute runs two functions over the same inputs and
reports per-input agreement, making behavioral equivalence concrete and runnable.
differential_execute row contains {"args", "a", "b", "match"} where a/b are the
result stacks (or None if that side raised UnsupportedExecution), and match is whether
both sides produced identical results. UnsupportedExecution is raised (or recorded as None)
whenever an opcode or construct falls outside the modeled integer subset. Callers treat it as
“cannot decide” rather than an error.
The interpreter models the i32 integer instruction set, structured control flow (block/loop/if),
memory loads and stores, and direct calls. Floating-point, SIMD, multi-value blocks, and
indirect calls raise
UnsupportedExecution.Specialized analyzers (warden.analysis)
Concurrency (warden.analysis.concurrency)
analyze_concurrency recovers the thread model in one deterministic pass: it detects shared
memory, locates every atomic-class instruction, and collects pthread-ish import/export names.
When a KnowledgeBase and version_id are supplied, each atomic site is persisted as a
kind='atomic' thread fact.
ConcurrencyReport fields: shared_memory (bool), atomic_sites (list[AtomicSite]),
pthread_markers (list[str]), facts (list[dict]), is_threaded (property).
AtomicSite fields: func_index, func_name, mnemonic, offset, instr_offset, site
(property).
Struct layout (warden.analysis.structs)
analyze_structs recovers candidate struct layouts from memory-access patterns. For each defined
function it groups fixed-displacement i32 loads and stores by base pointer local; each base with
at least one access yields a StructLayout. When a KnowledgeBase and version_id are supplied,
every layout is persisted via kb.upsert_struct at provenance="agent", confidence=0.5.
StructLayout fields: name (str), fields (list[StructField]), source_function (str | None).
StructField fields: offset (int), size (int), type (str), name (str).
Call graph (warden.analysis.callgraph)
build_call_graph extracts the intra-module call graph from a parsed module. Direct calls (call / return_call) are exact. Indirect calls (call_indirect / return_call_indirect and dynCall wrappers) are over-approximated to every table target whose type matches the call-site type index, since the static instruction only carries a type, not a concrete target. The result is a conservative skeleton that is always safe to schedule from.
layered_schedule condenses strongly-connected components (mutual recursion) via strongly_connected_components, then assigns a bottom-up depth to each component: layer 0 contains leaves (functions that call no other defined function), and each later layer’s functions have all their defined callees in earlier layers. Members of the same SCC share one layer. Functions within a layer are independent.
strongly_connected_components runs iterative Tarjan SCC on any node/successor pair you supply. It returns components in reverse-topological order (callees before callers), which is exactly the order the bottom-up schedule needs.
layered_schedule accepts an already-built CallGraph via graph=. When omitted it calls build_call_graph internally. The returned list is sorted at every level for determinism.
Static HTML report generator (warden.report)
render_report produces a single, self-contained HTML file (inline CSS, no external assets)
from a KB version. The report contains a coverage summary, a confidence heatmap of functions
colored by provenance and confidence, a thread/memory model section, and the diff changelog
from the nearest prior version. The output is deterministic: the same KB state always produces
byte-identical HTML, so reports diff cleanly alongside the binary artifacts they document.
module parameter is optional. When omitted, the report is driven entirely by the KB and
works even without the original .wasm on hand. write_report writes UTF-8.
Oracle LSH index (warden.oracle.index)
SignatureIndex adds a banded-MinHash + structural-hash index over a SignatureStore for
sublinear candidate lookup. identify_indexed is a drop-in replacement for identify that
queries the index rather than scoring every signature. The results are identical to a full scan
because any function with no index candidates falls back to scoring everything.
SignatureIndex.build signature:
identify_indexed signature:
--indexed flag in warden oracle identify <label> --store <path> --indexed calls this
function. Pass write=False to run a dry-run without touching the KB.
Agent crew (warden.agents.crew)
run_agent_pass drives one propose-verify-write-back sweep over all functions in a version.
It accepts two keyword arguments that control scheduling and parallelism.
strategy accepts "call-graph" (also accepted as "callgraph") or "flat".
The call-graph strategy works in five steps:
- Build the intra-module call graph with
build_call_graph. Direct calls are exact. Indirect calls are over-approximated to table targets of the matching type. - Condense strongly-connected components (mutual recursion) and produce bottom-up layers with
layered_schedule. Every function in a layer has all of its defined callees in earlier layers. - Run the concurrency and struct analyzers and route their findings into per-function notes (
notesfield onFunctionFacts). Atomic sites become synchronization hints. Struct accesses become field-layout hints. - Process layers bottom-up. Each function’s
FunctionFactsis enriched withcallee_names(the recovered names of its defined callees) before the backend proposes a name. Naming a caller with its callees’ meanings already established is the main quality gain over flat ordering. - Functions in the same layer are independent. They are proposed concurrently in-process using
asyncio. Blocking LLM backends run in worker threads, capped byconcurrency. Writes still go through the provenance/confidence economy, so concurrent branches that share a callee cannot overwrite each other.
flat strategy preserves the original single-pass ordering and runs serially. It is still available for backends where concurrency is undesirable.
The CLI passes --strategy directly: warden agent <label> --strategy call-graph.
FunctionFacts
FunctionFacts is the dataclass handed to every backend. The call-graph strategy populates two fields that are empty under flat: