Bitcoin and quantum computing: counting for yourself with a node

A typical case where anyone, with a little effort, can apply «don't trust, verify» all the way down: no number in this manual has to be taken on faith, you can count each one for yourself.

The principle: every number published here must be reproducible by anyone with a Bitcoin node, without trusting third-party reports or trusting us. It is «don't trust, verify» taken literally, and it is the thing the reader should carry away even before the numbers: on Bitcoin whatever is verifiable is available to everyone, and where verification is possible trust simply is not needed. The tools in this manual replace trust with repetition: the artifacts they build are a deterministic function of the chain — same heights, same bytes, same fingerprint, on any machine — and so two strangers compare a fingerprint instead of comparing trust. This manual collects the exact commands, the code (public and clonable: github.com/amenano/nodsig) and the practical cases that emerged doing it for real. Everything is read-only: no wallet involved, nothing written to the chain.

What you get, and at what price. The manual comes in three levels, which answer different questions and cost very different amounts of time. They are independent, in the sense that you can stop after the first or after the second and still have a complete, verifiable result.

The question What it takes What it costs
Level 1 how many coins exist right now, how many bitcoin, and what fingerprint acts as the anchor for that count a reachable node minutes
Level 2 how they are distributed by lock type and by age, and how many sit by construction on already visible keys the node, plus a snapshot of the UTXO set to be counted on another machine an afternoon
Level 3 how many coins sit today behind locks already opened in the past, that is, reuse one full pass over the history of the chain days, unattended

Whoever reads without a node in front of them can stop here and take the rest as the story of how these things are counted: the numbers, the fingerprints and the real timings are in the text and stand as a report even for those who do not redo the counts. Whoever does have a node will find the exact commands, in the form in which we ran them.

The numbers this procedure produced, together with what they say, are published separately: Bitcoin and quantum computing: the data (July 2026). Here you find how to obtain them, there what they mean.

Reference environment for the runs reported here (stating it is what makes the timings honest): a full non-pruned node on Umbrel (Raspberry Pi, external SSD), «Bitcoin Node» app v1.1.0. The commands hold for any Bitcoin Core; only the prefix used to invoke bitcoin-cli changes (see below).

How to read this manual: every command comes with a plain-language account of what it does, what it returns, why it is needed; the minimal concepts (UTXO set, height, hash, pruned node) are explained at first use. Those who already know skip the explanations; those who do not follow everything. Snags are reported as our own worked examples, at the end of the step, without making them the point: the point is that all of this is verifiable for oneself.

The stages. The first three parts read in sequence: you set up, you build, you ask. The fourth is reference material, opened when needed.

Before you begin

Building, in order

Asking

The design and the formats

Before you begin

Six pairs that get confused

Before the commands, six distinctions. They are not lexical pedantry: each one, if it is skipped, produces a wrong conclusion of the kind one sees going around a lot.

Revelation ≠ reuse. A public key is revealed on the chain the moment it spends: that is the norm, it happens to everyone, and in itself it leaves nothing in danger. Reuse is the coin that stays, or comes back, behind a lock whose key had already been revealed. The first is an event, the second is a state. The numbers in this manual count the second.

Exposed ≠ vulnerable. Exposed means the public key is readable on the chain. Vulnerable would mean somebody can use it to take those coins, and that would take a machine that does not exist today. Confusing the two terms turns a measurement into an alarm.

Lock ≠ wallet. A lock is a scriptPubKey identical to itself. A wallet is a set of keys in somebody's hands. Counting locks is not counting people: a single entity can sit behind millions of them, and a single wallet can use one alone for years.

Key ≠ address. The address is a readable encoding of a lock. The same key can present itself behind addresses of different shapes, its «faces», and it stays the same key. It is the reason the faces perimeter exists, and the reason it has to be declared every time.

UTXO set ≠ history. The UTXO set is the photograph of what exists right now, the unspent coins. History is everything that happened, including the coins already spent and their closed accounts. A question about the present and one about the past have different answers, and this manual keeps them on two separate levels on purpose.

Fingerprint ≠ signature. A fingerprint says that two files are the same file, and anyone recomputes it in order to compare. A signature says who produced something. Here we use fingerprints: they serve to make comparison possible between strangers who have no reason whatsoever to trust each other.

Reaching bitcoin-cli

All the commands in the manual are bitcoin-cli commands, the command-line interface of Bitcoin Core. Only how you invoke it changes with the installation; in the rest of the manual the commands are written in the generic form, with the Umbrel variant given where it matters.

Classic install on a PC (Bitcoin Core installed directly, the daemon running): bitcoin-cli is invoked and that is it, it finds the RPC cookie by itself in the default datadir (~/.bitcoin on Linux):

bitcoin-cli <command>

If the datadir sits in a non-standard place: bitcoin-cli -datadir=<path> <command>.

Umbrel: the daemon runs in a Docker container, and with the new Bitcoin app (v1.x, 2025 rewrite) there is no longer a separate bitcoind container: the daemon lives inside bitcoin_app_1, together with the interface. So:

ssh umbrel@umbrel.local          # password: the dashboard one

# to get your bearings: the list of containers with name and image
sudo docker ps --format '{{.Names}}\t{{.Image}}'
# → look for the image ghcr.io/getumbrel/umbrel-bitcoin → name bitcoin_app_1

# every bitcoin-cli in the manual becomes:
sudo docker exec bitcoin_app_1 bitcoin-cli <command>

Verified in the field: bitcoin-cli inside bitcoin_app_1 finds the RPC cookie by itself, no -datadir and no credentials needed.

Worked example, from our own setup: filtering docker ps for «bitcoin» also turns up the surrounding containers (bitcoin_tor_1, bitcoin_i2pd_daemon_1, bitcoin_app_proxy_1): those are Tor, I2P and the web proxy, not the node.

Other packaged installs (Start9, RaspiBlitz, MyNode…) each have their own way of exposing bitcoin-cli; the principle does not change: find where the daemon runs and prepend whatever is needed to the command.

Privacy and sensitive data

Before the commands, the data perimeter: what, of all that these steps produce, can be shared, and what cannot.

Two classes of data, always to be kept apart:

  • Shareable without thinking about it — public chain data or aggregates: heights, block hashes, muhash, totals (txouts, BTC), tables by type/age, the output of getblockchaininfo and gettxoutsetinfo, container names. They say nothing about whoever produced them;
  • SENSITIVE — never in a chat, never on a forum, never to third-party services: YOUR OWN addresses and anything that links them to you; the input list file of the check (it is the map of your wallet); its OUTPUT (addresses + balances + answers: worse still); xpubs; RPC credentials; the seed, obviously, NEVER anywhere in any form.

Two command traps worth knowing, because the second class likes to slip out by side routes: docker inspect <container> and printenv inside the container EXPOSE the environment variables, RPC credentials included; and bitcoin.conf contains them too. Never paste their raw output anywhere — not even when asking for help.

Design rules for the scripts (checkable by reading the code):

  • they talk ONLY to your own node: no calls to external endpoints, ever — whoever reads the code must be able to establish that in a minute;
  • the check writes its outcome to a local FILE (not to the screen by default: a screen ends up in screenshots, in shares, in remote sessions) and prints a warning at the top: «this file describes your wallet: treat it as such»;
  • the input/output files of the check are to be treated as wallet metadata: delete them after use or keep them encrypted;
  • the aggregate outputs (levels 1-2) are shareable by construction: it is a design CRITERION, not an accident (the summary never contains individual addresses).

Rule of this manual: if a step produces second-class data, it is said BEFORE the command — with the warning about the classic leak, which is asking for help on a forum (or from an AI) by pasting output with your own addresses inside it.

Architecture: the node extracts, another machine processes

Underlying choice: on the node only the read-only bitcoin-cli commands run, plus the production of the snapshots; all our own processing (parser, analysis, tables) runs on another machine, where there is CPU and development agility. Reasons: the node stays clean (no dependency installed, no risk: it also runs Lightning and other services); on a Pi, parsing ~166M entries in Python would cost hours against the minutes of a PC; transferring the file (~12 GB over scp on the local network, ~5-10 minutes, once) is the false problem.

For the reader nothing changes conceptually: the scripts take a file as input and run ANYWHERE there is Python — whoever has the node on a PC runs them there, whoever has a Pi runs them wherever they prefer. Trust does not move (they are your own machines either way) and the verification hook stays the height+muhash pair computed by the node.

Exceptions consistent with the principle: the address check, for the balance and for that alone, talks to the LIVE node and not to a file, so it runs from the PC over an SSH tunnel and only if you ask it to with --rpc; level 3 does not copy the ~900 GB of blocks but READS them from the local network (over RPC, or over the node's REST interface if enabled: see the Prerequisites), grinding on the PC.

Levels 2 and 3 process on a machine other than the node (see «Architecture»): they need a channel towards the node's RPC and its credentials. You set it up once, before the steps that query the node (level 1 only uses bitcoin-cli on the node and does not need it).

Access to the node, without touching its configuration. No changes to rpcbind/rpcallowip: the node's configuration is not touched. Verified on the pilot: the bitcoin_app_1 container publishes RPC on port 8332 of the host, and Docker publishes it on 0.0.0.0 — that is, the RPC of an Umbrel node is already reachable from the LAN as it stands, by construction (the other apps need it). This does NOT make the tunnel superfluous, quite the opposite: JSON-RPC authentication is HTTP Basic, that is, it travels in the clear, and without a tunnel the cookie would cross the LAN in the clear at every request. So: ssh -N -L 8332:127.0.0.1:8332 umbrel@<pi-ip> — the PC sees RPC on 127.0.0.1:8332 and the credentials stay inside the encrypted channel. The credentials: the RPC cookie (user __cookie__), read with docker exec from the container's datadir and saved to a local file with 600 permissions, NEVER shown on screen; end-of-session cleanup: the file is deleted (and the tunnel is closed). The cookie rotates at every restart of bitcoind — which is also how to REVOKE it if needed — and the script says so plainly if authentication fails. Typing passwords (SSH/sudo) is a step that stays in the user's terminal, outside any assisted session, as with signing.

How the cookie is copied, in practice. sudo on the node asks for the password interactively, and typing it while capturing stdout into a file (via a pipe or ssh -t) does not work: the prompt ends up in the output or the session hangs. So the two things are separated — the sudo in a real interactive SSH session, which writes the cookie into a temporary file belonging to the user on the node, then it is copied over with scp.

Step 1, on the node (the sudo password is typed here, in the user's terminal):

ssh umbrel@<pi-ip>
(umask 077; sudo docker exec bitcoin_app_1 cat <datadir>/.cookie > ~/cookie.tmp)
exit

The > redirect is done by the user's shell on the node, so ~/cookie.tmp is theirs and is born 600 (umask 077); sudo only elevates the docker exec, the cookie never appears on screen.

Step 2, from the PC — copy and delete the temporary file:

mkdir -p ~/utxo_work
(umask 077; scp umbrel@<pi-ip>:~/cookie.tmp ~/utxo_work/.btc_cookie)
ssh umbrel@<pi-ip> 'shred -u ~/cookie.tmp'
chmod 600 ~/utxo_work/.btc_cookie

The datadir is the one of bitcoind inside the container (on Umbrel usually /data/.bitcoin), the container is your own node's (here bitcoin_app_1, verified on the spot); the node's SSH user is usually not in the docker group, hence the sudo. The secret stays for an instant on ~/cookie.tmp (600), then shred -u; locally it is 600. Check it WITHOUT printing it — right format and no error message captured by mistake:

wc -c ~/utxo_work/.btc_cookie                       # ~70+ bytes
grep -q '^__cookie__:' ~/utxo_work/.btc_cookie && echo OK || echo INVALID

The __cookie__: prefix is not secret (it is always that one). Then you pass --cookie-file ~/utxo_work/.btc_cookie to the tools: no flag accepts a secret on the command line, by design — a process's argv is readable by anyone on the machine, for as long as the run lasts (the alternative, for a node with fixed credentials, is the NODSIG_RPC_AUTH environment variable). At the end of the session, shred -u on the local file and the tunnel down. The cookie rotates at every restart of bitcoind: if authentication fails, you do it again.

A cheaper wire, if the node offers it (--rest). Bitcoin Core's REST interface (rest=1 in bitcoin.conf, served on the same port as RPC) delivers blocks in binary instead of hex inside JSON: about half the bytes on the wire, for the one step whose cost IS the wire. And it authenticates nobody, so the fetch carries no credentials: the cookie only serves the RPC calls that remain. The transport is chosen for the bytes, never for trust: the integrity checks do not change in the slightest — every byte is hash-verified anyway, see «The incremental filing cabinet» — and the two transports are proven to produce byte-identical artifacts. REST has no batching, two requests per block, so it goes together with --prefetch-depth <n> to keep more requests in flight. Through the SSH tunnel nothing changes: same port, same encrypted channel.

Building, in order

Level 1 — the anchored totals (minutes)

The concept to have in hand before the commands, and here an extended definition is worth it because it is the object the whole manual works on: the UTXO set (Unspent Transaction Output set: the set of transaction outputs that have not been spent).

Bitcoin has neither accounts nor balances. The ledger never says «address X owns Y bitcoin»: it records transactions and nothing else. Every transaction consumes some existing coins entirely (the outputs of previous transactions) and mints new ones; every new output is a coin, with an arbitrary amount and a lock (the script) that declares what it takes to spend it. More cash than current account: coins of arbitrary denomination, each with its own padlock, and spending means melting down the chosen coins and minting new ones (including the change, which is a new coin towards oneself). The «balance» a wallet shows is a convenient fiction: the sum of the coins its keys know how to open.

The UTXO set is the set of all the coins not yet consumed, at this instant: the present state of the system, distilled from the entire history. Whoever starts again from the genesis block and replays every transaction arrives at exactly this set; and it is the set every node maintains in order to validate, because a transaction is valid only if it consumes coins that are inside it (that is how double spending dies: the consumed coin leaves the set, and a transaction that tried to spend it again would find nothing). At the same block height, the set is IDENTICAL for every honest node in the world: which is why a fingerprint computed over it can be compared between strangers.

Every entry in the set carries four things with it: the amount, the lock, the «is this a mining reward?» flag and the height of the block it was born in. The last two are a gift for us: they make the census by type AND by age possible without reading the history.

It is the set on which we will do all the counting: whoever wants to know how many coins have their key exposed has to count in there. Level 1 establishes the control totals: how many entries, how many bitcoin in all, and a fingerprint of the set that lets anyone verify we are counting the same thing.

1a. Photograph of the node

  • Goal: confirm that the node is complete and in step with the network, and fix the starting height.
  • What it is for: the height is the «date» every number in the manual travels with; it is the premise of every count.
  • Requires: a synchronised, non-pruned Bitcoin Core node (see «Reaching bitcoin-cli»).
  • Made of: bitcoin-cli getblockchaininfo.
bitcoin-cli getblockchaininfo

(on Umbrel: sudo docker exec bitcoin_app_1 bitcoin-cli getblockchaininfo)

What it does: asks your own node where it stands on the chain. What it returns: a little JSON block with the node's state. Why we need it: it proves you are counting on a complete node in step with the network, and it fixes the starting point (the height).

What to check in the output, entry by entry:

  • "chain": "main" — you are looking at the real Bitcoin network, not a test one;
  • "blocks" = "headers" and "initialblockdownload": false — the node has downloaded and verified the whole chain, it is not still catching up;
  • "pruned": false — non-pruned node: it keeps all the blocks since 2009, not only the recent ones (needed for the historical analyses of the later levels). Note that the detour is worth it: to validate the NEXT block the history is not needed, the current state is enough (a pruned node validates with the same rigour: it verified the whole history once, then discarded it). The history is needed by the NEWCOMERS: whoever turns on a node today can verify from scratch only if somebody keeps the blocks and serves them — if nobody kept them, one would have to accept a state packaged by others, on trust. This is why the history lives in many independent copies and not in an official archive (which would be an arbiter all over again). Three layers, three degrees of necessity: consensus needs only the state; verifiability by anyone needs the history; convenience needs the indexes (see the note in the address check);
  • "blocks" is the height: the sequential number of the last block. It is the «date» of every on-chain measurement: every number we publish travels with the height at which it was counted.

Our run, at block 957,281:

  • height: 957,281; bestblock 000000000000000000015137f080f2ca0fa0be807f32ca206aac0f810805dfa9
  • pruned: false, chain on disk ~858 GB, no warnings

1b. Counting the UTXO set

  • Goal: the control totals of the UTXO set (number of entries, total bitcoin) and its muhash fingerprint.
  • What it is for: it is the frame of every number that follows; the fingerprint lets a third party verify they are counting the same set.
  • Requires: the node from step 1a.
  • Made of: bitcoin-cli gettxoutsetinfo muhash.
bitcoin-cli -rpcclienttimeout=0 gettxoutsetinfo muhash

(on Umbrel: sudo docker exec bitcoin_app_1 bitcoin-cli -rpcclienttimeout=0 gettxoutsetinfo muhash)

What it does: it reviews the entire UTXO set, entry by entry, and meanwhile computes an overall fingerprint of it, the muhash. It deserves its own definition, because it is the verification hook of the whole manual: it is a fingerprint designed for sets. Every entry is reduced by a hash to an enormous number, and the numbers of the entries are combined with one another by multiplication: since the order of the factors does not change the product, the fingerprint depends only on what is in the set, not on the order in which it is visited; and since adding or removing an entry amounts to multiplying or dividing, the node can keep it up to date block after block without ever redoing the count from scratch. Two equal sets give the same muhash; one different entry is enough for the fingerprint to change completely, with no recognisable relation to the previous one. What it returns: the total of the entries (txouts), the total of the spendable bitcoin (total_amount), the height at which it counted (height) and the fingerprint (muhash). Why we need it: it is the frame of every number that follows. The fingerprint is the point: two honest nodes at the same height obtain the same muhash; whoever redoes our counts can first verify they are counting on the same set, then compare results.

Caveats:

  • it scans ~166 million entries: the command seems to be doing nothing and then answers all at once. On a recent PC it is minutes; on a Raspberry Pi the CPU is the bottleneck and it can take 30-90 MINUTES (verified in the field). To check that it is working, from another window: sudo docker stats bitcoin_app_1 --no-stream → CPU% ~100 = it is computing. It is read-only, the node goes on working normally;
  • on slow hardware it is worth launching it inside tmux (or with nohup … > result.json &): if the SSH session drops, the node finishes the computation but the answer has nobody left to reach. Do NOT relaunch the command in parallel «just in case»: a second full scan would start alongside the first;
  • -rpcclienttimeout=0 tells the client to wait without a time limit: the default (15 minutes) on slow hardware can expire before the answer arrives;
  • the count happens at the height the node has at that moment (not necessarily the one of step 1a — the network does not wait for us): height and muhash are always to be published together, they are the pair that makes the count comparable. At a different height, a different muhash but comparable totals.

Fields to report: height, bestblock, txouts, total_amount, muhash.

Our run, at block 957,282 (duration ~1h+ on the Pi: the chain advanced by 7+ blocks during the computation):

{
  "height": 957282,
  "bestblock": "000000000000000000011e8f11c5e0333ed259047f7c4db5fe84805de0243000",
  "txouts": 166238015,
  "muhash": "38dd237f9f83e14d2e8fefa396871a29c9a6226fc7f1f89eda2d9279e314671b",
  "total_amount": 20053779.26088761,
  "transactions": 114967902,
  "disk_size": 11478352648
}

Consistency check, and this is the didactic point: at height 957,282 the maximum theoretical issuance is ~20,054,009 BTC (10.5M + 5.25M + 2.625M + 1.3125M for the first four subsidy epochs + 117,283 blocks × 3.125). The node counts ~230 BTC less, and it is RIGHT: what is missing is the genesis coinbase (50 BTC, unspendable by construction) and the rewards claimed incompletely by some miners over the years. The independent count agrees with the protocol to the cent.

Notes: the count is at the height the command STARTED at (957,282, one block after photograph 1a); disk_size ~11.5 GB confirms the expected size of the level 2 transfer.

Level 2 — the table by type and by age (an afternoon)

The tools are ready and tested; the execution sits on the node (steps 2a-2b), then 2c on the working machine. Two-machine flow, see «Architecture»: the node takes the photograph, another machine counts it.

The concept before the commands: level 1 measured the UTXO set (how many entries, how many bitcoin, a fingerprint); level 2 writes it to a file — the snapshot — and counts it entry by entry: for each coin, what type of lock closes it and at what block height it was born (every entry in the UTXO set carries the creation height with it: that is what makes the distribution by age possible without reading the history). Out of it come the table by type, the certain lower bound of the coins with an exposed key (P2PK + P2TR + bare multisig) and the answer to «how much has been sitting still since the era of the origins?».

2a. The snapshot on the node

  • Goal: write the UTXO set to a file (utxos.dat) at a fixed height, with its content fingerprint txoutset_hash.
  • What it is for: it is the raw material of the census (2c) and of the level 3 prepare; it fixes the height everything holds at.
  • Requires: a node with Bitcoin Core ≥ 28 and disk space for ~12 GB.
  • Made of: bitcoin-cli dumptxoutset.

Prerequisite: Bitcoin Core 28 or later (the v2 snapshot format). To check the version:

bitcoin-cli -version

(on Umbrel: sudo docker exec bitcoin_app_1 bitcoin-cli -version)

Then the snapshot proper:

bitcoin-cli -rpcclienttimeout=0 dumptxoutset utxos.dat latest

(on Umbrel: sudo docker exec bitcoin_app_1 bitcoin-cli -rpcclienttimeout=0 dumptxoutset utxos.dat latest)

What it does: it reviews the entire UTXO set (the same scan as level 1) and writes it compressed into a file, utxos.dat. What it returns: a JSON with coins_written (how many entries it wrote), base_height and base_hash (the height and the block of the photograph), path (where it wrote the file) and txoutset_hash (a fingerprint of the CONTENT of the snapshot). Why we need it: it is the raw material of the census; every level 2 count holds at height base_height.

Caveats:

  • a relative path (utxos.dat and nothing else) ends up INSIDE the node's datadir: in our environment that is the right choice, because the datadir sits on the SSD — never give absolute paths on the system SD card (19 GB: the ~12 GB file would fill it). Before starting, check the space: df -h /mnt/data;
  • same timings and same cautions as level 1, plus writing the ~12 GB: on a Pi expect 1-2 HOURS; launch it inside tmux, do not relaunch it in parallel, and in the meantime avoid other heavy RPC commands;
  • to be RECORDED (they go in the log at the end): base_height, base_hash, coins_written, txoutset_hash.

Our run, at block 957,301 (early July 2026; in tmux on the Pi):

{
  "coins_written": 166224061,
  "base_hash": "000000000000000000005f3f0d5434e16bb6ea806ecd140be91bf6a15562c077",
  "base_height": 957301,
  "path": "/data/bitcoin/utxos.dat",
  "txoutset_hash": "ec79aed99b8d22d7315c76488fe09975ef9258039ee5583c7e8c7b6dc8d4cc58",
  "nchaintx": 1393498474
}

An observation worth reusing in the text: 166,224,061 entries at 957,301 against the 166,238,015 of level 1 at 957,282 — ~14,000 entries FEWER in 19 blocks. The UTXO set can also shrink: it happens when transactions consume more coins than they create (consolidations). The two numbers do not have to coincide, they have to be CLOSE; and it is a good example of why every count travels with its own height. (nchaintx = total transactions in history up to that block, of no use to us here.)

The verification point, and it is the twin of level 1's muhash: anyone with a node running Core ≥ 28 can redo the snapshot AT OUR VERY HEIGHT even after the chain has moved on, with the rollback variant (dumptxoutset utxos.dat rollback=<base_height>: the node temporarily rewinds its own state back to that height — a legitimate but heavy operation, to be done on a machine with room to spare, and not required in order to follow the manual). Same height → same content → same txoutset_hash: our numbers are reproducible EXACTLY, not «roughly». Whoever instead takes the snapshot at their own current height will get slightly different totals (the chain has grown in the meantime) but the same proportions: both checks are honest, they say different things.

2b. The transfer to the working machine

  • Goal: bring utxos.dat from the node to the machine that processes it, with integrity verified by an identical fingerprint at both ends.
  • What it is for: the census runs where the CPU is, not on the node (two-machine architecture).
  • Requires: the snapshot from 2a and SSH access to the node.
  • Made of: docker cp / scp for the transport, sha256sum for the check.

The file has to go where the CPU is (see «Architecture»). In our environment it sits inside the Docker container, and the route verified in the field goes in two stages: first an interactive SSH session on the node, to bring the file out of the container into a place readable by your own user, on the SSD:

ssh umbrel@umbrel.local
# inside the session:
sudo docker cp bitcoin_app_1:/data/bitcoin/utxos.dat /mnt/data/utxos.dat
sudo chown umbrel /mnt/data/utxos.dat
sha256sum /mnt/data/utxos.dat        # fingerprint on the node side, while you are there
exit

What it does: copies the file from the container to the node's disk and makes your own user its owner, so the next step needs no permissions. Why in an interactive session: sudo must be able to ask for the password.

Then, from the working machine, an ordinary scp:

scp umbrel@umbrel.local:/mnt/data/utxos.dat .

(classic install on a PC: only the scp, or nothing at all if node and working machine are the same)

Timings: local network, ~9.6 GB → from 5 to 20 minutes depending on network and disk; the docker cp and the sha256sum on the Pi also grind away for a few minutes in silence, which is normal.

There is only one destination, however you get there with your own setup: the file on the working machine with a matching fingerprint. Worked examples from our own path, in case something gets in the way:

  • the «all in one» command (ssh node "sudo docker exec … cat …" > file) is more direct but needs passwordless sudo; otherwise it stops with sudo: a terminal is required to read the password, and that is the reason for the two-stage detour above. Worth knowing: getting around it with ssh -t is no good for binary files (the virtual terminal alters the bytes in transit);
  • Host key verification failed on the first connection from a new machine is SSH asking you to recognise the node, not a password problem: one interactive test connection, yes to the fingerprint, and you carry on (without disabling the check with StrictHostKeyChecking=no);
  • on Windows the fingerprint is computed with Get-FileHash utxos.dat -Algorithm SHA256 (PowerShell; it comes out in uppercase, the comparison ignores case).

Integrity check of the transfer, on both machines:

# on the node:
sudo docker exec bitcoin_app_1 sha256sum <path>
# on the working machine:
sha256sum utxos.dat

What it does: computes the SHA-256 fingerprint of the file, on this side and on that. Why we need it: if even a single byte had been corrupted on the way, the two fingerprints would differ (and the census would fail in strange ways). Equal fingerprints = identical copy, carry on.

Privacy: the snapshot is the public UTXO set, identical for every node — first class, shareable. Once the transfer is verified, TWO copies are left on the node to delete (the one in the container and the one from the docker cp):

sudo docker exec bitcoin_app_1 rm /data/bitcoin/utxos.dat
sudo rm /mnt/data/utxos.dat

The copy on the working machine is instead worth KEEPING: it is an anchored and reproducible artifact (same height → same txoutset_hash), reusable for other analyses at the same height without redoing the dump.

2c. The census (nodsig census)

  • Goal: from utxos.dat, the table by lock type, the certain floor of the coins with an exposed key, the distribution by epoch and a CSV of the aggregates.
  • What it is for: these are the published numbers of level 2; the floor goes into the exposure total.
  • Requires: utxos.dat on the working machine (2b).
  • Made of: nodsig census, with its test in the battery.

The tools live in the public repo nodsig (github.com/amenano/nodsig, MIT licence; see «Public GitHub repo» at the end). It takes Python 3.10 or later and nothing else: no dependencies, the standard library is the whole runtime. Every command in the manual has the form nodsig <group> …; from a clone, without installing anything, the same program is invoked with python3 -m nodsig <group> …, and a single module with python3 -m nodsig.<module>, which is the way to read the code while it runs.

  • the census (nodsig census): it reads the snapshot in streaming (never all of it in memory: it runs on modest machines too) and produces the table by type, the certain lower bound of the coins with an exposed key, the distribution by subsidy epoch of the exposed types and a CSV with the totals by type and by band of 50,000 blocks. ONLY aggregates: a single address never appears in the output — shareable by construction;
  • its test: it builds a synthetic snapshot of invented coins (writing the format with an independent implementation, mirroring the reader) and verifies every number of the census, including the treacherous case of a coin straddling an epoch boundary (height 215,000: a band that starts in epoch 1, a coin in epoch 2). Run: PASS.
python3 -m pytest -k census -q     # the census test, with no real data
nodsig census utxos.dat --csv census.csv

(The test is the only thing in the whole repo that asks for an external dependency, pytest. The tools do not: those run on the standard library alone, and it is a property you can check by reading the imports.)

What it does: it decodes the format of the snapshot (it is the same one read by the reference converter published by Bitcoin Core in contrib/utxo-tools/utxo_to_sqlite.py, from which the details were verified at the source) and classifies every entry. What it returns: the summary on screen; the CSV of the aggregates on file. At the top it prints the hash of the block written in the snapshot: it MUST match the base_hash from step 2a, and the entries read must match coins_written — it is the thread that ties the census to the photograph. Why we need it: it is the table by type and by age, with our own numbers.

Measured timings (benchmark on a synthetic 2M-entry snapshot): ~10 seconds for 2 million entries on a recent PC → ~15 minutes for the ~166M real ones. Python, standard library only, no dependencies.

Cross-checks:

  • census totals vs level 1 (txouts, total_amount): at the same height they must match EXACTLY; at nearby heights, almost;
  • the same snapshot fed to Bitcoin Core's utxo_to_sqlite.py → a few SQL queries on the totals → they must match our CSV;
  • possibly bitcoin-utxo-dump (Go, reads the chainstate on its own account) as a third opinion.

Our run, at block 957,301:

  • transfer: an 8.9 GiB file (~9.6 GB), sha256 verified on both shores. Note: the «~12 GB» estimate came from level 1's disk_size, which measures the chainstate on disk (LevelDB), not the compact dump: ~57 bytes per entry;
  • census on the PC (WSL, file read from a Windows disk): 10 minutes for 166,224,061 entries. Checks passed: hash of the block at the top of the snapshot = base_hash of the dump (…5562c077); entries read = coins_written, file consumed down to the last byte;
  • the cross-check: census total 20,053,838.63588761 BTC against 20,053,779.26088761 of level 1 → difference 59.375 BTC = 19 blocks × 3.125 EXACTLY. Fees do not create money (they move it from senders to miners): the total grows only by the subsidy. Two independent measurements, 19 blocks apart, validate one another down to the satoshi.
=== Census by script type ===                (height 957,301)
type                       pubkey             entries                    BTC
P2PK uncompressed key      EXPOSED             34,343     1,709,682.97233657
P2PK compressed key        EXPOSED             10,271         6,136.56019352
bare multisig              EXPOSED          2,621,788            70.29199448
P2TR (Taproot)             EXPOSED         54,433,089       216,699.01357424
P2PKH (1…)                 behind hash     44,412,522     4,599,483.71774068
P2SH (3…)                  behind hash     12,144,817     3,937,453.24907191
P2WPKH (bc1q…, short)      behind hash     49,599,918     8,163,121.81137845
P2WSH (bc1q…, long)        behind hash      2,853,141     1,418,572.89611611
witness other/future       other               93,402             0.51578406
other / non-standard       other               20,770         2,617.60769759
TOTAL                                     166,224,061    20,053,838.63588761

key exposed by construction (certain lower bound):
57,099,491 entries, 1,932,588.83809881 BTC (9.64% of total)

What these numbers say:

  • certain lower bound: ~1.93M BTC (9.64%) with the key exposed by construction, WITHOUT counting reuse (which on its own brings public estimates to 25-34%: our number is the verifiable floor, not the total);
  • the «was it Satoshi» question, with our numbers: of the ~1.716M BTC in P2PK, 1,658,384 BTC (~97%) were born below height 100,000 (≈ before the end of 2010); 1.294M below 50,000. The distribution by epoch confirms it: P2PK is a fossil of epoch 1, almost nothing after 2012;
  • P2TR is the opposite: 54.4M entries but only ~217k BTC, almost all born in epoch 5 — many coins, little value, all of it recent;
  • an honest curiosity worth telling: bare multisig has 2.6M entries and SEVENTY BTC — dust, largely a residue of the use of those scripts to inscribe data on the chain (2013-2014). Entries are not value;
  • «other / non-standard»: 20,770 entries, ~2,617 BTC (it includes P2PK with malformed keys and exotic scripts): a declared category, not a hidden one.

Language decision: code, comments, identifiers and the textual output of the tools are in ENGLISH — they are born for an international public repo, and English output is not a barrier (any more than the JSON of bitcoin-cli is). The didactic layer is THIS manual, which shows the output and explains it. Consequently the address check is invoked as nodsig check.

Why Python and not C: (1) the code is part of the argument — «read for yourself what is being counted» works if the ~300 lines read almost like pseudocode, not if the logic drowns among buffers and pointers; (2) no compilation = no new trust: a binary is what we want to avoid, a C source would have to be compiled (toolchain, platforms), python3 runs identically everywhere with the standard library alone; (3) memory safety: 12 GB of binary parsing in C = a class of vulnerability, in Python at worst a clear exception; (4) C's speed buys nothing here: 15 minutes once per publication, the bottleneck is the dump on the Pi (hours); (5) it is the differential declared in the survey (the serious parsers are all compiled; Core's reference one is in Python like ours).

Level 3 — reuse (days, optional)

The third level's question: how many coins sit today behind locks already opened in the past. «Optional» in a precise sense: total exposure is made of two addends, and the first two levels close the first one on their own — the floor by construction, the 9.64% — which is already a complete, verifiable result. This level adds the second addend, reuse; whoever stops earlier does not have a maimed number, they have a floor that by construction errs only on the low side. The price of the second addend: one full pass over the history of the chain, days of unattended machine time. And it is the only level that can also be left half done while still giving an honest number: every block read can only add burnt ones, never take any away. The design that holds it up — why a single pass suffices, and what exactly is compared — is in «The three ideas of level 3», in the part on design; here are the commands.

The shape of the result: total exposure = floor by construction (9.64% at 957,301) + reuse at a declared perimeter, to be compared with BIP-361's >34%. The two addends, with the real numbers of our run, come out of the steps below; what they mean is the data post's business.

Run and published: the commands, the timings and the fingerprints below are the real ones from our run at block 957,301, with version 1.0.0 of the tools. Whoever repeats the sequence at the same height obtains, by construction, the same fingerprints.

A clarification on version numbers, before the two numberings meet: the tools are at release 1.0.0; the formats they write — reveal-archive-v2, graph-v2, and so on — carry the number of their own specification, which is a separate scale, documented in the repo (docs/formats/). A format changes number only when the bytes that compose it change, not when a release comes out: the -v2 in the names below does not allude to a «nodsig 2».

One rule before all the steps: you choose ONE height and use it everywhere. An artifact is defined by where it stops, and pieces cut at different heights do not join.

The reader needs only one scan of the chain, the archive route (nodsig archive): from there the archive is sealed and the reuse table and the curve are derived. The routes we walked were two, because the second served to check the first by an independent way. That check is a result, reported at the end of the steps: it is not a task the reader has to redo.

The timings given are ours, on a modest node (Umbrel on a Raspberry Pi, blocks over RPC on an SSH tunnel): they depend on the hardware and on the channel, take them as orders of magnitude. The ETA the tools print at every checkpoint is naive (it projects the current rate, but blocks fill up as height rises): trust the timestamps, not that. A table of timings is more useful when it describes the hardware somebody already has, not the hardware the tool would prefer: ours is measured on the slow setup.

Step 1 — prepare: distil the current locks from the snapshot.

  • Goal: from the level 2 utxos.dat, the list of the «behind hash» locks now in the UTXO set, in sorted, searchable files (locks_p2pkh.bin, _p2sh, _p2wpkh, _p2wsh).
  • What it is for: it is the target set against which every revelation is checked, and the input of derive/crosscheck.
  • Requires: the level 2 snapshot (utxos.dat, same height 957,301).
  • Made of: nodsig reuse prepare.
nodsig reuse prepare "$SNAPSHOT" --out "$LOCKS"
  • Verification: the counts by type must correspond to the level 2 UTXO set — in our run 21,267,062 p2pkh, 6,240,320 p2sh, 23,060,044 p2wpkh, 1,166,793 p2wsh. Quick (one pass over the snapshot).

Step 2 — scan: the single pass over the archive of revelations.

  • Goal: walk the chain up to the height of the snapshot, recording every revelation of a key/script — together with the height of its first appearance — into an appendable archive (reveal-archive-v2). With --graph, the same pass co-emits the raw graph (graph-v2); with --headers, the headers archive (~150 MB), which makes the pass's integrity checks repeatable offline and gives calendar dates to the commands that want them, with no more asking the node.
  • What it is for: it is the source from which derive extracts reuse and curve and merge seals the archive; the graph feeds the history and co-spend capabilities of the check, and the re-derivations by epoch.
  • Requires: the node reachable (see «Reaching bitcoin-cli»; for the SSH tunnel and the cookie see the Prerequisites; --rest is the leaner alternative for fetching the blocks). It does NOT require the prepare: the archive records everything, the comparison with the locks happens afterwards.
  • Made of: nodsig archive scan.
nodsig archive scan \
  --rpc http://127.0.0.1:8332 --cookie-file "$COOKIE" \
  --end 957301 --archive "$ARCHIVE" --graph "$GRAPH" --headers "$HEADERS" \
  2>&1 | tee -a ~/utxo_work/scan.log

--graph is optional: leave it out if you do not need to build the index and the derivatives afterwards. --headers is optional too, but it costs 150 MB and not hours: convenience is almost always on its side. There is a third co-emitted artifact, --nonces (59.7 GB and ~10% more CPU): the census of the nonce points of every signature, the only thing a later pass could not rebuild, because the artifacts that are kept contain no unlocking data. It answers a different question — whether a key, by signing different messages with the same nonce, has made itself derivable — and it has its own page in the repo (docs/nonce-check.md): this manual does not cover it.

  • Verification: at the end of the pass the archive covers heights 1..957,301. In our run: 7,704,658,388 revelations in 2,787 runs, malformed scriptSigs: 0. Duration: 58 h 47 on our setup (dominated by the wire towards the Pi).

Step 3 — derive: the reuse table and the curve, before the seal.

  • Goal: from the archive plus the locks, the table of burnt ones by type and the curve of reuse vs share of history read.
  • What it is for: they are the published numbers of exposure by reuse and the data of the curve.
  • Requires: the archive from step 2 and the locks ($LOCKS, step 1). It must run BEFORE the merge: the curve is born from the tiling of the scan runs into intervals, and the fusion spends that tiling to produce the canonical form. First you derive, then you seal.
  • Made of: nodsig archive derive.
nodsig archive derive \
  --archive "$ARCHIVE" --locks "$LOCKS" \
  --curve ~/utxo_work/archive_curve_957301.csv \
  2>&1 | tee -a ~/utxo_work/derive.log
  • Verification: TOTAL 8,784,364 burnt locks over 5,084,725.41330985 BTC (p2pkh 3,925,128 / 1,185,751.57; p2sh 1,155,896 / 1,283,224.01; p2wpkh 3,612,318 / 1,924,332.48; p2wsh 91,022 / 691,417.35), perimeter faces=on, cosigners=on, fingerprint 5fd579dbc6190f5bf08e35947ab77a55e067aa366acd13bd49fa30d3ba9d0ee7 (the «locks/curve» GATE), a curve of 96 rows on the 10,000 grid. Our time: 5 h 39.

The curve is a CSV, and it is read back with a command of its own: nodsig curve deltas archive_curve_957301.csv shows it interval by interval, that is, at what rate reuse grew rather than what it grew to. nodsig curve dates pairs it with the real dates of the blocks: with the headers archive in hand it takes them from there, offline; without it, it asks the node — and in both cases that join is declared rather than smuggled in, because the timestamps are not in the curve.

Step 4 — merge: fuse the runs and seal the archive.

  • Goal: fuse the runs into a sorted archive and compute its canonical fingerprint.
  • What it is for: the sealed archive is what the check reads for the exposure capability, and what crosscheck and lookup query.
  • Requires: the archive from step 2, with the curve already derived (step 3).
  • Made of: nodsig archive merge.
nodsig archive merge --archive "$ARCHIVE" 2>&1 | tee -a ~/utxo_work/merge.log
  • Verification: it prints merged through height 957,301 and the fingerprint aacaf02dca2fc5ba8532e54fa75159041fc99051efa68eb63e59bc9537369ced (it is the «archive» GATE of the posts). Our time: 4 h 11.

Step 5 (if you used --graph) — sealing the graph.

nodsig graph fingerprint --graph "$GRAPH"
  • Verification: fingerprint a014f787256e1831c90290e04c2adbcf1fe00cfc3f2d26bb668bff491aa54190 (301,123,338,474 bytes, 1,393,498,473 tx). Our time: 5 h 13 (a re-read and nothing else, over the LAN: it is July's measurement on the same bytes — the new seal changes the fingerprint's recipe, not the reading). It is the «graph» GATE and the base of the provenance chain for index and derivatives.

The cross-check: done once, reported here.

The five steps above are the route that suffices. A second one exists — it walks the chain again comparing every revelation with the locks as it passes, building no archive at all — and it serves to answer a question it is legitimate to ask of any count: what if the total were an artifact of this code rather than a fact of the chain? We walked it once, at the same height, and the comparison asked the two routes for the same number:

TOTAL 8,784,364 locks, 5,084,725.41330985 BTC
fingerprint 5fd579db…ba9d0ee7            (the same as step 3)
CHECK PASSED: the two independent roads meet on the same fingerprint.

Our time for the comparison alone: real 258m (~4h20m); the direct scan that precedes it costs another full pass over the chain. It is the reason why this is a result to inherit and not a task to assign: whoever wants to redo it finds the commands in the repo, and how the two routes are built — and why they do not resemble each other — is in «The two routes and the unattended run», in the part on design. The limit has to be stated together with the outcome: the two routes share the block reader and the starting locks, so the check proves the way of counting, not the way of reading a block. The reading has a guarantee of its own, which also covers the single route of this chapter: hashes are recomputed while reading — the block that was asked for, the Merkle root, the hook onto the previous one — and a byte out of place stops the pass before it becomes a number.

One thing concerns even those who take only the short route: the perimeter (faces/cosigners) must be kept identical between scan and derive. The --no-faces/--no-cosigners flags exist for exploring narrower perimeters, but they have to be mirrored everywhere, or the comparison refuses.

Outpoint index and derivatives (nodsig index, nodsig derived)

This chapter is optional and sits downstream of level 3. It serves whoever wants to query the chain on three questions the archive of revelations does not cover: the history of a lock, the fee of a transaction, what was spent together with what. They are the three capabilities the address check plugs in with --index and --derived, and they are the reason why at step 2 of level 3 it is worth passing --graph.

The scheme, which is worth understanding before the commands. The index numbers the chain once and for all: every transaction and every output receives its own ordinal number, and a file whose records sit in that order does not need to repeat any key, because the n-th record sits at the n-th position and is read with a jump, without searching. The question «which output is the outpoint (txid, index)?» is answered by a single file sorted by txid; inputs are hooked to their outputs by a merge between two sorted streams, not by random access over hundreds of gigabytes. The gain is paid once and collected forever: every derivative that wants to cite an output cites its number, five bytes instead of the thirty-six of the (txid, index) pair. The derivatives then reorder the same facts by lock, by transaction and by co-spend: one expensive derivative, three questions, a linear join in place of three separate indexes.

The boundary declared elsewhere holds here too: «address» means an identical lock (same scriptPubKey fingerprint), not the wallet, and not the same key under different shapes.

Space and time, before you begin. The index takes up 248.5 GB and the derivatives 190.5 GB, which add to the 301.1 GB of the starting graph: for the complete chain reckon on about 740, to which the archive of revelations adds another ~98 if you keep it (the whole balance sheet, and the list of what can be deleted afterwards, are in the artifacts section at the end). (The tables below count in GiB, that is, the same thing in binary: 248.5 GB are 231.42 GiB. It is why further on the same index seems to shrink.) Our build took 23 h 24 for the index and 13 h 44 for the derivatives. The load is I/O, not CPU, and what decides the timings is the disk under the artifacts: measured on the same hardware and the same file, a 9p network mount reads at 14.4 MB/s, a CIFS one at 45 (75 writing), a local USB disk at 75 (97 writing). The choice of mount alone moves the timings by 3-5 times; the durations above come from the two fastest mounts. They are machine hours, not surveillance hours: both commands write checkpoints, and after an interruption you relaunch the SAME command, which resumes instead of starting over.

Keeping them up to date costs something else, and it is worth knowing before you size the disk. The artifacts are extended with the same command as the first time, but two sums change. A fusion writes the new generation before deleting the old one, so it wants roughly twice the free space of the files it is fusing: that is the price of the guarantee that an interruption leaves the previous one whole. And an append re-reads spends.bin in full, once per run and not per block, because that file is re-sorted at every fusion and a position inside it does not survive. A practical rule follows: it is better to let blocks accumulate and append them together, because one at a time pays for that pass every time.

Step 1 — index build: numbering the outpoints.

  • Goal: from the graph-v2 graph, a sealed outpoint-index-v2 index: the ordinal coordinate of every output of the chain, and next to each one its spend, already resolved.
  • What it is for: it is the only input of the derivatives, and on its own it already answers lookup, that is, the complete history of an outpoint (when it was born, what it is worth, under which lock, by whom and when it was spent).
  • Requires: the level 3 graph (step 2 with --graph, sealed at step 5). The node is no longer needed: from here on you work on files.
  • Made of: nodsig index build, then verify and stats.
nodsig index build --graph "$GRAPH" --index "$INDEX" --end 957301 \
  2>&1 | tee -a ~/utxo_work/index.log

nodsig index verify --index "$INDEX" --graph "$GRAPH"
                  # every byte against the manifest, every ladder rebuilt, the parent confirmed
nodsig index stats  --index "$INDEX"   # phase, watermark, counts, fingerprint

--end 957301 freezes the height at the published snapshot: same height, same fingerprint, on any machine. It is the replication contract, and it is also what makes the timings comparable between different environments.

  • Verification: verify re-reads everything — and rebuilds every search ladder from the file it indexes, so a ladder is checked for being RIGHT and not merely intact — then prints fingerprint verified: with the fingerprint; stats says the same things by reading the manifest, and is instantaneous. In our run:
phase: sealed   heights 1..957,301
  transactions    1,393,498,473
  outputs         3,819,356,162
  inputs seen     3,417,883,234
  spends_g0002.bin          3,417,883,234 records
  txid_index_g0001.bin      1,393,498,471 records
  overwritten txids: 2, duplicate spends: 0, unresolved: 0
fingerprint: 338c6c48f6e6c806c6d0a494bb9ca5060adcb83167c0db45328d39b40b14a69d

The last count line is the index's hygiene and is to be read, not skimmed. overwritten txids: 2 are the two twin coinbases of BIP-30, a known fact of the chain (and it is the reason the txid_index has two records fewer than the transactions: at equal txid the last one wins, which is precisely BIP-30's semantics). duplicate spends: 0 and unresolved: 0 say that every input found its output and that no output turns out to have been spent twice. An index that tolerates holes is called loose, and the derivatives refuse to be born on top of one.

Real sizes per file, 231.42 GiB in all:

file GiB
outputs.bin 99.60
txid_index_g0001.bin 51.91
txids.bin 41.53
spends_g0002.bin 31.83
tx_first_out.bin 6.49
blocks.bin 0.01
.lad ladders (3 files) 0.05

Step 2 — derived build: history, fees and co-spends.

  • Goal: three sorted files on top of the index. history_g0001.bin holds one row per output, and that row carries both events, the receipt and the spend; tx_inputs.bin gathers the inputs per transaction; fees.bin one fee per transaction.
  • What it is for: the history, fee and co-spend capabilities, both from the command line and plugged into the address check with --derived.
  • Requires: a sealed and tight index (step 1). derived build refuses an index with unresolved > 0, and checks by fingerprint that it is hooked to THAT index: a stale pairing does not pass.
  • Made of: nodsig derived build, then verify and stats.
nodsig derived build --index "$INDEX" --out "$DERIVED" \
  2>&1 | tee -a ~/utxo_work/derived.log

nodsig derived verify --derived "$DERIVED"
nodsig derived stats  --derived "$DERIVED"

The command goes through four phases (scan, merge-history, merge-inputs, seal) driven by its own state file, with no relaunching by hand. The two fusions print nothing until the final line: the silence is normal, not a hang.

  • Verification:
phase: sealed   cursors: 3,819,356,162 outputs, 3,417,883,234 spends, 1,393,498,473 txs
  fees.bin                1,393,498,473 records
  history_g0001.bin       3,819,356,162 records
  tx_inputs.bin           3,417,883,234 records
  fees total 30,047,764,560,047 sats, updated rows 0
fingerprint: 44689372f169a5c503bdf128a082c31fef767e35c77696e8e60843b42afa1c80

The rows of history are exactly as many as the outputs of the index, and they split into 3,417,883,234 spent plus 401,472,928 unspent; the distinct locks are 1,554,718,932; the total fees 300,477.64560047 BTC.

Real sizes per file, 177.46 GiB in all (the entries are rounded to the second decimal and the total is not, so adding them up comes out one hundredth short):

file GiB
history_g0001.bin 135.17
tx_inputs.bin 31.83
fees.bin 10.38
.lad ladders (2 files) 0.07

The seal does not merely add things up: before writing the fingerprint it demands a cross identity, that is, that the satoshis history gives as spent coincide with those consumed by the fee accounting. They are two independent paths that must meet on a single number; if it did not add up, the seal would stop instead of being written.

The provenance chain. The fingerprint of an artifact covers what the artifact IS — format tag, coverage, digests of the files — and nothing else: two honest builds of the same chain at the same height agree on the fingerprint, whoever made them and from whichever copy of the tools. Where it comes from is a separate question, with a separate answer: every manifest declares the parent next to the fingerprint — the derivatives name the index, the index the graph — and verify confirms the hook when it is given both. Whoever redoes the path at the same height must find the same chain again:

graph    a014f787256e1831c90290e04c2adbcf1fe00cfc3f2d26bb668bff491aa54190
  └─ index   338c6c48f6e6c806c6d0a494bb9ca5060adcb83167c0db45328d39b40b14a69d
       └─ derivatives 44689372f169a5c503bdf128a082c31fef767e35c77696e8e60843b42afa1c80

And checking costs little, in proportion. In our run: archive verify --deep 1 h 41, index verify --graph 1 h 03, headers crosscheck --index 59 minutes, derived verify --index 42. Checking everything costs about a twentieth of the construction, and that is the reason it is not the step to skip.

Going back to a height already covered (rewind). The replication contract has two routes, not one. Building with --end 957301 reaches that state from below; rewind reaches it from above, taking a sealed artifact back to a height it already covered, and into the same bytes a build that had stopped there would have written. It serves precisely those who keep the artifacts in step with the chain: to check the numbers of this manual you have to be at 957,301, and without rewind the only route would be to rebuild everything from scratch.

nodsig index   rewind --index "$INDEX" --graph "$GRAPH" --to-height 957301
nodsig derived rewind --index "$INDEX" --derived "$DERIVED"

First the index, then the derivatives, which do not choose their own coverage but follow the index: it is the same order in which a build extends them. The graph serves for one thing only, the hash of the block at that height, which the index keeps but cannot recompute.

It costs one filtering pass per file plus the seal, not a chain to be read again, and the reason lies in a single property: removing records from a sorted file leaves it sorted, so there is nothing to re-sort. The files that grow at the tail are truncated at the counts blocks.bin already knows for every height; the others are re-fused, dropping whatever sits above the cut.

That it works is not to be taken on faith here either: rewinding to 957,301 must give back the two fingerprints printed just above, the index's and the derivatives'. If they do not come back, the rewind is wrong, and the comparison says so without anyone having to trust us.

One case cannot be served, and the command stops instead of trying. If the cut falls between the two instances of a duplicate txid, that is, the twin coinbases of BIP-30 at heights 91,722/91,812 and 91,842/91,880, the older record was overwritten at build time and cannot be recovered: a rewind would make that txid disappear, whereas a rebuild would have it. Outside those two windows the problem does not exist, and after BIP-34, which puts the height in the coinbase, a new one can no longer arise.

The .lad files do not enter the fingerprint. They are ladders: one sample every few thousand keys, which spare the search a blind bisection over the big file. They are cache, not data: if they are missing, the search falls back on a direct bisection, slower and with the same answer. This is why they stay outside the computation of the fingerprint, and why an artifact built before the ladders existed remains valid.

Asking

Is your address exposed?

To know whether one of your own addresses is exposed you need neither your own public key nor any particular tool: the address and a single question are enough. Has the key that protects it already appeared in the clear on the chain?

The answer depends on the type of address.

A Taproot address (bc1p…) is exposed by construction: the address already is the public key, it is only its readable form. There is nothing to check, but it concerns the most recent type, not the most widespread ones.

The «behind a hash» addresses (1…, 3…, bc1q…) instead keep the key hidden behind a hash until it is used: the key is revealed only at the first spend, when the signature has to be attached in the clear so that nodes can validate it. For these, the check is one blunt question, «has this address ever spent?», with three outcomes:

  • receipts only, no spend: protected, the key is not on the chain yet;
  • it has spent and still holds a balance: reuse — exposed with funds still at stake, and it is the case the rest of the section teaches you to close;
  • it has spent and the balance is zero: exposed, but with nothing at stake (just do not reuse it).

How to look at it without relying on third parties. An explorer run at home (the mempool app on Umbrel, for instance) answers at once: you search for the address and look at whether it has at least one outgoing transaction. The address check automates the same criterion over a whole list. The privacy rule is one and it is not waived: your own addresses go to your own node, never to a public explorer.

Exposure belongs to the key, not to the address. It is the distinction that matters, and it carries three consequences. An address that has spent stays burnt forever: even funds arriving in the future would be born exposed, because the chain does not forget. The same key can present itself with several faces (a 1…, a bc1q…, a 3… that encapsulates it): expose the key and all its faces are exposed. The other addresses of a modern wallet stay safe instead, because each one is born from a different key, and from an exposed child key you can go back neither to the seed nor to the sisters: derivation passes through a hash, which quantum computation does not undo. The one exception is a shared xpub, which exposes a whole branch, but off-chain.

Hence the healthy behaviour, which many wallets already adopt by themselves: receive once, spend by emptying, send the change to a new address. An exposed but empty address is harmless; the case to avoid is reuse, that is, going back to receive on an address that has already spent.

«If I move everything to a new address, am I protected?» Yes, on three conditions. The destination must be of the «behind a hash» type (bc1q…, 1…) and never used before, not a Taproot, which is born with the key in plain view and is therefore the wrong type for this purpose. The protection lasts as long as the behaviour stays healthy, with no subsequent reuse and no xpub going around. And during the transaction itself the keys of the inputs stay visible for a few minutes in the mempool: today that is immaterial, and it is the reason the move should be read as prevention, to be made before machines exist that are capable of exploiting that window.

Why every spend exposes, and receiving does not. The signature of a transaction is verified with the public key, so the transaction carries it along in the clear: otherwise nodes could not validate it. Receiving reveals nothing, except in the Taproot case. And the revelation does not go towards a single node: the transaction propagates to the whole network in seconds, anyone listening sees it, not only whoever mines. Confirmation in a block does not close the key back up, and it stays there forever: the danger ends not because the key becomes secret again, but because the coin has moved.

A partial spend does not save you. A transaction moves the individual coins it chooses, not «the address»: funds left on an address that has already spent stay exposed permanently, even with no new receipts. The compact rule is not «do not reuse», it is «when you spend, empty»: receive once, spend in full, send the change to a new address. Many wallets do this automatically, but it is not guaranteed everywhere, and the answer «it has spent and holds a balance: exposed» covers exactly this case.

An outcome that surprises. An address can turn out to be exposed even without ever having spent on its own account, if its key appeared inside a script revealed by somebody else: a cosigner of a multi-signature script that has spent, for example. The answer reads the chain, not expectations, and this is why it is worth checking instead of deducing.

The xpub case. A seed does not have one xpub but one per derivation branch (the one usually exported is the first account of the branch in use). A leaked xpub compromises that sub-branch, not the seed: upwards it is protected by hash-based derivation, sideways by the separation between accounts. In theory moving to a new account of the same seed is enough; in practice, if you do not know for certain what the service saw, a new seed is the answer without asterisks. The scale of the problem, for now, is one of privacy more than of funds: it becomes about funds only the day capable machines exist, and the «harvest now, decrypt later» logic applies (what the service saw, it saw forever). The move is not urgent, but it is better done early, within the same wide calendar as the migration.

What stays invisible. Off-chain exposure, like the xpub above, leaves no trace anywhere. And for multi-signature addresses the inner script stays hidden until the spend: «protected», in that case, speaks of the hash, not of who could spend behind it.

The address check (nodsig check)

Given a list of addresses, it says for each one what follows from the sources you possess: whether the key that protects it has already appeared on the chain, how much sits behind it now, what came in and went out, and which other coins it was spent with. Every answer comes from a distinct capability, and the set of capabilities is made to grow: when there is no source to answer with, the answer is «undetermined» with the reason written out, never a silence and never a guess.

It automates the criterion of the section above over a whole list, your own historical wallet for instance, querying only your own infrastructure.

Why an index is needed, and why Core does not have one. Bitcoin Core cannot answer «has this address ever spent?», and that is not a shortcoming: the node's job is to validate, and validation never speaks of addresses. It only asks whether a coin exists, whether it is still unspent and whether the unlocking data satisfies its lock, questions for which the UTXO set suffices, indexed by coin and not by owner. Strictly speaking, addresses do not even exist on the chain: they are a readable encoding of certain script patterns.

The complete history is there, but filed in order of arrival, like a library catalogued by shelf: to answer «by address» it has to be read through once and a separate catalogue has to be built. It is what Electrs or a self-hosted mempool do, and it is what we do here, because an index is nothing but a scan done once and filed: a cost paid once and amortised over every subsequent question. Anyone can build their own, keeping whatever serves their purpose. Indexers sit outside the protocol, and you can see it from here: were they all to disappear, not one block would change validity.

An interface by capability. An answer is assembled by querying distinct capabilities, each with its own backend, each replaceable without touching the others:

Capability The question Where it answers from today
exposure have the key or the script behind this address ever been revealed on the chain? our archive of revelations
balance how many satoshis sit there right now? scantxoutset on the node, which accepts several descriptors in a single call: one scan for the whole list, not one per address (on a Raspberry Pi that is the difference between minutes and hours)
history which coins came in, which went out, what is left? outpoint index and derivatives
co-spends what was spent together with its coins? the same two artifacts

The default we publish is not the only route: Electrs, or Fulcrum, remains a backend of equal standing for those who prefer to run a ready-made indexer on their own node. That is exactly what the interface is for, and its implementation is on the development plan, with the instructions to set it up.

Three properties make it a virtue and not scaffolding. A missing backend does not pretend: the capability degrades to «undetermined» with the reason written out, never to a false «protected», which is the same rule with which the archive treats absence. Silence and «no» are different answers, and confusing them is how tools deceive people about their own money. Adding one is a single class registered in a single place, and nothing else changes. The defaults are a declared choice, and the flags exist so that a third party can explore others, in the same spirit as the scanner's perimeter flags. The code, in short, does not know where an answer comes from: it only knows which question it is asking. The node, in particular, is contacted ONLY if you pass --rpc; without it, no balance and no calls, a property that also serves the test suite, which runs without a node.

File by default. The privacy rule is in the code, not in the recommendations: the report goes to check-results.txt (or --out PATH) with the sensitivity warning at the top, because a screen gets shared and a terminal gets recorded; on screen only with an explicit --stdout, meant for public fixtures and for piping. On stderr only the pointer to the file remains, never an answer.

Decoding addresses is pure arithmetic, with no network (base58check for 1… and 3…, bech32 and bech32m for bc1…, from the BIP-173 and BIP-350 specs). The test suite ties it to known public vectors and cross-checks it with a mirror encoder written in the test, which also serves to manufacture addresses whose digests match the synthetic archive: the whole exposure proof runs on public data, without a single real address.

Using the check — the commands

Goal — given a list of addresses, a report for EACH one: type, exposure answer, and (if you plug in the backends) balance, history and co-spends. It comes out in check-results.txt (plus --csv if you ask for it).

What it is for — answering «are MY addresses exposed?» by querying only your own node and your own files, without giving the list to any third party.

Requires — the artifacts of the capabilities you want to plug in: the archive of revelations for exposure, index and derivatives for history and co-spends. The archive comes from the level 3 pipeline; index and derivatives from the chapter that follows it, «Outpoint index and derivatives». The repo does not publish these artifacts, which are large: whoever wants them rebuilds them, and the fingerprints say whether the rebuild is right. A check without an archive does not lie, it degrades honestly to «undetermined». The node is needed only for the balance, and it is contacted only with --rpc — and whoever asks for it makes the node acquainted with the list of addresses: it is the reason that flag is used with YOUR OWN node and with nobody else's.

Commands. The minimal answer needs nothing — it decodes the type and recognises Taproot straight away, exposed by construction:

# decoding only: bc1p… → EXPOSED by construction; the hashes → undetermined
nodsig check --file list.txt
# → writes check-results.txt, created readable by its owner alone (0600):
#   the file lists YOUR addresses, treat it as sensitive

It is worth running it like this, empty-handed, before anything else, perhaps on a single address passed directly (nodsig check --stdout <address>, with --stdout printing on screen instead of writing the file: fine for a public address, not for your own). With no artifacts the tool settles what the encoding alone settles, answers UNDETERMINED for everything else, names which flag would enable each capability it is missing, and prints the perimeter of what any answer can mean. It is the behaviour worth seeing from a program before entrusting it with a question that matters.

The full answer, plugging in the backends you have. The node's credentials are passed with --cookie-file: no command accepts a secret on the command line, because a process's argv is readable by anyone on the machine (ps, pgrep). --csv for further processing; --stdout only for public fixtures or for piping:

nodsig check --file list.txt \
  --archive  "$ARCHIVE" \
  --index    "$INDEX" --derived "$DERIVED" \
  --rpc http://127.0.0.1:8332 --cookie-file "$COOKIE" \
  --out check-results.txt --csv check-results.csv

Verification — the shape of the report. At the top, one # line per plugged-in capability: it says which artifact answered (by format, not by path), up to which height, and under which fingerprint. It is the declared perimeter of every answer, and it is also what makes the report shareable: it does not describe the machine that produced it. An artifact not yet sealed declares as much instead of keeping quiet, and the node, which cannot have a seal, presents itself with its height. Then, per address:

<address>
    <type>: <answer>          # EXPOSED (by construction) | EXPOSED (by reuse)
    <detail>                  #   | PROTECTED until first spend | UNDETERMINED
    balance: <N> sats         # only with --rpc; «… but empty: nothing at stake» if 0
    history: received … spent … unspent … (up to height …)
    co-inputs: spent in … tx(s), with … outputs of … other lock(s) …

and at the end, the caveats the script prints by itself (the perimeter of EVERY answer: off-chain exposure invisible, P2SH/P2WSH scripts unknown until the spend, mempool = key already revealed). The EXPOSED (by reuse) answer also says WHERE the key appeared (scriptSig / witness / inside a revealed script — this last one is cosigner exposure, not necessarily by your own hand) and — from the archive, which records the height of the first appearance — since WHEN. The text of the answers is in English like all the code. The co-inputs line ALWAYS carries the common-input/CoinJoin caveat with it: adjacency among inputs is a hint, never a certainty.

A real report. This is the output captured on three well-known public addresses, with the archive, the index and the derivatives plugged in and without --rpc (no balance line: the balance is the only thing that asks the node). They are textbook addresses in the literal sense: the vanity 3P14159… and the two examples from the BIP-173 and BIP-350 standards. The text is reported exactly as it came, with nothing cut:

# exposure: reveal-archive-v2 (confirmed blocks 1..957,301, sealed aacaf02d…9ced)
# history: outpoint-derived-v2 (confirmed blocks 1..957,301, sealed 44689372…1c80)
# co-inputs: outpoint-derived-v2 (confirmed blocks 1..957,301, sealed 44689372…1c80)

3P14159f73E4gFr7JterCCQh9QjiTjiZrG
    p2sh: EXPOSED (by reuse)
    script revealed by a spend (1 key inside, co-signer exposure counts), first seen at height 184,727
    history: received 9× 0.06212337 BTC, spent 7× 0.06002337 BTC, unspent 2× 0.00210000 BTC (heights 183,082–521,596, index at 957,301)
    co-inputs: spent in 7 tx(s), co-spent with 87 output(s) under 31 other lock(s) — common-input HINT, not ownership proof (CoinJoin breaks the assumption)
bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4
    p2wpkh: EXPOSED (by reuse)
    key seen in a scriptSig; key seen in a witness; seen inside a revealed script (co-signer exposure counts), first seen at height 215,929
    history: received 92× 0.01407024 BTC, spent 92× 0.01407024 BTC, unspent 0× 0.00000000 BTC (heights 511,207–957,250, index at 957,301)
    co-inputs: spent in 80 tx(s), co-spent with 35 output(s) under 18 other lock(s) — common-input HINT, not ownership proof (CoinJoin breaks the assumption)
bc1p0xlxvlhemja6c4dqv22uapctqupfhlxm9h8z3k2e72q4k9hcz7vqzk5jj0
    p2tr: EXPOSED (by construction)
    taproot (bc1p…): the program IS the key
    history: received 14× 0.00105034 BTC, spent 14× 0.00105034 BTC, unspent 0× 0.00000000 BTC (heights 700,829–918,525, index at 957,301)
    co-inputs: spent in 5 tx(s), co-spent with 0 output(s) under 0 other lock(s) — common-input HINT, not ownership proof (CoinJoin breaks the assumption)

caveats (the perimeter of every answer above):
- off-chain exposure is invisible here: an xpub shared with a service
  exposes descendant keys without any on-chain trace;
- a P2SH/P2WSH address hides its script until it spends: "protected"
  speaks of the hash, not of who could spend behind it;
- perimeter is CONFIRMED blocks up to the stated heights: a spend
  sitting in the mempool has already revealed its keys.

How to read it. The three addresses show three different routes to the same answer. The P2SH is exposed because its script was revealed by spending: before that spend it would have read PROTECTED, and the line says how many keys the script contained (here one) and since when they are in view — that count comes from the spend that revealed the script, not from the hash, which on its own says nothing. The P2WPKH is exposed three times over, and the third line counts: the key was seen inside a revealed script, that is, by a cosigner, and not necessarily by the hand of whoever owns this address. The Taproot needed no archive at all: its exposure is read off the string, because the program IS the key.

They are three public and very busy addresses, and the report says so without mincing words: whoever checks their own will nearly always find it shorter and duller than this. The proof on the three is in any case the only one that can be shown in full without asking anyone to publish their addresses.

The timing: ~2 seconds for the three, with every backend plugged in and the artifacts on a local disk (over the LAN it was 13: the mount rule holds here too).

Querying the index and the derivatives

Four commands, and they ask only for the files: no network, no node.

nodsig index   lookup   --index "$INDEX" TXID:VOUT
nodsig derived fee      --index "$INDEX" --derived "$DERIVED" TXID
nodsig derived cospends --index "$INDEX" --derived "$DERIVED" TXID
nodsig derived history  --index "$INDEX" --derived "$DERIVED" --lock HASH160

history alternatively accepts --spk with the raw scriptPubKey in hex, and computes the fingerprint itself: it is the convenient route when you start from a decoded address instead of a lock already in hand.

A lookup on the two outputs of the first transaction between people, in block 170, with the chain read from the files alone:

f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16:0
  created  height 170 (2009-01-12 03:30 UTC)
  value    10.00000000 BTC (1,000,000,000 sat)
  lock     hash160(scriptPubKey) a8cd299d425fe2f9e3ebde46abc94201ce6f579b
  spent    height 92,240 (2010-11-16 20:39 UTC) by ea44e97271691990157559d0bdd9959e02790c34db6c006d779e82fa5aee708e
f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16:1
  created  height 170 (2009-01-12 03:30 UTC)
  value    40.00000000 BTC (4,000,000,000 sat)
  lock     hash160(scriptPubKey) e01507f88b6dcc026c7062029c03adb11553de10
  spent    height 181 (2009-01-12 06:02 UTC) by a16f3ce4dd5deb92d98ef5cf8afeaf0775ebca408f708b2146c4fb42b41e14be

The 50 BTC of the input come out split into 10 and 40, and the fee is zero: the change went back to the payer's lock, as was the custom then. It is also the shortest way to check that the index answers: these are numbers anyone can compare with an explorer.

Another check with a value known outside our files, the fee of the transaction for the two pizzas:

$ nodsig derived fee --index "$INDEX" --derived "$DERIVED" \
    a1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d
a1075db55d416d3ca199f55b6084e2115b9345e16c5cf302fc80e9d5fbf5d48d: fee 99,000,000 sat (0.99000000 BTC)

The same transaction, asked of cospends, shows what the third derivative is for: 131 outputs spent at the same moment, 111 of them of exactly 0.01 BTC, and all 131 under a single lock. It is the typical shape of a wallet scraping the bottom to put a sum together. It is worth repeating the caveat the command prints by itself: adjacency among inputs is a hint about common ownership, never a proof, and CoinJoin exists precisely to break it. Here the hint is not even needed, because the lock is one alone: the case in which a co-spend says something is the one in which the locks are different, and it is also the one in which it can be wrong.

What a question costs. On our files, reached over the LAN: about 11 seconds for the first cold read, ~7 seconds for a short history, ~13 seconds for the check of three addresses with the three local backends. With the balance switched on (--rpc) you go up to a minute, but that time belongs to the node, not to the files: scantxoutset walks the entire UTXO set. There is however one case that costs much more, and it is worth knowing before meeting it: heavily reused locks. The lock of the genesis block has 75,454 outputs for 57.22278291 BTC, and its history takes about 18 minutes, because for every row the climb back to the height is an access of its own. It is not an error in the result, it is the shape of the query: a lock with tens of thousands of events deserves a climb done in batches, which today is not there.

The design and the formats

Below is the why: how the level 3 pass is conceived, how the files it produces are made, and the map of every artifact. It serves whoever wants to understand or reimplement, not whoever merely runs the steps.

One thing this chapter does not do: duplicate the repo. The normative format specifications (docs/formats/), the interface contracts (docs/contracts/), the architecture (docs/ARCHITECTURE.md) and the complete map of every file (docs/ARTIFACTS.md) live there, next to the code they describe, and they stay aligned even when the code moves. What stays here are the ideas, the choices with their why, and the experience of our own run: enough to understand the whole flow without opening the repo — which is needed only by those who want the bytes.

The three ideas of level 3

The steps are run without knowing any of this; here is why they work. Goal: to reproduce for oneself the class of «>34%» estimates of BIP-361, that is, to count the current coins on locks already opened in the past. Not a statistical sample: an exact count at a declared perimeter, reproducible bit for bit at the same height. Three ideas hold it up.

Idea 1 — invert the comparison. The naive way would call for an archive of ALL the keys ever revealed in history (billions of entries). But what matters is only which of the locks CURRENTLY in the UTXO set have already been opened: from the level 2 snapshot you distil the set of hashes of the current «behind hash» locks and then scan the history, checking every revelation against that set and keeping only the hits. Small memory, a single pass.

It is worth looking at how much the set shrinks, because the number that comes out of it recurs everywhere afterwards. The P2PKH/P2WPKH/P2SH/P2WSH entries in the UTXO set are ~109M (add them up from the census table), but they sit behind 51.7 million DISTINCT locks: several coins under the same lock are reuse seen from the UTXO set's side, and the target set needs each lock only once. Distilled and sorted, those 51.7M fit into about 1 GB, which is held in a sorted buffer with binary search: it is the prepare of step 1.

Idea 2 — the spent lock is reconstructed from the unlocking data, with no indexes. An input refers to the spent coin as (txid, position), it does not show the lock; getting back to it with a lookup would require an index of the history. But for the standard types the unlocking data already contains what it takes to RECOMPUTE it: P2PKH → hash160 of the pubkey in the scriptSig; P2WPKH → hash160 of the pubkey in the witness; P2SH → hash160 of the redeem script (last push of the scriptSig); P2WSH → sha256 of the witness script (last element of the witness). Reading the inputs is enough. P2PK, P2TR and bare multisig are not needed here: they are already counted as exposed by construction.

At the spend, however, the type is not really resolved: the scan treats the last push as a candidate (redeem/witness script, or pubkey) and verifies its hash against the partitions, without having to know in advance what type the spent coin was. The candidate mechanism — and the expected noise of the «malformed inner scripts», which is its consequence — is detailed in «The incremental filing cabinet».

Watch the direction, because this is where naive intuition trips: you are NOT looking for whether the lock «has appeared» on the chain. Appearing is its trade — the hash is the destination of payments, it appears every time it receives, and it reveals nothing. What you are looking for is whether its SECRET has ever been shown: it is at the spend that the protocol forces you to exhibit the preimage («here is the key whose hash is the lock, and here is the signature»), and from that moment the key is public forever. So the scan walks the unlocking data, reconstructs the revealed candidates, re-hashes them and asks the list: does this REVELATION open a lock that today still holds satoshis? Every yes lights a bit that never goes out again.

The construction of the bitmap, in a figure (the file of locks is the table; the bitmap is its yes/no column, detached and paired by position):

HISTORY (blocks in streaming)        SNAPSHOT (the files of the prepare)
every input carries its
unlocking data                       locks_p2pkh.bin (sorted)
(scriptSig / witness)                row 0    digest | satoshi
     │                               row 1    digest | satoshi
     │ rebuild the candidates        row 2    digest | satoshi
     ▼                               …        (binary search +
key · redeem · witness script        index of the first 3 bytes)
     │ re-hash (hash160 / sha256)              │
     ▼                                         │
«is this digest a row?» ─────────── lookup ────┘
     │
     │ yes, at row i
     ▼
hits_p2pkh.bin — a FLAT sequence of bits, one per row
position    0 1 2 3 4 5 6 7 8 …
bit         0 1 0 0 1 0 0 0 0 …     → bit i = 1, forever
(on disk packed 8 per byte: bit i → byte i//8, pos. i%8;
 21,267,062 p2pkh rows → 2,658,383 bytes exactly)

What the lock holds, and how its type is read. In the UTXO there is no key: the «behind hash» lock holds a digest — 20 bytes (hash160) for P2PKH/P2SH/P2WPKH, 32 bytes (sha256) for P2WSH — plus the amount. The type is NOT deduced from the size: it is read from the opcode template of the scriptPubKey (P2PKH 76a914…88ac, P2SH a914…87, P2WPKH 0014…, P2WSH 0020…). That size is not enough is shown by Taproot: 5120…, 34 bytes exactly like P2WSH, different only in the version opcode (51/OP_1 against 00/OP_0). And it is the template that says both which digest length to expect AND which hash function to use on the revealed preimage. In the filing cabinet the type is not even written down: it is the partition (one file per type) that encodes it.

What exactly is compared — and the two levels of the «script» case. The comparison is always digest against digest, never shape against shape: you re-hash the preimage exhibited at a spend (not what appears at a receipt) and look for that digest among the locks still full today. Comparing the digest and not the shape is what catches the faces: the same key under 1… (P2PKH) and bc1q… (P2WPKH) has the same hash160 — different shapes, identical digest: looking for the shape you would miss them, looking for the digest you take both. For scripts, then, the comparisons are TWO, and they open different locks:

  • outer level: the digest of the revealed script (hash160 of the redeem for P2SH, sha256 of the witness for P2WSH) against the P2SH/P2WSH locks — a match says that that script-lock has been opened, and it burns the coin behind it;
  • inner level (cosigners): every pubkey EXTRACTED from the revealed script, re-hashed to hash160, against the single-sig locks — a match burns the ordinary face of that cosigner (1…/bc1q…), even if they never spent on their own account. It is the criterion of the «cosigners» perimeter: without it, only script-locks are counted; with it, also the single-sig faces those scripts expose.

An example with the real bytes (redoable by hand). A real, public public key — the one from the genesis coinbase — and its «behind a hash» face:

pubkey (65 B)   04678afdb0…6bf11d5f
hash160(pubkey) 62e907b15cbf27d5425399ebf6f0fb50ebb88f18
→ P2PKH lock    1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa

The hash160 of the key IS the digest written in the P2PKH lock. Now the SAME key inside a script (a 1-of-1 multisig — a single cosigner, to keep the example to one key: OP_1 <pubkey> OP_1 OP_CHECKMULTISIG):

redeem script   514104678afd…11d5f51ae
hash160(script) a7086d18c683c4df2897a557b7446e2eb275f64a
                → P2SH lock   3GvCs3EGuPTH29CQY7Kuj1w6HUDRjsWg5Y
sha256(script)  3e3baf2f87a3f725e48beb9484493be9bf5934af1f43715f10b310b0b767bbe9
                → P2WSH lock  bc1q8ca67tu850mjteytaw2ggjfmaxl4jd90raphzhcskvgtpdm8h05slrfwct

At the spend that reveals this script the scan makes the two comparisons:

  1. it re-hashes the script (hash160 → a7086d18…, or sha256 → 3e3baf2f… for the witness version) and looks for it among the P2SH/P2WSH locks: a match → the script-coin is burnt;
  2. it extracts the pubkey and re-hashes it (hash160 → 62e907b1…), looking for it among the single-sig locks: a match → the 1A1zP1eP… face is burnt too, even though nobody has ever spent on that address.

A single real key, three different locks (P2PKH, P2SH, P2WSH), a single criterion: re-hash what the spend revealed, look for the digest among the current locks. Every value above can be redone with RIPEMD160(SHA256(·)) for the hash160s and SHA256(·) for the P2WSH — it is the operational definition of the «match» on which all the reuse counts rest.

Idea 3 — every partial scan is a valid lower bound. An address is burnt by its FIRST spend: every block read can only add burnt ones, never take any away. An interrupted run therefore produces an honest number («40% of the history read, reuse is AT LEAST X BTC») which grows monotonically towards the complete value: the floor RISES as the scan proceeds. The run can be broken up by height intervals and resumed; the curve (BTC burnt vs share of history read) is publishable and is itself didactic.

Perimeter to declare (the choices, each one switchable in the code):

  • base criterion: a current lock whose identical hash has already been spent = «has ever spent», the criterion of «Is your address exposed?», at chain scale;
  • faces extension: every revealed pubkey also burns the other faces of the same key (same hash160 for 1…/bc1q…; the encapsulated 3… face = hash160 of the script 0014<hash160>);
  • cosigners extension: the pubkeys inside the revealed redeem/witness scripts burn their respective single-sig faces;
  • declared exclusions (invisible to ANY scan of the blocks): keys seen only in the mempool and never confirmed; xpubs off-chain; P2SH/P2WSH never spent (unknown). The «understated» character of the count survives level 3 too.

Architecture and reproducibility: identical in spirit to level 2: the node extracts, the PC processes. Raw blocks over RPC on the LAN (getblockhash + getblock <hash> 0, hex → bytes), OUR OWN block/transaction parser in Python (more demanding than the snapshot parser, but a stable and documented format; tested with synthetic blocks from a mirror implementation + known real blocks). Cut at the SAME height as the level 2 snapshot (today 957,301): the utxos.dat file already in hand serves twice, for the set of hashes and, downstream, to sum the BTC of the burnt ones by type and epoch with a variant of the census. Same height + same perimeter = same bytes for anyone. Orders of magnitude: ~680 GB of blocks, ~3.5 billion inputs; bottlenecks are reading on the Pi and Python parsing on the PC; estimate hours-to-days, in tmux, unattended.

The two routes and the unattended run

Operational goal: a run that, once started, goes by itself for days and survives interruptions (of the network, of the PC, of the Pi) without losing work or requiring supervision. The manual's rule holds: the node extracts and nothing else, all the processing sits on the PC.

The two routes and their meeting, in a figure:

              Bitcoin node (RPC over an SSH tunnel)
                raw blocks, integrity recomputed
                │                     │
   direct route │                     │   archive route
        (reuse) ▼                     ▼  (archive)
compares each revelation        archives EVERY revelation
with the locks at once,         with its provenance
keeps only the lit bits         (sorted runs on disk,
(bitmap, ~6 MB)                 tens of GB)
     │                                │
     │                                ▼ merge (periodic fusion)
     │                          fused, canonical archive
     │                                │
     │                                ▼ crosscheck: re-reads
     │                          the archive AGAINST the locks
     │                          and rebuilds the bitmaps by an
     │                          independent route
     ▼                                ▼
sha256 fingerprint ══ must coincide ══ sha256 fingerprint
          (the cross-check of level 3)

(--graph, optional on BOTH routes: the graph is co-emitted
 on the side and touches neither bitmap nor archive)

The direct route has its own consistency checks, with the perimeter said right: reuse stats recomputes the fingerprint from the bitmaps on disk and compares it with the one recorded in the checkpoint BEFORE reporting any number — no node, no new scan; if it does not match, it fails instead of producing statistics on bitmaps that do not correspond. And it attests that the hit bitmaps are those, not that the starting set of locks is right: that is attested by the per-type digests in the locks' manifest. The two things together cover the whole chain of the count; one alone does not.

The incremental filing cabinet: appendable by construction

The underlying idea: the archive of revelations does not stay a one-off scan but is born APPENDABLE (sorted runs + height watermark + periodic fusion), so that at every new block you process that block, you do not rescan the history. It grows in HEIGHT (new blocks) and in WIDTH (every question = a derivative with its own reconstruction rule and its own fingerprint). It is not «a mempool of our own»: only confirmed blocks, on purpose (the mempool is not consensus data, it would be irreproducible).

Transport and efficiency. Two ways to have the blocks delivered, and the choice looks at the bytes, never at trust: integrity does not depend on the transport, because every byte is hash-verified anyway (just below). Over JSON-RPC the requests go in batches (25-50 getblock <hash> 0 per HTTP call) to amortise latency, but the blocks travel in hexadecimal: twice the bytes (~1.4 TB for ~680 GB of blocks, sustainable on a LAN in hours). Over --rest (see the Prerequisites) the blocks arrive in binary, half the traffic and no credentials on the wire, with no batching but with --prefetch-depth keeping more requests in flight. The hashes by height (getblockhash) are taken over RPC in batches at the start either way, once and for all, up to the cut-off height. The two transports are proven to produce byte-identical artifacts.

Integrity without trust in the transport. Every block received is verified: the recomputed hash of the header (sha256d) must match the hash requested, and the prevhash of every header must hook onto the previous one: the chain self-certifies as you read it, the tunnel is not to be taken on faith. An important clarification, which emerged from testing the parser: header + Merkle of the txids do NOT cover the witness bytes, because the txid excludes the witness by construction (BIP 141), and it is precisely in the witness that most of the revealed keys counted by level 3 live. The protocol closes the gap elsewhere: the witness commitment in the coinbase (a second Merkle, over the wtxids, committed in an OP_RETURN output marked aa21a9ed). Our parser verifies that too: every byte handed to the scan, witness included, traces back to the hash of the block. The first sketch of the parser did not do this, and the corruption test (a byte altered in the witness passed clean) exposed it: it is the mirror test doing its job.

Checkpoint and resumption (the heart of the «unattended»). State on disk at every completed interval (e.g. every 10,000 blocks): last closed height + the set of hits in append-only files + partial counts. On restart you resume from the last checkpoint. The monotonicity of the lower bound makes every checkpoint an already valid, publishable result («history read up to height H, reuse is at least X»): the curve is made of the checkpoints themselves. Log with progress, blocks/s and arrival estimate; at the end of the run, a summary with the perimeter exclusions printed out (the tool declares by itself what it does not see).

The trades. The code sits in the nodsig package, one module per trade. Here the trades are listed by command, not by file name: commands are public surface and do not change within a major version, while the names of the internal modules carry no such protection and have already moved once. Whoever wants to see how they are split up today opens the repo, which is the place where that information stays true.

Trade Where it shows
reading blocks and transactions (header, legacy and segwit, scriptSig and witness) while verifying what is read: header hash, Merkle of the txids, witness commitment, no trailing bytes internal, under every command that touches the chain
the direct route: distilling the locks from the snapshot, then walking the chain keeping only the bitmaps of the hits reuse prepare, reuse scan, reuse stats
the archive route: recording, fusing and sealing, making the two routes meet, querying archive scan, archive merge, archive derive, archive crosscheck, archive lookup
re-reading the graph co-emitted by the passes with --graph graph fingerprint, graph stats, graph show
numbering the chain and querying it by outpoint index build, index verify, index stats, index lookup, index rewind
the three sorted files on top of the index, and their reads derived build, derived history, derived fee, derived cospends
the arithmetic of distributions, shared, so that statistics mean the same thing everywhere internal, under reuse stats and curve deltas
the test suites: synthetic chains and snapshots written by independent mirror implementations, plus known real blocks as public fixtures python3 -m pytest

One single design choice has to be mentioned here, because it explains a number you see in the output and not an internal detail: extraction works by candidates, not by classification. Every plausible revelation is gathered and the set of locks decides, not a judgement made upstream about the input. A false positive would require a preimage of hash160, that is, it does not happen; a false negative ends up in a declared counter. The lower bound, therefore, never inflates: at worst it stays lower than the truth.

The rest — subcommands, byte formats, invariants, testing strategy — is documented in the repo, in the module docstrings and under docs/. That is the canonical text, and this manual does not keep a copy of it that would age on its own account.

What it costs, measured on the real channel. Before the long run we did a pilot on the first 50,000 blocks, to try the mechanics and not the numbers: prepare on the snapshot in 17 minutes (51.7M behind-hash locks: 21.3M p2pkh, 6.2M p2sh, 23.1M p2wpkh, 1.2M p2wsh), scan at ~380 blocks/s over the tunnel. The two tests that mattered more than the speed: SIGKILL to the live process after checkpoint 30,000 and resumption, with a final fingerprint identical to the clean run; and the first cross-check on real data, PASSED, the same fingerprint a00b230a… reached by the two routes.

On the whole chain, probe measurements on the real channel: fetch+parse pipeline ~6.7 MB/s over the tunnel (fetch ~11 MB/s, parse with full integrity ~16 MB/s, in series); chain ~767 GB estimated from samples per epoch (the node's size_on_disk says 859 GB, but that includes the undo files). One pass is therefore ~32-36 hours of pure measurement, realistically 1.5-2 days with extraction and lookups; the two we made ourselves, 3-4 days, ~1.5 TB of hexadecimal traffic each (the definitive re-run of the archive route, with the three co-emitted artifacts, closed in 58 h 47). The checkpoint every 10,000 blocks limits the loss from a crash to ~40 minutes of work, proved with the pilot's SIGKILL. A prefetch layer overlaps fetch and parse and pushes throughput towards the ceiling of the fetch; it is switched off with --no-prefetch.

The direct route (reuse scan), at the cut at height 957,301. Reuse at the full perimeter (faces=on, cosigners=on): 8,784,364 reused locks over 5,084,725.41 BTC (p2pkh 3,925,128 / 1,185,751.57; p2sh 1,155,896 / 1,283,224.01; p2wpkh 3,612,318 / 1,924,332.48; p2wsh 91,022 / 691,417.35). Fingerprint of the hits 5fd579dbc6190f5bf08e35947ab77a55e067aa366acd13bd49fa30d3ba9d0ee7. malformed scriptSigs: 0; malformed inner scripts: 2,918,749,560. The big number is candidate extraction declaring itself: it is the re-reading as a script of the last push of every ordinary single-sig spend (a pubkey in P2PKH/P2WPKH, or a DER signature in the old P2PK), which does not parse because a script is not what it is. The outer hash of the candidate is computed and compared with the locks anyway, and reuse grows only on a match. The zero of the malformed scriptSigs says that every scriptSig decomposed into regular pushes; the big number lives one level further in, where by design one over-collects.

Freshness of the data. The numbers are at the cut 957,301, and the date is declared as always: the value of the count is methodological and about orders of magnitude, a few weeks move nothing material. Future freshness is not bought by redoing the round, but with the incremental updates of the filing cabinet: a new snapshot, a single append pass from the new blocks, derive against the fresh locks. Whoever wants the strongest proof runs this manual to the letter at the same height 957,301 and reproduces the published fingerprints TO THE BYTE.

The co-emitted graph

The decision: during the complete run the RAW records of the transaction graph are emitted as well — for every block, which coins each transaction creates (amount + lock) and which coins it consumes (the references to previous outputs). The expensive part of the reading (fetch + parse with full integrity) is paid for by the scan anyway: emission only adds local sequential writes; redoing it afterwards would cost another full pass over the chain. It is the raw material of future questions (payment history of an address, common-input clustering, the sentinel on spends from doors-without-a-key), which the query model of Electrs — one scripthash at a time, online, without amounts — cannot serve.

What it is, and what it is NOT. It is not an index: the records stay as faithful as possible to the block, in chain order, minimal transformation (a stupid emitter has a small bug surface). Every future index (by address, by cluster, by script class) is a DERIVATIVE: it is built from the records with its own reconstruction rule and its own fingerprint, as per the filing cabinet (growth in width). They stay OUT of the format, with the reason on record: scriptSig and witness (they are the REVELATIONS, their home is the reveal archive — the graph is who pays whom, under which lock); version/locktime/sequence (consensus bookkeeping, not flow); the fees (they are a JOIN of these same records, keeping them would be transformation, not fidelity). The first derivative already exists and serves as a miniature example of the scheme: nodsig blockstats distils from the graph one row per block — transactions, edges, tiles, satoshis put into new outputs — into a CSV sorted by height, with its replication contract in its meta.json (the fingerprint of the starting graph, the height, the rows): rebuilt from the same graph at the same height, it is byte-identical. With the honest boundary declared: the satoshis are movement, not issuance and not balance, and the fees are not there — they are the join the outpoint index pays for once and for all.

The same disciplines as the rest. Runs that tile height intervals + watermark in the state + sha256 per file (the graph is not taken on faith when re-read, as blocks were not taken on faith at fetch); runs orphaned by a crash deleted on resumption; CANONICAL FINGERPRINT = sha256 of the stream concatenated in order of height, blind to the boundaries of the runs (artifacts of buffering, not data): an interrupted and resumed run = the same bytes as the single run, whoever re-emits at the same height must land on the same string (the twin, for the graph, of the muhash and of the scanner and archive fingerprints). graph fingerprint re-reads everything and doubles as an integrity audit; graph digest makes the same comparison WITHOUT rewriting anything, interval by interval, and it is the way a rescan checks that it is still emitting the same bytes as a graph that already exists, without paying its ~300 GB again. The normative spec of the format — records, ordering, canonical form — lives in the repo, docs/formats/Graph-v2.md.

Where it lives, and on what terms. Emission is OFF by default (--graph): whoever clones the tools can redo the frugal count without paying the ~300 GB. The path is free and the write profile — large sequential appends, files written tmp-then-rename — is made precisely to hold up on a network volume too: the graph is born already next to its backup. Honest implications: if the volume hiccups the run dies and resumes from the checkpoint (a discipline proved with the pilot's SIGKILL); the final fingerprint re-reads everything, and at full size that is hours.

What makes these files «everybody's»

The formats of the files are documented in the sections above; here is the criterion that unifies them, because it is the criterion (not the details) that whoever wants to build verifiable tools takes away.

Inside the bytes there is nothing of our own run. In the binary files (locks, bitmaps of the hits, runs of the graph, runs of the archive) there is no time of ours, no machine name, no order of discovery: only data the chain dictates to anyone (the time that appears in the graph is the one from the block header, declared by the miner, not the hour of the scan). Everything that is biography of the run — which files it produced, with what sha256, how many transactions it went through — lives separately, in the state.json logbook. The rule in one line: THE DATA BELONGS TO THE CHAIN, THE LOGBOOK TO US. It is also the reason of principle for the per-run statistics of the graph (above, the operational one is given, clean recovery from a crash): a global counter inside the state would be a function of the history of the interruptions, that is, it would contaminate with the biography of the run an archive that must depend only on the chain.

The replicability guarantee lives on the canonical form, not on the individual files. The boundaries between the run files are artifacts of the work (cadence of the checkpoints, buffers, interruptions): whoever replicates can break up the work as they like and land on the same bytes. Every family declares its own canonical form, and it is there that the fingerprint lives: for the graph the concatenation of the runs in order of height; for the archive of revelations the fused files of the merge (sorted and globally deduplicated); for the scanner the bitmaps of the hits, which are canonical already as files because the order of their bits is the order of the files of locks, themselves sorted and fingerprinted. The things that are deliberately NOT replicable (names and sha256s of the individual runs, statistics) sit in the logbook, never in the data.

The replication contract: three things to fix. Whoever wants to reproduce our exact bytes has to fix (1) the scan height (--end); (2) the height of the snapshot of the locks: same dumptxoutset = same txoutset_hash = the same locks files byte for byte; (3) the perimeter (the declared defaults; the --no-faces/--no-cosigners flags change only the scanner's bitmaps — graph and archive do not depend on them, because they apply the perimeter at read time). With the three fixed, every published fingerprint is a string anyone must land on.

How small the result is. It is worth saying it plainly: the entire answer of pass 1 — which of the 51.7 million locks of the snapshot the history has revealed — fits in about 6 MB: one bit per lock, in the order of the files of the prepare. The canonical fingerprint published at every checkpoint is the sha256 of those 6 MB.

The bitmap lives off a join (and the type needs no tables). Two things the bitmap does NOT contain, and where they live. The TYPE of lock is written nowhere because it is encoded in the partition: the bitmaps are four, one per mechanism, each paired with its own file of locks — the count by type is free by construction. Everything else, though, the bitmap does not know: the bit i says only «row i», and the meaning is born by setting it alongside row i of the file of locks (digest and satoshis). It is a boolean column detached from its table, paired by POSITION, not by key: this is why the checkpoint refuses to resume on files of locks other than the ones it was born with — a positional join on the wrong table would not give errors, it would give silent garbage — and this is why the fingerprint of the bitmaps is citable only together with the sha256s of the locks in the manifest. The pattern continues upwards: the file of locks does not have the creation height of the coins, so «burnt by age» will be a join with the data that does have it (the utxos.dat snapshot), as history will be a join with the graph. The bitmap stays minimal on purpose: the bare, fingerprintable result; every richer question is a setting-together of canonical pieces, never information stuffed into the result.

A free cross-check between level 2 and level 3. The manifest of the locks adds up the satoshis of the four behind-hash categories: ~18,118,632 BTC. The level 2 census, by a route entirely of its own, arrives at the same figure: total UTXO (20,053,839) minus exposed by construction (1,932,589) minus the dust outside the perimeter. Two independent readings of the same snapshot that bear each other out: the 19×3.125 delta of level 1-2 has found its twin.

A single pattern for everything that does not fit in memory: sort in pieces, then fuse. The prepare accumulates locks up to a ceiling (8 million records), sorts the little pack in RAM, dumps it into a temporary run file, and starts again; at the end it fuses the packs in streaming, looking only at the first card of each and drawing the smallest every time (heapq.merge: k records in memory, not k files). And in the fused stream deduplication is free: in a sorted sequence the equal ones arrive adjacent, a comparison with the previous record is enough to add up the satoshis of the same lock (several UTXOs behind the same hash: reuse seen from the other side — the right unit is the lock, because one revelation burns everything that is behind it). The sha256 of the final file is updated as it is written (the fingerprint is born with the file, not from a re-read), the temporaries are deleted: they were scaffolding, and indeed the result is identical byte for byte whatever the number or the boundary of the packs. The same pattern holds up the archive of revelations (sorted runs + periodic fusion) and its reading (fused + subsequent runs, fused on the fly): one idea, reused three times.

The prepare in a figure, from the heap to the term of comparison:

UTXO snapshot at 957,301 (utxos.dat: ~166M coins)
     │ streaming, one coin at a time
     ▼
distil the 4 behind-hash types      (p2pk / p2tr / other:
p2pkh · p2sh · p2wpkh · p2wsh        discarded, already exposed
     │                               by construction)
     │ accumulate in RAM (ceiling: 8M records)
     ▼
pack full → sort in RAM → dump to disk
run_p2pkh_0.tmp · run_p2pkh_1.tmp · …
(~7 per type, each sorted within itself)
     │
     ▼ streaming fusion (heapq.merge: you look at the
       first card of every pack, you draw the smallest)
a single sorted stream → the equal ones arrive ADJACENT
→ same lock: satoshis added up (look-behind dedupe)
     │ sha256 updated as it is written
     ▼
locks_p2pkh.bin … locks_p2wsh.bin   +   manifest.json
(51.7M rows: digest | satoshi)          (fingerprints, base_hash)

the .tmp files are deleted: scaffolding that leaves no trace

Where the ideas come from (the honesty of the pedigree). Not one brick of this architecture was invented here, and it is a merit worth declaring: they are classical techniques, tested for decades, and the reader who wants to go deeper finds the pointers in the References at the end of the manual. Sort-in-pieces- then-fuse is the external merge sort of the textbooks (Knuth, 1970s, born for magnetic tape; Unix's sort(1) uses it, and so does every database when RAM is not enough). Sorted runs that accumulate with periodic fusion are, conceptually, a hand-made LSM-tree (the idea behind LevelDB and RocksDB) — with the difference that the files stay bare and readable instead of living inside an engine. The checkpoint with watermark and resumption is standard stream-processing discipline. The immutable raw log with indexes as rebuildable derivatives is event sourcing (the graph is the log, the derivatives are the views). Fingerprints over a canonical form are content addressing, the same family as Core's muhash which we cite as a twin. Two independent routes that must coincide are as old as double-entry bookkeeping. In the specific domain, the nearest relative of the co-emission is BlockSci (parsing the chain once into a compact form for batch analysis); Electrs is the cousin we chose NOT to imitate, because it serves a different model of question. What was not to be found ready-made — and what guided every choice of composition — is the objective function: these systems optimise for serving queries fast, here one optimises for THIRD-PARTY VERIFIABILITY. From there come the things that do not exist in the projects cited: the fingerprint at every checkpoint, the canonical form blind to the boundaries as a format requirement, the perimeter moved to read time for an exact cross-check, the monotone lower bound as the publishable form of the result, the bare files and the standard library alone where readability and the absence of dependencies ARE the argument. Taking up tested bricks is not laziness: it is a guarantee of quality (you stand on known shoulders, not on notions of our own) and at the same time a declaration of honesty about where tradition ends and the project begins.

The principles of the formats

The route chosen can be said in one line: FILES IN OUR OWN FORMAT THAT REDUCE SUPERSTRUCTURE TO A MINIMUM, WITH THE EFFICIENCY BUILT BEYOND THE RAW DATUM, NEVER INSIDE IT. In summary, the rules that hold it up:

  • The raw is the source of truth. Minimal, faithful to the block, append-only; whatever can be re-derived does not go in (the fee is a join, not a field). Acquisition and processing separated: the pass extracts and does not interpret, the questions are redone for free on local files.
  • Minimal, documented, neutral formats. Every format is born with its own spec and its own canonical form, byte order and primitives declared; neutral by construction (the canonical fingerprint is the cross-implementation test); it evolves by versions (graph-v2 alongside graph-v1), never by mutation.
  • Order is the first index. A file sorted on the right key is already an index; the intelligence sits beyond the record (positional joins, bitmaps set alongside the tables), not inside it; standard tools where they suffice (SQLite-first), our own format only where scale and canonicity are needed together.
  • Every question is a derivative. A declared reconstruction rule and a fingerprint of its own: derivatives are deleted and remade, the raw is not. The replication contract (height, perimeter, version) travels with every number: a number without a contract is an opinion with too many decimal places.
  • Efficiency is measured, not estimated. It is bought only where it changes what is feasible (the category jumps: hours instead of days, GB instead of hundreds); a saving that costs readability is refused. One extra pass is justified only if it adds information that a single pass cannot hold.

The honest judgement: it is the classical architecture (immutable log plus derived views) with an unusual objective function, third-party verifiability. The two things to keep an eye on: the citizenship cost of our own formats (no ecosystem reads them — acceptable only as long as the formats stay minimal, documented and neutral, otherwise it is lock-in towards ourselves) and efficiency, which stays the third priority after verifiability and readability.

The artifacts: the flow at a glance

An ASCII figure. It is the single map both tables rest on.

             ┌─ census ─────────────────► census_<h>.csv        (context: how many coins, of what type)
utxos.dat ──►│
(snapshot)   └─ reuse prepare ──────────► locks_<h>/            (the «locks»: the current UTXOs)
                                              │
                    ┌─────────────────────────┴───────────────────────┐
   DIRECT ROUTE     │                                   ARCHIVE ROUTE │
     reuse scan ────┴──► checkpoint/                  archive scan ───┴──► utxo_reveal/
   (lean, over RPC)      ├ hits_<type>.bin         (over RPC, appendable)  ├ runs/…_keys.bin
                         ├ state.json                                      ├ runs/…_scripts20.bin
                         └ curve.csv     (the curve)                       ├ runs/…_scripts32.bin
                              │                                            └ state.json
                              │                   merge ──► manifest.json  │   (canonical fingerprint)
                              └──────────────► crosscheck ◄────────────────┘   PASS / FAIL

   (only with --graph)  graph/ ──► index build ──► index/ ──► derived build ──► derived/
                                                (history of a lock, fees, co-spends)

Two independent routes (the reuse count and the archive of revelations) converge in the crosscheck, which demands the same number from both. In the figure the commands are written without the nodsig prefix, which is understood. The same pass can also co-emit the headers archive (--headers) and the census of the nonces (--nonces), which do not appear in the figure: they are side branches of the scan, described at step 2 of level 3.

What to keep afterwards, and what can be deleted. Build everything and keep everything and you are around ~910 GB: graph ~301, index 248.5, derivatives 190.5, archive 97.7, plus the snapshot, the locks and — if co-emitted — the census of the nonces (59.7). But the artifacts that answer the questions are THREE — archive, index, derivatives — and the graph, which is the largest of them all, is read by no query: it is the raw material the index was built from, and once that work is done it just sits there. It is therefore the single largest saving available, at the price that rebuilding or rewinding the index would mean walking the chain again. It is worth keeping for whoever means to follow the chain forward; whoever had a single question can free it. Snapshot and locks serve census, reuse prepare and archive derive: once you have the numbers, they become useful again only together with a new snapshot, at a new height. The complete map — every file, who produces it, who reads it, what can be skipped depending on the question — is docs/ARTIFACTS.md in the repo, which keeps it aligned with the code.

Twin fingerprints. The reuse count (reuse-scan), the archive of revelations (reveal-archive) and the graph (graph) each carry the same idea of a canonical fingerprint, a relative of the node's muhash: redoing the step, anyone obtains the same bytes and the same fingerprint on their own machine.

The published canonical fingerprints. These are the values a third party compares in order to know whether they have redone the same counts. The first three pin the INPUT (the state of the chain, verifiable at once with a node, without our code); the others the RESULT of our count (reproducible with the tools). It is the two-level replication contract.

Fingerprint Block Produced by Value
base block of the snapshot 957,301 dumptxoutset 000000000000000000005f3f0d5434e16bb6ea806ecd140be91bf6a15562c077
txoutset_hash (content of the set) 957,301 dumptxoutset ec79aed99b8d22d7315c76488fe09975ef9258039ee5583c7e8c7b6dc8d4cc58
muhash (the set as a set) 957,282 gettxoutsetinfo 38dd237f9f83e14d2e8fefa396871a29c9a6226fc7f1f89eda2d9279e314671b
locks / curve of reuse 957,301 reuse scan / archive derive 5fd579dbc6190f5bf08e35947ab77a55e067aa366acd13bd49fa30d3ba9d0ee7
archive of revelations 957,301 archive merge aacaf02dca2fc5ba8532e54fa75159041fc99051efa68eb63e59bc9537369ced
raw graph 957,301 graph fingerprint a014f787256e1831c90290e04c2adbcf1fe00cfc3f2d26bb668bff491aa54190
outpoint index 957,301 index verify / stats 338c6c48f6e6c806c6d0a494bb9ca5060adcb83167c0db45328d39b40b14a69d
derivatives (history, fees, co-spends) 957,301 derived verify / stats 44689372f169a5c503bdf128a082c31fef767e35c77696e8e60843b42afa1c80

The last two are born of the first, which each manifest declares as parent next to its own fingerprint: whoever redoes the path must find the whole chain graph → index → derivatives again, not just the last link. The locks fingerprint is of a different nature from the other four — not the seal of an artifact but the result of the scan, recomputable offline from the bitmaps (see level 3) — and it is the reason it is cited together with the digests of the locks, never on its own.

The muhash is at block 957,282 because it comes from the live count of level 1, a few blocks before the snapshot; everything else is pinned to 957,301, the height of the snapshot the count rests on.

Shareability. Every artifact listed here derives from public chain data. None contains addresses of your own: they are all shareable. The census, in addition, is aggregated by construction.

Public GitHub repo

The code of the tools (UTXO parser, address check, index and derivatives) is here, under the MIT licence:

github.com/amenano/nodsig

The repo carries the check and all the code to re-run the analysis from scratch; NOT the downloadable artifacts (archive, graph, locks): they are large and whoever wants them rebuilds them with the tools of the repo — the published fingerprints say whether the rebuild is the right one.

From a clone there is nothing to install, and one thing alone is needed: Python 3.10 or later. No dependencies, the standard library is the whole runtime — which is the point, not an affectation: what you run is what you can read.

git clone https://github.com/amenano/nodsig.git
cd nodsig
python3 -m nodsig --version
python3 -m nodsig                 # the map of commands

nodsig <group> … and python3 -m nodsig <group> … are the same program, and the manual always writes the first form; whoever prefers the command on their PATH installs it from the clone with pip install .. python3 -m nodsig.<module> runs a single tool, and it is the way to read the code while it runs; `nodsig

-h` prints the options of each. The repo's README also declares the perimeter the project presents itself with, and it is worth reading before taking an answer as good: it is a proof of concept, with no roadmap and no maintenance promised, and the rule it applies to itself — a publishable number is a number two independent routes arrived at — does not stop at the boundary of the repo. A single answer from a single tool is a lead, not a fact, and this tool is no exception to a principle it is built on. Two pages of the repo deserve a mention of their own, because they answer the two questions whoever gets this far asks first. `docs/why-artifacts.md` takes seriously the objection «I already have a node, why pay for the disk twice»: the answer, in short, is that the objection measures the bytes and not the work — the node files the chain in the order it happened, and every question in this manual is a question in another order; with the sums of how little it really takes to keep (for single-key addresses, one partition of the archive alone; for a Taproot, no file at all). And `docs/exposure-check.md` walks the exposure question on its own, down to the procedure on the machine with no network. **This manual describes `nodsig` 1.x**, and the version is told by the command itself (`nodsig --version`). The FORMATS are the contract, the commands the convenient tool for obtaining them. The question «which code produced this artifact?» is answered by the artifact itself, not by a promise about process: the manifest records the producer — version, commit when it can be determined, and a `dirty` flag that says whether the tree carried never-committed changes. It is the distinction a convention cannot guarantee: a rule is broken in silence, a recorded field is not. The field is declared, not attested — the fingerprint remains a function of the bytes alone, and the producer does not enter it: it is read, not verified. ## Log of the runs The fingerprints are not repeated here: they are in full under «The published canonical fingerprints», and this log is for saying *when* and *what happened*. | date | height | txouts | total BTC | outcome | |------|--------|--------|-----------|------| | 2026-07-09 | 957,282 | 166,238,015 | 20,053,779.26 | level 1 | | 2026-07-09 | 957,301 | 166,224,061 | 20,053,838.64 | level 2 CLOSED | Of the first: Umbrel node confirmed non-pruned, and consistency with theoretical issuance verified (the −230 BTC, known). Of the second: `base_hash` `…5562c077`, census in 10 minutes on the PC, a delta against level 1 of 59.375 BTC = 19×3.125 exactly, and exposed by construction 1,932,588.84 BTC (9.64%). The direct route of reuse (`reuse scan`) was walked once, in ~2 days: 8,784,364 locks over 5,084,725.41330985 BTC, fingerprint `5fd579db…0ee7`, `malformed scriptSigs: 0`; and the `crosscheck` that compares it with the archive closed PASSED (`real 258m`), the two routes on the same fingerprint. It is the check of method that is inherited, not repeated. The other long runs are logged by duration rather than by totals, and they all sit at the same height 957,301. The durations are the log's wall clock, from start to outcome, sealing pass included: it is the reason the index reads 24 h 51 here and 23 h 24 in its own step — there it is the manifest's `build.seconds`, which counts the construction alone. Neither number is wrong; they are two different questions. | finished on | step | duration | outcome | |---|---|---|---| | 3 Aug 06:37 | `archive scan --graph --headers` (plus `--nonces` co-emitted) | 58 h 47 | 7,704,658,388 revelations in 2,787 runs, `malformed scriptSigs: 0` | | 3 Aug 07:33 | graph comparison (in-scan digest) | inside the scan | 1,178/1,178 intervals, `result: ok` | | 3 Aug 13:12 | `archive derive` (on the runs) | 5 h 39 | reuse table, fingerprint `5fd579db…0ee7`; curve of 96 rows | | 3 Aug 17:29 | `archive merge` | 4 h 17 | seal, fingerprint `aacaf02d…9ced` | | 3 Aug 20:11 | `nonces merge` (co-emitted) | 2 h 41 | seal of the nonce census | | 3 Aug 20:11 | `headers fingerprint` | 13 s | fingerprint `6af1fed6…e5d4` | | 3 Aug 21:25 | `archive v1-digests` | 1 h 13 | the three July digests found identical | | 4 Aug 05:03 | `archive derive` (fused base) | 4 h 56 | the same `5fd579db…0ee7`, curve identical byte for byte | | 4 Aug 06:46 | `archive verify --deep` | 1 h 41 | PASSED | | 4 Aug 23:09 | `index build` | 24 h 51 | fingerprint `338c6c48…a69d`; `overwritten txids: 2, duplicate spends: 0, unresolved: 0` | | 5 Aug 00:15 | `index verify --graph` | 1 h 03 | every byte, ladders rebuilt, parent `a014f787…4190` confirmed | | 5 Aug 01:14 | `headers crosscheck --index` | 59 min | 957,301 Merkle roots recomputed, PASSED | | 6 Aug 07:39 | `derived build` | 14 h 30 | fingerprint `44689372…1c80`, cross identity fees/spent | | 6 Aug 08:22 | `derived verify --index` | 42 min | every byte, parent confirmed | The reuse table came out twice with the same fingerprint, from two states of the code: on the runs on 3 Aug and on the fused base on 4 Aug, after the correction of the tiled curve. The second header row differs only in the count of presences in perimeter (the runs overlap, the fused base does not): the table and the fingerprint do not. Plus the measurements already taken on 30 July on the real material: `census` ~18 min, `reuse prepare` ~35 min / ~1.4 GB. ## References — the algorithms and the ideas taken up Declaring where the bricks come from is a form of honesty towards the reader and at the same time a guarantee: the project stands on techniques tested for decades, not on off-the-cuff inventions. Here are the pointers for whoever wants to study them at the source (see also «Where the ideas come from»). - D. E. Knuth, *The Art of Computer Programming*, vol. 3: *Sorting and Searching*, 2nd ed., Addison-Wesley, 1998 — §5.4, «External Sorting»: the sort-in-pieces-then-fuse of the `prepare` and of the archive, born when data lived on tape and RAM was a luxury. - P. O'Neil, E. Cheng, D. Gawlick, E. O'Neil, «The Log-Structured Merge-Tree (LSM-Tree)», *Acta Informatica* 33(4), 1996 — sorted runs accumulated + periodic fusion: the filing cabinet is a hand-made LSM-tree, with readable files in place of the engine. - J. Kreps, «The Log: What every software engineer should know about real-time data's unifying abstraction», LinkedIn Engineering, 2013, https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying-abstraction — the immutable log as truth and indexes as derivatives; it is also the frame of the resumption watermarks. - M. Kleppmann, *Designing Data-Intensive Applications*, O'Reilly, 2017 (and the talk «Turning the Database Inside-Out», Strange Loop 2014) — materialised views rebuildable from the log: the conceptual frame of our «derivatives with a reconstruction rule and a fingerprint of their own». - M. Bellare, D. Micciancio, «A New Paradigm for Collision-Free Hashing: Incrementality at Reduced Cost», EUROCRYPT 1997 — incremental hashing of sets: the idea behind Bitcoin Core's muhash, the twin of all our canonical fingerprints. - H. Kalodner, M. Möser, K. Lee, S. Goldfeder, M. Plattner, A. Chator, A. Narayanan, «BlockSci: Design and applications of a blockchain analysis platform», USENIX Security Symposium 2020 (arXiv:1709.02489) — the nearest relative of the graph co-emission: parsing the chain once into a compact form meant for batch analysis. - Electrs, https://github.com/romanz/electrs — the per-scripthash indexer we chose NOT to imitate: useful precisely for understanding the boundary between its model of question (one address at a time, online) and the global questions the graph serves.