# kb_exfiltrator

A statically-linked ARM binary that pulls data off a Kobo e-reader. Zero runtime
dependencies — the Kobo has no `curl`, no `sqlite3` CLI, and a busybox `wget` that cannot
POST, so a compiled tool is required rather than convenient.

## Modes

```
kb_exfiltrator snapshot   <db> [endpoint] [timeout-secs]   # primary
kb_exfiltrator highlights <db> [endpoint]                  # legacy
kb_exfiltrator            <db> [endpoint]                  # legacy alias
```

**`snapshot`** uploads a consistent copy of the *whole database*, letting the server parse
whatever it wants — so adding a new statistic never means touching the device again. It:

1. `VACUUM INTO`s a snapshot to `/tmp` (tmpfs, so nothing touches flash)
2. **closes the database before any network I/O**
3. POSTs it as `application/octet-stream` with `x-kobo-sha256` and `x-kobo-device` headers
4. deletes the temp file

Omit the endpoint to leave the snapshot on disk for inspection.

**`highlights`** is the original mode: extract `Bookmark` rows and POST them as JSON. Retained
for debugging and backwards compatibility; the snapshot pipeline supersedes it.

## Why `VACUUM INTO` rather than copying the file

Nickel keeps `KoboReader.sqlite` in WAL mode, and recent commits live in the `-wal` file until
a checkpoint. Copying the bare `.sqlite` silently loses reading progress — **measured at 19
hours of drift on a live device**. Copying all three files instead is not atomic: a checkpoint
landing mid-copy yields a pre-checkpoint main file plus a post-reset WAL, losing data with no
error. `VACUUM INTO` runs inside a read transaction, so it sees WAL-resident commits and emits
a single compacted `journal_mode=delete` file.

## Timeouts

A watchdog thread hard-exits the process after `timeout-secs` (default 120), armed for **every**
mode before dispatch. This is not theoretical: a v1 run hung for **16 days** because the server
accepted the connection and never responded, and `std.http.Client` has no timeout. It held the
SQLite handle open the whole time — which is also why snapshot mode releases the database
before it touches the network.

## Building

**Zig 0.15.2 specifically.** Not "or later" — 0.16 (what pacman ships) breaks this code.

```bash
zig build -Dtarget=arm-linux-musleabihf -Doptimize=ReleaseSmall   # the Kobo
zig build test -Dtarget=x86_64-linux-musl                         # tests
```

The **host** build fails on CachyOS with `unhandled relocation type R_X86_64_PC64` in glibc's
`crt1.o` `.sframe` sections. That is an environment quirk, not a code problem — build and test
against musl as above.

## Kobo portability constraints

Both were found only by running on real hardware:

- **No `statx`.** The Kobo Elipsa runs Linux 4.9.77; `statx` landed in 4.11. Zig's
  `File.stat()` *and* `File.getEndPos()` both issue it, returning ENOSYS as a bare
  `error.Unexpected` — and it surfaces *after* the snapshot is written, so it presents as a
  post-write failure. Use `readToEndAlloc` with no size hint. **Avoid `File.stat()` anywhere
  in this codebase.**
- **`usize` is 32-bit on armv7**, so a `u64` stat size will not coerce. Only the ARM build
  catches this.

Zig also collects `test` blocks only from a test unit's root file, hence
`test { _ = @import("snapshot.zig"); }` in `main.zig` — without it the snapshot tests silently
do not run (symptom: `1/1 tests passed` instead of `4/4`).

## Deployment and operations

This repo is only the binary. The pipeline it feeds — ingest service, database schema, the
automatic dhcpcd trigger, router-based monitoring, and the recovery runbook for when a Kobo
firmware update wipes the rootfs — lives in **`c0smere_devops/kobo-sync/`**:

| doc | what |
|---|---|
| `kobo-sync/README.md` | how the whole pipeline fits together |
| `kobo-sync/NOTES.md` | device findings and design rationale — **read before changing device behaviour** |
| `kobo-sync/RECOVERY.md` | restoring the trigger after a firmware update |
| `kobo-sync/STORYGRAPH-PLAN.md` | plan for the not-yet-built StoryGraph connector |

## Bundled

- SQLite 3.48.0 (amalgamation), `SQLITE_THREADSAFE=0` + `SQLITE_OMIT_LOAD_EXTENSION`
- musl libc for the ARM target