wesandClaude Opus 5
f55e9d0253
Adds `kb_exfiltrator snapshot <db> [endpoint] [timeout-secs]`, which uploads a
consistent copy of the whole database instead of just extracted highlights. The
server can then parse whatever it wants, so adding a new statistic never means
touching the device again.
Uses VACUUM INTO rather than copying the file. Nickel keeps the database in WAL
mode and recent commits live in KoboReader.sqlite-wal until a checkpoint, so
copying the bare .sqlite silently loses reading progress — measured at 19 hours
of drift on the live device. Copying all three files instead is non-atomic: a
checkpoint landing mid-copy yields a pre-checkpoint main file plus a post-reset
WAL, losing data with no error. VACUUM INTO runs in a read transaction, so it
sees WAL-resident commits and emits one compacted journal_mode=delete file.
Also fixes the failure mode that left a process hung for 16 days: the server
stopped responding and std.http.Client has no timeout, so the process blocked
forever while holding the SQLite handle open. Snapshot mode closes the database
before any network I/O, and a watchdog thread hard-exits after a deadline
covering the whole run (snapshotting can stall on a locked database too).
Two portability constraints, both specific to the Kobo Elipsa:
- Linux 4.9.77 predates statx (4.11). Zig's File.stat() and getEndPos() both
issue it, returning ENOSYS as a bare error.Unexpected — after the snapshot
is already written, so it presents as a post-write failure. readToEndAlloc
with no size hint uses plain read() calls instead.
- usize is 32-bit on armv7, so a u64 stat size will not coerce. Only the ARM
build catches this.
Legacy invocations are unchanged, so the existing NickelMenu item keeps working.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Kobo Highlight Exfiltrator
A simple, statically-linked binary to extract highlights from your Kobo e-reader.
Features
- Zero dependencies: Fully statically linked with musl libc and SQLite
- Cross-platform: Builds for both x86_64 (development) and ARM (Kobo)
- Small binary: ~780KB for ARM release build
- Simple: Just point it at your KoboReader.sqlite database
Building
For local testing (x86_64)
zig build
./zig-out/bin/kb_exfiltrator local-dev.sqlite
For Kobo (ARM)
./build-kobo.sh
# or manually:
zig build -Dtarget=arm-linux-musleabihf -Doptimize=ReleaseSmall
Usage on Kobo
-
Copy
zig-out/bin/kb_exfiltratorto your Kobo (via USB or SSH) -
Run:
./kb_exfiltrator /mnt/onboard/.kobo/KoboReader.sqlite -
Or save to a file:
./kb_exfiltrator /mnt/onboard/.kobo/KoboReader.sqlite > highlights.txt
How It Works
The program:
- Opens the SQLite database at the path you provide
- Queries the
Bookmarktable for all highlights (entries with non-NULLText) - Displays each highlight with:
- Date created
- Book identifier (ContentID)
- Highlight text
- Any annotations/notes you added
Dependencies
All dependencies are bundled into the binary:
- SQLite 3.48.0 (amalgamation build)
- musl libc (for ARM target)
Requirements
- Zig 0.15.2 or later
Notes
- The binary is statically linked, so you can copy just the single file to your Kobo
- Uses musl libc for smaller binary size and better portability
- SQLite is compiled with:
SQLITE_THREADSAFE=0: No threading support (smaller, faster)SQLITE_OMIT_LOAD_EXTENSION: No dynamic extensions (more secure)