Skip to content
Building the dataset

Building the dataset

The dataset is compiled by the builder CLI from two committed layers, so a rebuild can never clobber authored work:

LayerWho writes itHolds
config/overrides/you (hand-edited)Structure the open sources can’t express: Series / Franchise boundaries, ordering, alternateCutOf, watchOrders, which series are linearly numbered.
data/builder build (generated)The resolved records: overrides + facts from open data + computed absoluteNumber. Never hand-edit.

(absoluteNumber is the running episode count across a whole series, computed by the builder for linearly-numbered series.)

Because builder build treats overrides/ as read-only input, builds are deterministic and idempotent: the same overrides plus the same pinned sources always produce the same data/.

Prerequisites

Go (see the module’s go.mod for the version). Build the CLI from the repository root:

go build ./cmd/builder

Commands

./builder init                 # download the pinned sources into .sources/
./builder build                # (re)build data/ for all overrides
./builder build demon-slayer   # build/rebuild just one franchise or series
./builder refresh              # update sources to latest, bump pins, rebuild all

The build fails on any unknown id, dangling reference or schema violation, so a successful build is always a valid dataset. Where it makes a low-confidence guess (chiefly title-language tagging) it prints a report — pin those cases with an override.

Adding an entry

Create config/overrides/series/<id>.yaml describing the structure, then run ./builder build <id>. The builder fills in facts (titles, episode counts, cross-IDs) from the open sources and writes the resolved record to data/series/<id>.yaml.

Sources

Facts come from openly-licensed, redistributable sources (AniList is not used — its ToS forbids redistribution):

Sources are not committed. builder init downloads them into a gitignored .sources/ cache at the versions pinned in config.yaml, and builder refresh updates those pins. (The build verifies pinned checksums, so a rebuild uses exactly the sources it was authored against.)

Serving it

The same dataset is compiled into the API binary with go:embed, so the server is stateless and self-contained:

go run ./cmd/api                 # listens on :8080

See Using the API for the request format.