Skip to content

v0.32.0 - Migration fixes

2026-06-11

Bugs

  • [x] Big map doesn't work in prod (probably a vtt-modules issue)
    • GET https://huggingface.co/Xenova/real-esrgan-x4-anime6B-onnx/resolve/main/onnx/model.onnx fails with 401, response "Invalid username or password."
    • (Root cause: the Xenova HF repo was deleted/made private — the HF API 401s for everyone, and no trusted ONNX mirror of anime6B exists, so this could never self-heal. Fixed in the genai repo (not vtt-modules) as v0.5.2 (committed + tagged there, needs git push --follow-tags so CI publishes before this manifest deploys): the weights are now self-hosted — assets/models/real-esrgan-x4-anime6b.onnx (17.9 MB, converted from the canonical official-release RealESRGAN_x4plus_anime_6B.pth with dynamic H/W axes, parity-verified < 5e-6 vs torch), served by the module's own router at GET /api/modules/genai/local-models/:file (whitelist, immutable cache headers, unauthenticated like the client's model fetch), with a new catalog modelUrl that loadUpscaler prefers over the HF-derived URL. modules.hosted.json bumped genai 0.5.1 → 0.5.2 here. Old broken Cache Storage entries are bypassed naturally since the cache is keyed on the (now different) URL.)
  • [x] Prod > Generate Encounter > auto-select art > tokens created, but...
    • Tokens have no art on map, despite the Edit Token modal showing non-tokenized art. After opening this modal and tweaking the tokenized art, it appears on the map. Correct behavior: tokenized art should appear on the map immediately after generation, having used AI-tokenization on the generated monster's art. (Root cause: auto-selected art kept the raw Google-hit URL, and the WebGL texture loader fetches with crossOrigin='anonymous' — arbitrary external hosts send no CORS headers, so the texture load failed (the Edit Token modal kept "showing" the art because plain <img> needs no CORS; tweaking there ran the modal's save path, which uploads to our storage — that's why it healed). Fix: handleInsert now runs each group's art through the new utils/encounterArtIngestion.ts before building token payloads — uploadImage ingests the external URL server-side into our storage (cache-busted URL, mirroring TokenModal), then BlazeFace tokenizeImage computes the face-centered transform persisted as tokenizeSettings: { transform, mapAppearance: 'tokenized' } (the renderer applies it in-shader; monster-red #dc2626 border matches tokenDataFromGeneratedMonster). Per-group in parallel, Insert button shows loading, every step best-effort: upload failure keeps the old behavior for that group, tokenize failure still inserts the stored (renderable) art. A drawer-made user transform is never overwritten, and already-stored URLs skip the upload. 8 behavioral tests (encounterArtIngestion.test.ts, injected deps) + 4 source contracts.)
    • Tokens have no portrait art. The auto-selected art exists, but wasn't assigned to the portrait. Assigning it manually works fine. Correct behavior: portrait art should be assigned immediately after generation, having used the same auto-selected art as the token. (The empty portrait was deliberate v0.25-era behavior: a workaround for a legacy WebGLMap art-priority rule (portrait imageUrl beat tokenized art) that no longer exists — the v0.22.0+ renderer only reads the token-art slot. ingestAndTokenizeGroupArt now always assigns a portrait at insert: a user-picked portrait is kept (a pre-insert ensurePortraitArtwork clone of the same source is re-pointed at the stored URL rather than duplicated); otherwise the stored source is cloned into a second artwork WITHOUT tokenizeSettings (portrait consumers render the raw image via getCharacterDisplayArt-style cutoutUrl || imageUrl), and portraitArtworkId points at it. Applies even when ingestion fails (raw-URL portrait beats none). The three stale "leave portraitArtworkId undefined" comments in GenerateEncounter were rewritten to describe the insert-time flow. +3 portrait tests, upload-failure test updated for the new fallback shape (15 total in encounterArtIngestion.test.ts).)
    • I am the Campaign Owner (GM), but the Character Context seems to think I don't have edit permission for these tokens. I can't edit their actions. In the Edit Token modal, I can edit their actions. Note: the context is Impersonated Token, not Impersonated Character. It's possible Impersonated Token simply doesn't have the ... button for actions allowing them to be edited. (Exactly right — no permission check was involved: TokenSheet (the Impersonated Token Context body) rendered plain action buttons with no edit affordance at all. Ported the Character Context treatment: each non-campaign action gets a split-button … menu (Edit / Delete, PlayModeView's action.source !== 'campaign' gate) and an Add Action button renders under the list (section now shows for action-less tokens too, so Add is always reachable). ImpersonatedTokenContext mirrors CharacterContext's wiring — a modal-only ActionEditor controlled by editActionId ('__new__' sentinel for create), backed by the useTokens mutations and fed campaign functions + property definitions for its pickers; the handlers use the executor hook's IToken-typed live token (same source CharacterContext uses), so edits appear without a reload. All three TokenSheet edit props are optional — omitted, the sheet renders exactly as before. 4+5 source contracts (TokenSheet.actionEditMenu, ImpersonatedTokenContext.actionEditor); the v0.24.0 editTokenLink icon-import contract was loosened to not pin the exact icon list.)
  • /characters/:id route
    • [x] The source artwork thumbnail cannot be dragged to the token or portrait drop zone. The cursor indicates draggability, but it doesn't work. No console error. (The panel is CharacterArtworkPanel (ui-components). Its DnD differed from the TokenModal GeneralTab implementation — which uses the identical strip→slot pattern and works — in three mechanism details, now aligned: (1) handleDragStart restyled the dragged thumbnail (opacity: 0.5) via synchronous setState — React 18 flushes that re-render before the handler returns, and Chrome silently aborts a native drag when the source node is restyled before the drag-ghost capture ("cursor shows grab, nothing drags, no error"); the affordance is now deferred via setTimeout(0) (GeneralTab's equivalent state is never even used for styling). (2) effectAllowed/dropEffect switched from the odd-one-out 'link' pair to GeneralTab's proven 'copy'. (3) slots now also cancel dragenter (spec says a valid drop target must cancel both). Hardening that rode along: drops validate the payload is a known artwork id (foreign drags — e.g. an image from another tab — also carry text/plain), and the drag-over visual state is cleared even for rejected drops. Verification caveat: the Chrome-extension tooling cannot synthesize OS-native drags (left_click_drag errors), so the abort couldn't be reproduced live; a synthetic DragEvent chain confirmed handlers + payload + slot callbacks work end-to-end, and 5 new jsdom behavioral tests (CharacterArtworkPanel.test.tsx, 20 total) pin the full drag flow, the no-sync-restyle constraint, dragenter/dragover cancellation, copy effects, and foreign-payload rejection. Please re-test the real drag by hand.)
    • [x] The token drop zone's thumbnail doesn't reflect the tokenization settings. No matter what is changed in those settings, the thumbnail always shows the raw source art. (Two stacked causes. (1) CharacterDetail generates the slot's circular-crop canvas preview ONCE per artwork id (!artworkPreviewUrls[art.id]) — every later settings edit kept serving the first render for the rest of the session; and if that first generation failed (e.g. CORS-blocked source), the slot fell back to raw art forever. The preview cache is now keyed by the tokenizeSettings that produced it (JSON.stringify([transform, backgroundColor])), so drawer edits regenerate the thumbnail (drawer state updates per slider tick → effect re-runs; the browser image cache makes re-renders cheap), failed generations retry on the next change, and switching appearance away from tokenized deletes the stale entry. The drag-assign auto-tokenize path stores its instant preview under the same key shape so the effect neither redoes nor wrongly skips it. (2) CharacterArtworkPanel.getTokenDisplayUrl ignored mapAppearance: 'original' — the slot now shows the raw source in that mode instead of a stale legacy bake. Tests: 5 source contracts (CharacterDetail.tokenPreviewRegen.test.ts) + 2 render tests (panel, 22 total). Live-verified: slot renders the webp canvas preview on /characters/:id.)
  • /play route
    • [x] In the Edit Token modal, the token drop zone's thumbnail doesn't reflect the tokenization settings. No matter what is changed in those settings, the thumbnail always shows the raw source art. (The slot renders via TokenThumbnail (the v0.23.1 hotfix for this same symptom), but its gate tokenized = !processedUrl && … pinned any artwork carrying a legacy server-baked processedUrl to that stale snapshot forever — the TokenizationEditorDrawer only persists tokenizeSettings (by design, "we no longer persist the canvas-baked processedUrl"), so edits never showed. Verified against real data: existing artworks commonly have BOTH a legacy bake and modern settings. TokenThumbnail now live-renders whenever a transform exists (mapAppearance !== 'original' && (transform || !processedUrl)), with the bake demoted to fallback for render failures and transform-less artworks; GeneralTab now feeds it originalUrl || imageUrl — the source the transform was authored against (drawer + WebGL renderer resolve it the same way), so the crop matches the map. 4 new source contracts in GeneralTab.tokenThumbnail.test.ts (7 total). The pre-existing GeneralTab.facingLogic baseline failure (drifted v0.28.0 contract) is unrelated and unchanged.)

Database Migrations (fix prod schema drift)

Root cause analysis (2026-06-11): generated drizzle migrations are never executed against prod. The postgres branch of initializeDatabase() (packages/server-core/src/db.ts) only runs the hand-mirrored postgresCreateTables() + postgresApplyColumnMigrations(); the migrations/*.sql files are consumed solely by the SQLite dev path, and even there only the 14 entries journaled in migrations/meta/_journal.json (latest: 0028) — the rest are dead files. The runtime Docker image doesn't include migrations/ at all (only dist is copied), CI has no migrate step, and addColumn swallows DDL errors with a console.warn. Every schema change therefore requires a second, manual postgres implementation in db.ts; forgetting it = green deploy, stale prod schema, manual psql session. The TODOs below make generated postgres migrations the single source of truth for prod, in dependency order:

  • [x] Add a schema-drift reconciliation script: db:check-drift (packages/server-core/src/scripts/check-schema-drift.ts) that connects to DATABASE_URL, introspects the live schema via information_schema (tables, columns, types, defaults, indexes), diffs against what postgresSchema in dbSchema.ts expects, and prints missing/extra/mismatched items with suggested DDL. Run it against prod by hand and fix any drift via psql BEFORE the baseline items below land — baseline adoption assumes prod matches dbSchema.ts. (Implemented with pure, unit-tested diff functions (14 tests): drift = missing tables/columns (CREATE/ALTER DDL emitted, NOT-NULL-without-default gets a backfill note), type mismatches (with USING casts), nullability both directions, missing declared uniques/PKs → exit 1; extra live tables/columns and default-presence diffs are informational only (prod legitimately carries legacy objects). Indexes: drizzle's pg schema declares none (the legacy idx_* ones come from hand DDL), so index drift is intentionally out of the verdict. Validated end-to-end against a throwaway Postgres 18 initialized by the legacy initializeDatabase() path, run twice to let its self-healing ordering bugs converge (first boot WARNS relation "bars" does not exist — the swallowed-error behavior this section exists to kill): 17 real drift findings remain even on a fully-converged legacy DB — e.g. tokens.token_artwork_id/original_image_url/mounted_on_token_id, equipment.custom_name/parent_id, all v0.18+ bars.* columns, roll_tables+roll_table_entries missing entirely; user_settings.enabled_modules, vbl_shapes.geometry, campaign_actions.target are text-vs-jsonb mismatches; slug uniques absent. Prod's actual drift is whatever subset your past manual psql sessions didn't already cover — ACTION REQUIRED: run DATABASE_URL=<prod> pnpm --filter @purple-cat/server-core db:check-drift and apply the suggested DDL before the baseline deploy.)
  • [x] Give postgres its own migrations dir + baseline: make drizzle.config.ts emit per-dialect output (sqlite keeps out: ./migrations; postgres gets ./migrations-pg with its own meta/_journal.json), add an explicit db:generate:pg script (DB_CLIENT=postgres drizzle-kit generate) since db:generate/db:push currently default to sqlite silently, then generate the initial postgres baseline migration capturing the full current postgresSchema. (Done, and it surfaced WHY generation has been dead for years: drizzle-kit (v0.31) only discovers tables exported at a module's top level — the object-bundled postgresSchema/sqliteSchema exports yield "0 tables" for BOTH dialects (probe-verified), which is why the journal is mostly hand-written. New src/dbSchema.pg.ts re-exports all 43 pg tables individually as the pg-dialect drizzle-kit entry (extensionless ./dbSchema import — the kit's CJS loader can't map .js specifiers to .ts sources; bundler moduleResolution accepts it), with a sync test (dbSchema.pg.test.ts, 3 tests) that fails listing the exact missing export if a table is added to postgresSchema without re-exporting. Second blocker removed: ~100 stale compiled artifacts (*.js/*.d.ts/*.js.map) were COMMITTED inside src/, shadowing module resolution for CJS/esbuild-based tools (drizzle-kit initially generated 15 tables from a months-old src/dbSchema.js); all deleted (every one had a .ts twin; the one genuine ambient file, src/types/express.d.ts, kept), .gitignore gains packages/server-core/src/**/*.js, and the five "dbSchema.js mirror" source-contract tests that pinned the deleted artifact were removed (their .ts assertions remain; net suite: 83 files / 772 tests green, stale .test.js dupes no longer run). migrations-pg/0000_baseline.sql (43 CREATE TABLEs + slug uniques incl. nulls not distinct) validated: applied to a fresh Postgres 18 and db:check-drift reports 0 drift / 0 informational vs dbSchema.ts.)
  • [x] Run the drizzle migrator in the postgres branch of initializeDatabase(): call migrate() from drizzle-orm/node-postgres/migrator with the pg migrations folder BEFORE the legacy functions. Brownfield adoption: if the DB already has app tables but no drizzle migrations table (note: the pg migrator keeps it in the drizzle.__drizzle_migrations schema/table by default), record the baseline as applied without executing its SQL (insert the journal row — drizzle stores a sha256 hash of the migration text + millis timestamp) so existing prod/staging DBs adopt cleanly. Wrap the whole step in pg_advisory_lock so concurrent ECS tasks can't race. A migration failure must throw and fail boot loudly — no warn-and-continue. (postgresRunMigrations() runs first in the pg branch: takes pg_advisory_lock(723202501) on a dedicated client (unlock in finally), runs postgresAdoptBaselineIfBrownfield() (gates: campaigns exists in information_schema AND drizzle.__drizzle_migrations is empty → inserts the baseline's exact journal row — sha256 of the raw .sql text + the journal when, matching drizzle's readMigrationFiles/dialect.migrate semantics, which gate on created_at < folderMillis of the LAST row), then migrate(); failures propagate and abort boot. Also rode along here (needed for the migrator to work at all in-container): resolvePgMigrationsFolder() resolves module-relative with two candidates — ../migrations-pg from src/ (tsx) and ../../../migrations-pg from dist/server-core/src/ (compiled) — and throws listing candidates when absent; TODO 4 below covers the Dockerfile COPY. Live-verified against a throwaway Postgres 18: fresh DB → baseline executed, journal row recorded, db:check-drift = 0 drift; legacy-DDL-built DB → "Brownfield adoption: recorded baseline 0000_baseline as applied (not executed)", no CREATE TABLE collisions; re-run → pure no-op. Legacy postgresCreateTables()/postgresApplyColumnMigrations() still run after the migrator this release (all IF NOT EXISTS; gated off next TODO+1). 6 source contracts (db.pgMigrator.test.ts); suite 84 files / 778 tests green.)
  • [x] Ship migrations in the prod image: resolve the migrations folder relative to the module file (import.meta.url), not process.cwd() — in the container cwd is /app so /app/migrations doesn't exist — and COPY packages/server-core/migrations-pg into the Dockerfile runtime stage. Verify the compiled dist path math works inside the image. (The module-relative resolvePgMigrationsFolder() landed with the previous item; this one adds the runtime-stage COPY --from=builder /app/packages/server-core/migrations-pg ./packages/server-core/migrations-pg. Dist path math verified WITHOUT docker (not available on this machine) by importing the actually-compiled dist/server-core/src/db.js from an unrelated cwd (/tmp): it resolves ../../../migrations-pg → <package root>/migrations-pg, exactly the image's /app/packages/server-core/migrations-pg. Side discovery while testing: importing the compiled module under the system Node 24 trips better-sqlite3's NODE_MODULE_VERSION (built for Node 20) — irrelevant to prod, which runs Node 20 and DB_CLIENT=postgres (the pg Pool is lazy, so module init makes no connection). +1 source contract pinning the Dockerfile COPY (db.pgMigrator.test.ts, 7 total).)
  • [x] Retire the legacy postgres DDL path: skip postgresCreateTables()/postgresApplyColumnMigrations() whenever the drizzle migrations table exists (i.e. the migrator is authoritative), keeping them reachable for one release via LEGACY_PG_DDL=1 as an emergency fallback, with a deletion TODO for v0.33.0. From this point a schema change = edit dbSchema.ts + db:generate:pg (+ the sqlite dev mirror) — no more hand-written postgres DDL in db.ts. (Gated purely on LEGACY_PG_DDL === "1" rather than a journal-existence check — after postgresRunMigrations() the journal always exists (it throws otherwise), so the check would be constant-true; TODO(v0.33.0) deletion marker in place. One non-DDL behavior was rescued from the legacy path before gating: the idempotent default-actions seeding (Inflict Damage/Heal × dnd5e-2014/2024/system8, ON CONFLICT (id) DO NOTHING) moved to postgresSeedDefaultActions(), which always runs — data, not schema. Live-verified: a fresh DB initialized WITHOUT the flag is pure-baseline (db:check-drift: 0 drift, 0 informational — the legacy-only tokens.pronouns/vbl_shapes extras are gone) and still gets the 6 seeded actions; LEGACY_PG_DDL=1 on a brownfield DB runs the old path after the migrator without errors. +2 source contracts (gate placement, unconditional seeding); suite 84 files / 781 tests green.)
  • [x] Add a CI drift guard: a job with a postgres service container that (1) runs the migrator from zero against an empty DB, (2) runs drizzle-kit generate (pg dialect) against the result and fails if a non-empty migration is produced — so a dbSchema.ts change without a checked-in pg migration fails the PR instead of failing in prod. Keep it out of the deploy path (lint-and-test job or a sibling). (New sibling migration-guard job (postgres:16-alpine service container, pg_isready health gate) with three checks: db:init (new script + CLI: boot-equivalent initializeDatabase() without starting the server) migrates the empty DB from zero — proves the committed SQL applies; db:check-drift against the result — proves migrations ≡ dbSchema.ts (note: drizzle-kit generate diffs schema vs its own snapshots, not vs a live DB, so this is the step that actually needs postgres); and db:generate:pg must leave migrations-pg/ untouched per git status --porcelain — proves no uncommitted migration, with an ::error:: annotation + diff excerpt when it fires. One deliberate scope addition: build-docker now needs: [lint-and-test, migration-guard] — an image whose migrations can't build the expected schema would fail every ECS task at boot, so it must not ship. Full three-step sequence simulated locally against the throwaway Postgres: migrate-from-zero OK, 0 drift, generate emits "No schema changes" and leaves the folder clean. Slack failure notification mirrors the other jobs.)
  • [x] Quarantine dead migration files: move every migrations/*.sql not listed in migrations/meta/_journal.json (most of 0003–0063, including the duplicate-numbered 0012/0029/0060 files) to docs/legacy-migrations/ so migrations/ only contains files a migrator will actually execute. Verify migrateSqlite still works on both a fresh and an existing dev sqlite DB afterwards. (53 dead files moved (journal-driven script), 14 journaled files remain; docs/legacy-migrations/README.md explains why they exist and why they must not move back. Verification surfaced a pre-existing landmine in the KEPT files: the hand-written journaled migrations (0007, 0010, 0028) lack drizzle's --> statement-breakpoint markers, and better-sqlite3 can't prepare multi-statement strings — migrateSqlite would have CRASHED on any DB that actually needed to apply them (it never fired because dev DBs take the fresh-create path or predate the journal). Markers inserted (hash changes are safe — the migrator gates on the journal when, not the hash). Verified with the dialect explicitly forced to sqlite (gotcha discovered en route: the root .env sets DB_CLIENT=postgres now, so "sqlite" boots silently run postgres unless DB_CLIENT is overridden — earlier session verifications hit this; see the workflow doc): (1) fresh DB boots and creates the schema, (2) a simulated legacy DB (full schema + populated __drizzle_migrations) boots with migrateSqlite executing against the slimmed folder, (3) a copy of the real 168 MB sqlite-era data/app.db boots clean. Note: the journal chain alone cannot build a DB from zero (e.g. nothing journaled creates characters) — that was always true; the fresh path owns creation for sqlite. 11 test files that pinned moved files as source contracts now read them from docs/legacy-migrations/; suite 84 files / 781 tests green.)
  • [x] Document the new schema-change workflow in CONTRIBUTING.md: edit dbSchema.ts → pnpm db:generate:pg → commit schema + migration together; note that migrations must stay additive/expand-contract (old tasks briefly run against the new schema during ECS rolling deploys); note the sqlite dev path still uses its hand-rolled mirror functions; describe what the CI drift guard enforces and how to fix a red run. (New "Database schema changes (v0.32.0+)" section: the 4-step recipe (including the dbSchema.pg.ts re-export step with its failing-test safety net, and the warning that plain db:generate/db:push silently target sqlite), the expand-contract constraint with the ECS rolling-deploy rationale, what each of the three migration-guard checks enforces + how to fix a red run, db:check-drift for reconciling long-lived DBs, and a Dev databases subsection covering the postgres-first .env (DB_CLIENT=postgres — local dev gets boot migrations + brownfield adoption like prod), the sqlite-fallback mirror obligation, and docs/legacy-migrations/ as reference-only.)