feat(database): store gameplay replays in compact form
Build and Push Docker Container / build-and-push (push) Successful in 3m52s
Build and Push Docker Container / build-and-push (push) Successful in 3m52s
- Add compact storage package: codec, records, hydrate, replay blobs, cache. - Running games write game_turns and snake_states keyed by integer ids. - Encode moves as 1-byte codes and bodies as head plus step codes. - Store food and hazards only when they changed since the previous turn. - Strip reasoning fields that duplicate stored values and restore on read. - Pack finished games into one zstd blob per game in game_replays. - Merge late turns into packed blobs during the stale-game sweep. - Extract SQLite and PostgreSQL schemas into dedicated modules. - Add read/import game bundle and list_game_ids to the backend template. - Add legacy readers and converter for the old turns/snake_turns layout. - Add migrate_gameplay_to_compact.py with in-place or cross-DB migration. - Verify every migrated game and keep legacy rows on failure. - Support --since, --minimum-quality and guarded --drop-legacy. - Add just recipe migrate-gameplay-postgres. - Rewrite merge script to copy game by game via backend bundles. - Move benchmark state loading into benchmark_states for both backends. - Update PostgreSQL cleanup script to the new replay tables. - Drop sqlite-zstd build stages from the Dockerfile. - Remove old migration scripts, normalized_turn and their tests. - Add tests for compact storage, schema, legacy migration and PostgreSQL. - Document the compact layout and migration workflow in the README.
This commit is contained in:
-32
@@ -1,37 +1,5 @@
|
||||
# Stage 1 — compile sqlite-zstd + SQLite 3.49.1
|
||||
FROM ghcr.io/astral-sh/uv:python3.15-trixie-slim AS sqlite-zstd-builder
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl unzip git libzstd-dev pkg-config build-essential ca-certificates && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Compile SQLite 3.49.1 as a shared library so the runtime image can use it
|
||||
RUN curl -fsSL https://www.sqlite.org/2025/sqlite-amalgamation-3490100.zip -o /tmp/sqlite.zip && \
|
||||
unzip /tmp/sqlite.zip -d /tmp/ && \
|
||||
cd /tmp/sqlite-amalgamation-3490100 && \
|
||||
gcc -O2 -shared -fPIC -o /usr/local/lib/libsqlite3.so.0 sqlite3.c -ldl -lpthread
|
||||
|
||||
# Install Rust and compile sqlite-zstd (uses bundled SQLite 3.49.1 — no patching needed)
|
||||
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal
|
||||
ENV PATH="/root/.cargo/bin:${PATH}"
|
||||
|
||||
RUN git clone --depth=1 --branch v0.3.5 https://github.com/phiresky/sqlite-zstd.git /tmp/sqlite-zstd && \
|
||||
cd /tmp/sqlite-zstd && \
|
||||
cargo build --release --features build_extension && \
|
||||
cp target/release/libsqlite_zstd.so /usr/local/lib/libsqlite_zstd.so
|
||||
|
||||
# Stage 2 — runtime image
|
||||
FROM ghcr.io/astral-sh/uv:python3.15-trixie-slim
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends libzstd1 && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Replace system SQLite 3.46.1 with 3.49.1 so it matches the extension
|
||||
COPY --from=sqlite-zstd-builder /usr/local/lib/libsqlite3.so.0 /usr/local/lib/libsqlite3.so.0
|
||||
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf && ldconfig
|
||||
|
||||
COPY --from=sqlite-zstd-builder /usr/local/lib/libsqlite_zstd.so /usr/local/lib/libsqlite_zstd.so
|
||||
|
||||
# Install app
|
||||
COPY . /app
|
||||
WORKDIR /app
|
||||
|
||||
Reference in New Issue
Block a user