Files
UnheardCity/docs/decisions.md

10 KiB

Decisions

1. Stack: Python + Qt (PySide6), packaged with PyInstaller

  • Reuse: the toolkit (ackit) is Python and proven against the real game; the editor imports it unchanged.
  • Cross-platform GUI: Qt Widgets via PySide6 (LGPL-3.0, official Qt for Python) looks native on Windows and Linux, has a fast model/view table for 11,000+ rows, and runs offscreen for automated GUI tests.
  • Audio I/O: Qt Multimedia (QMediaDevices, QAudioSink, QAudioSource) for playback, recording and device selection. It ships inside the PySide6 wheels with its own backends (WASAPI on Windows, PulseAudio/PipeWire on Linux), so unlike PortAudio/sounddevice nothing has to be installed on Linux.
  • Distribution: PyInstaller one-folder build per platform. It bundles Python, Qt and all native libraries; the user unzips and runs UnheardCity(.exe), or on Windows double-clicks Install.cmd (see below). Nothing else is installed. Alternatives considered: Tauri/Rust or C# would mean rewriting the verified codec/injector (high risk, no benefit); a local web UI would still need a bundled runtime and makes audio recording harder.

2. No external programs at runtime

The first toolkit used ffmpeg (decoding inputs and libvorbis encoding) and vgmstream (checks). Both are gone:

  • Encoding: libsndfile (via the soundfile wheel) contains libvorbis 1.3.7. Tested: it yields exactly the same codebook compatibility and the same game setups as ffmpeg's libvorbis (findings 12). Encoding is in-process.
  • Decoding game WEMs: ackit's own WEM -> Ogg converter (the game's codebooks) + libsndfile.
  • Input formats: libsndfile reads WAV, FLAC, OGG/Vorbis/Opus, MP3, AIFF and more. M4A/AAC/WMA would need ffmpeg: an ffmpeg found on the system is used if present, otherwise the user gets a clear "convert to WAV/MP3" message. Bundling ffmpeg (~80-100 MB, GPL/LGPL build care) for rare formats was judged not worth it.
  • Resampling: soxr (LGPL-2.1). So there are no vendor binaries to fetch (no tools/fetch_vendor script was needed): every native dependency comes from version-pinned PyPI wheels (which embed libsndfile, libvorbis, libogg, FLAC, opus, mpg123, Qt and Qt Multimedia's FFmpeg backend) that PyInstaller bundles, resolved relative to the app (never PATH). tools/collect_licenses.py writes THIRD_PARTY_LICENSES.txt into each build from the wheels' own license files plus official texts kept in tools/licenses/ (sources in SOURCES.txt; no network at build time). Verified: the packaged Windows build runs its decode/encode/inject smoke test with PATH reduced to C:\Windows (no ffmpeg, vgmstream or Python). Licenses: PySide6/Qt LGPL-3.0 (dynamic linking, unmodified Qt shared libraries in the one-folder build, source at https://code.qt.io), libsndfile LGPL-2.1, mpg123 LGPL-2.1, soxr LGPL-2.1, libvorbis/libogg/FLAC/opus BSD, numpy BSD, Python PSF. One-folder (not one-file) builds keep the LGPL libraries replaceable. The only Windows-only part is the in-game test harness (ackit verify --ingame, uses WinDbg/cdb and WASAPI loopback); it is a developer tool and not part of the editor.

3. Where originals live (revert safety)

  • Originals store per game folder: <user data>/UnheardCity/stores/<id>/ (Windows %APPDATA%, Linux ~/.local/share/unheardcity), id = hash of the game path; configurable (--store, editor settings). Before a sound is first changed, its original WEM bytes (SHA-256-named blob) and metadata are saved; the applied replacement is saved too. Revert re-injects the stored original; Revert all does this for every record. Stream-pack padding bytes are stored so pack reverts are byte-identical.
  • Full-file fallback: before the first change to a WAD (and its pack index), one verified full copy is kept in the store (full/), default on, can be disabled. Used by "Restore from full backup" if the store is damaged.
  • Why the user data folder: it survives app updates and reinstalls, needs no write access next to the game, and works the same on Linux/Proton. Steam never touches it.
  • Safe writes: every WAD is rebuilt into a temp file and atomically renamed over the original, pack indexes first. Several sounds of one WAD are written in one pass. A locked WAD (game running) gives a clear message.
  • Changed underneath: each sound's state is computed from its bytes: original / modified (ours) / lost (ours was replaced by the original, e.g. Steam "verify") / changed (neither: update or another tool) / foreign (never touched by us, but not the catalog's original checksum). Writing over changed/foreign sounds requires explicit confirmation; the catalog is used only when the WAD fingerprint matches.

4. Size rule

Speech/ambience variations are addressed by offset+size from BKPK tables inside compressed .upk packages, which ackit does not rewrite. Replacements are written in place and must not exceed the original WEM's size; the encoder lowers quality, then sample rate, to fit, and refuses otherwise. Music stream packs have a loose index (SFX\*.bnk) that ackit rewrites, so music can be any length. In-memory sounds follow the same rule (below). Lifting the limit would mean re-laying out the uncompressed packages (export offsets, BKPK/DIDX tables): high risk for little gain, not done.

4b. In-memory sounds (soundbanks in loose .bnk files and .upk packages)

  • Model: one catalog row per Wwise media ID (archive BANK, target = media ID), not per bank: the same media is stored in up to dozens of bank copies (banks are cooked into every package that uses them), and the game may load any of them. A replacement is written into every copy (BANK_locations.csv.gz).
  • In place: each copy keeps its size (zero-padded, like WAD speech); DIDX/HIRC offsets and the package layout never change, so nothing else in the engine data has to be rewritten.
  • Packages: ackit reads the LZO packages itself (pure Python, ~47 MB/s, block-level access) and writes only the changed 128 KiB blocks, stored as a single LZO literal run (valid LZO1X without matches), then fixes the chunk table. No compressor and no external tool are needed; untouched blocks stay byte-identical.
  • Revert: before the first change to a bank file a verified full copy goes to the store (mandatory, not optional like the WAD copies: the file is rebuilt from it). The file content is always "original copy + replacements applied now", so reverting the last one restores the original bytes exactly, and an interrupted operation is repaired by the next one. Conflicts: the whole-file SHA-1 must be the original's or the last one ackit wrote.
  • Prefetch heads: 396 speech lines (about 79 per language) have their first bytes copied into banks; a replacement of such a line rewrites those heads too, otherwise the start would play the old line.
  • Names: the hash list names none of these media. Names come from Wwise events whose names are in the packages' name tables (FNV-1 of the name = event ID), following event -> action -> containers -> sounds (6,416 sounds); the rest are numbered per bank. Categories use the bank name.

5. Catalog

  • Shipped CSVs per archive (names, categories, characters, original size + checksum) make loading instant and enable change detection; any other build is scanned at runtime (header reads only, cached).
  • Verified speakers come only from the user's lists; name-based character inference is kept in a separate field.
  • Category scheme: docs/categories.md.

6. Repository cleanup choices

  • Windows installer: a per-user PowerShell installer shipped in the zip (Install.cmd) instead of Inno Setup or MSI, so building needs no extra tool; no admin rights, Start menu shortcut, Settings > Apps uninstall entry. The zip still runs without installing.
  • One entry point: ackit (CLI) and UnheardCity (GUI). The .bat launchers and the mods/ folder flow were removed (ackit apply <folder> and mod packages replace them).
  • The user's third-party downloads that sat in tools/ and repositories/ (quickbms, Ravioli, AbdAshes' guide, vgmstream/ww2ogg/revorb sources) were moved to the git-ignored external/ folder, since tools/ now holds build scripts only.
  • Experiment scripts and outputs were deleted; they are preserved in git tag pre-cleanup.
  • The old _backup/ (2.3 GB full copies) was deleted on 2026-10-08 with the game verified original; the originals store does not need it.

7. Editor UX choices

  • One window: filters (left), sound table (top), editor of the selected sound (bottom). The primary flow (search -> select -> Space to listen -> Load file/Record -> Apply) needs no menus; power features (tags, favorites, multi-select revert, mod packages, settings) are in the toolbar and context menu.
  • Long work (loading, encoding, writing 0.5 GB WADs) runs in background jobs with status messages; the window stays responsive. Errors are shown as plain sentences (game running/locked file, wrong folder, unsupported audio, too long, changed outside the tool).
  • No network access and no telemetry. User notes/tags/favorites live in the user data folder.
  • One look: neutral dark greys (no blue tint), a single dark red accent (#b0242c, theme and icon), solid panels. Icons are vector drawings in app/icons.py (no emoji, no icon fonts, no image files), so they look the same on every platform.
  • Artwork: the app icon (the Arkham City logo bat in flat black over a red audio waveform of rounded bars, on a white rounded tile; shown only in the window/taskbar and the toolbar corner) and the default background wallpaper come from images supplied by the user. tools/make_assets.py <folder> turns bac_logo_full.png + wallpaper.jpg into app/assets/ (app_icon.png, app_icon.ico, wallpaper.jpg), which the build bundles. The folder is committed (public artwork and generated content). Without the assets the app falls back to a plain waveform icon and no wallpaper.
  • Background: the wallpaper sits only behind the filter panel, scaled to the panel height so its left part fills the panel, under a light veil. Settings -> Background toggles it and sets its strength (default 10 %).
  • Loading feedback stays where the work is, never modal: a spinner inside the waveform being loaded (sound or replacement file, decoded off the GUI thread), next to the Apply/Revert buttons while encoding/writing, and in the status bar for game loading and bulk changes. All filter selectors (including Character) are plain, non-editable drop-downs so they look and behave alike.