UnheardCity/docs/decisions.md

# Decisions

## 1. Stack: Python + Qt (PySide6), packaged with PyInstaller
- **Reuse:** the toolkit (`ackit`) is Python and proven against the real game; the editor imports it unchanged.
- **Cross-platform GUI:** Qt Widgets via PySide6 (LGPL-3.0, official Qt for Python) looks native on
  Windows and Linux, has a fast model/view table for 11,000+ rows, and runs offscreen for automated GUI tests.
- **Audio I/O:** Qt Multimedia (`QMediaDevices`, `QAudioSink`, `QAudioSource`) for playback, recording and
  device selection. It ships inside the PySide6 wheels with its own backends (WASAPI on Windows,
  PulseAudio/PipeWire on Linux), so unlike PortAudio/`sounddevice` nothing has to be installed on Linux.
- **Distribution:** PyInstaller one-folder build per platform. It bundles Python, Qt and all native libraries;
  the user unzips and runs `UnheardCity(.exe)`, or on Windows double-clicks `Install.cmd` (see below). Nothing
  else is installed. Alternatives considered:
  Tauri/Rust or C# would mean rewriting the verified codec/injector (high risk, no benefit); a local web UI
  would still need a bundled runtime and makes audio recording harder.

## 2. No external programs at runtime
The first toolkit used ffmpeg (decoding inputs and libvorbis encoding) and vgmstream (checks). Both are gone:
- **Encoding:** `libsndfile` (via the `soundfile` wheel) contains libvorbis 1.3.7. Tested: it yields exactly the
  same codebook compatibility and the same game setups as ffmpeg's libvorbis (findings 12). Encoding is in-process.
- **Decoding game WEMs:** ackit's own WEM -> Ogg converter (the game's codebooks) + libsndfile.
- **Input formats:** libsndfile reads WAV, FLAC, OGG/Vorbis/Opus, MP3, AIFF and more. M4A/AAC/WMA would need
  ffmpeg: an ffmpeg found on the system is used if present, otherwise the user gets a clear "convert to WAV/MP3"
  message. Bundling ffmpeg (~80-100 MB, GPL/LGPL build care) for rare formats was judged not worth it.
- **Resampling:** `soxr` (LGPL-2.1).
So there are **no vendor binaries to fetch** (no `tools/fetch_vendor` script was needed): every native
dependency comes from version-pinned PyPI wheels (which embed libsndfile, libvorbis, libogg, FLAC, opus, mpg123,
Qt and Qt Multimedia's FFmpeg backend) that PyInstaller bundles, resolved relative to the app (never `PATH`).
`tools/collect_licenses.py` writes `THIRD_PARTY_LICENSES.txt` into each build from the wheels' own license files
plus official texts kept in `tools/licenses/` (sources in SOURCES.txt; no network at build time). Verified: the packaged Windows build runs
its decode/encode/inject smoke test with `PATH` reduced to `C:\Windows` (no ffmpeg, vgmstream or Python).
**Licenses:** PySide6/Qt LGPL-3.0 (dynamic linking, unmodified Qt shared libraries in the one-folder build,
source at https://code.qt.io), libsndfile LGPL-2.1, mpg123 LGPL-2.1, soxr LGPL-2.1, libvorbis/libogg/FLAC/opus
BSD, numpy BSD, Python PSF. One-folder (not one-file) builds keep the LGPL libraries replaceable.
The only Windows-only part is the in-game test harness (`ackit verify --ingame`, uses WinDbg/cdb and WASAPI
loopback); it is a developer tool and not part of the editor.

## 3. Where originals live (revert safety)
- **Originals store** per game folder: `<user data>/UnheardCity/stores/<id>/` (Windows `%APPDATA%`, Linux
  `~/.local/share/unheardcity`), id = hash of the game path; configurable (`--store`, editor settings).
  Before a sound is first changed, its original WEM bytes (SHA-256-named blob) and metadata are saved;
  the applied replacement is saved too. **Revert** re-injects the stored original; **Revert all** does this for
  every record. Stream-pack padding bytes are stored so pack reverts are byte-identical.
- **Full-file fallback:** before the first change to a WAD (and its pack index), one verified full copy is kept
  in the store (`full/`), default on, can be disabled. Used by "Restore from full backup" if the store is damaged.
- **Why the user data folder:** it survives app updates and reinstalls, needs no write access next to the game,
  and works the same on Linux/Proton. Steam never touches it.
- **Safe writes:** every WAD is rebuilt into a temp file and atomically renamed over the original, pack indexes
  first. Several sounds of one WAD are written in one pass. A locked WAD (game running) gives a clear message.
- **Changed underneath:** each sound's state is computed from its bytes: original / modified (ours) / lost
  (ours was replaced by the original, e.g. Steam "verify") / changed (neither: update or another tool) /
  foreign (never touched by us, but not the catalog's original checksum). Writing over changed/foreign sounds
  requires explicit confirmation; the catalog is used only when the WAD fingerprint matches.

## 4. Size rule
Speech/ambience variations are addressed by offset+size from BKPK tables inside compressed `.upk` packages,
which ackit does not rewrite. Replacements are written in place and must not exceed the original WEM's size;
the encoder lowers quality, then sample rate, to fit, and refuses otherwise. Music stream packs have a loose
index (`SFX\*.bnk`) that ackit rewrites, so music can be any length. In-memory sounds follow the same rule
(below). Lifting the limit would mean re-laying out the uncompressed packages (export offsets, BKPK/DIDX
tables): high risk for little gain, not done.

## 4b. In-memory sounds (soundbanks in loose `.bnk` files and `.upk` packages)
- **Model:** one catalog row per Wwise media ID (archive `BANK`, target = media ID), not per bank: the same
  media is stored in up to dozens of bank copies (banks are cooked into every package that uses them), and the
  game may load any of them. A replacement is written into **every** copy (`BANK_locations.csv.gz`).
- **In place:** each copy keeps its size (zero-padded, like WAD speech); DIDX/HIRC offsets and the package
  layout never change, so nothing else in the engine data has to be rewritten.
- **Packages:** ackit reads the LZO packages itself (pure Python, ~47 MB/s, block-level access) and writes
  only the changed 128 KiB blocks, stored as a single LZO literal run (valid LZO1X without matches), then
  fixes the chunk table. No compressor and no external tool are needed; untouched blocks stay byte-identical.
- **Revert:** before the first change to a bank file a verified full copy goes to the store (mandatory, not
  optional like the WAD copies: the file is rebuilt from it). The file content is always "original copy +
  replacements applied now", so reverting the last one restores the original bytes exactly, and an interrupted
  operation is repaired by the next one. Conflicts: the whole-file SHA-1 must be the original's or the last one
  ackit wrote.
- **Prefetch heads:** 396 speech lines (about 79 per language) have their first bytes copied into banks; a
  replacement of such a line rewrites those heads too, otherwise the start would play the old line.
- **Names:** the hash list names none of these media. Names come from Wwise events whose names are in the
  packages' name tables (FNV-1 of the name = event ID), following event -> action -> containers -> sounds
  (6,416 sounds); the rest are numbered per bank. Categories use the bank name.

## 4c. Subtitles and menu text
- **Subtitles** are written in place into every copy of the dialogue object, NUL-padded inside the original
  string (the game stops at the first NUL), so nothing in the package moves; the limit is the original
  string's length ("room"). Moving the object to the end of the package (new export offset/size and an extra
  chunk) would lift the limit, but no dialogue is loaded on the title screen, so the result could not be
  checked automatically in the game: not done.
- **Links** subtitle -> speech sound come from the soundbanks (event -> ... -> per-language media -> WAD), not
  from names: the `_NN` suffixes do not follow the WEM order inside WAD entries.
- **Menu text** files are plain UTF-16 text: no size limit; only the edited values change.
- Both use the originals store like sounds: texts are recorded as strings (`texts` in store.json) and every
  file is rebuilt from its original copy + the applied edits (sound and text edits combine in one package).
- Keys (`SUB:<LANG>:<speech>:<line>`, `MENU:<LANG>:<File>:<Section>:<Key>`) are stable and used in mod packages.

## 4d. One language setting
The editor has a single *Language* list (top bar) for voices, subtitles and menu text, starting at the language
the game is set to on every launch (not remembered: the game's setting is what the player hears). The game
has voices in 5 languages and texts in 11; for the other 6 it plays the INT voices, so the editor loads the INT
voice lines with that language's texts and says so next to the list. Localized in-memory speech (one bank
copy per voice language) is filtered the same way, so the sound list never mixes languages.

## 5. Catalog
- Shipped CSVs per archive (names, categories, characters, original size + checksum) make loading instant and
  enable change detection; any other build is scanned at runtime (header reads only, cached).
- Verified speakers come only from the user's lists; name-based character inference is kept in a separate field.
- Speech sounds are named after the game's line (speech event), not after their position in the WAD entry:
  the WEMs of an entry are stored in media-ID order and every language has its own media IDs, so position i
  is a different line in each language (75 % of the multi-variation lines). Line names make names, speaker
  labels, mod packages and subtitles agree across languages.
- Category scheme: `docs/categories.md`.

## 6. Repository cleanup choices
- Windows installer: a per-user PowerShell installer shipped in the zip (`Install.cmd`) instead of Inno Setup or
  MSI, so building needs no extra tool; no admin rights, Start menu shortcut, Settings > Apps uninstall entry.
  The zip still runs without installing.
- One entry point: `ackit` (CLI) and `UnheardCity` (GUI). The `.bat` launchers and the `mods/` folder
  flow were removed (`ackit apply <folder>` and mod packages replace them).
- The user's third-party downloads that sat in `tools/` and `repositories/` (quickbms, Ravioli, AbdAshes' guide,
  vgmstream/ww2ogg/revorb sources) were moved to the git-ignored `external/` folder, since `tools/` now holds
  build scripts only.
- Experiment scripts and outputs were deleted; they are preserved in git tag `pre-cleanup`.
- The old `_backup/` (2.3 GB full copies) was deleted on 2026-10-08 with the game verified original; the originals
  store does not need it.

## 7. Editor UX choices
- One window: filters (left), sound table (top), editor of the selected sound (bottom). The primary flow
  (search -> select -> Space to listen -> Load file/Record -> Apply) needs no menus; power features (tags,
  favorites, multi-select revert, mod packages, settings) are in the toolbar and context menu.
- Long work (loading, encoding, writing 0.5 GB WADs) runs in background jobs with status messages; the
  window stays responsive. Errors are shown as plain sentences (game running/locked file, wrong folder,
  unsupported audio, too long, changed outside the tool).
- No network access and no telemetry. User notes/tags/favorites live in the user data folder.
- One look: neutral dark greys (no blue tint), a single dark red accent (#b0242c, theme and icon), solid panels. Icons are vector drawings in
  `app/icons.py` (no emoji, no icon fonts, no image files), so they look the same on every platform.
- **Artwork:** the app icon (the Arkham City logo bat in flat black over a red audio waveform of rounded bars, on a
  white rounded tile; shown only in the window/taskbar and the toolbar corner) and the default background wallpaper come from images supplied by the user.
  `tools/make_assets.py <folder>` turns `bac_logo_full.png` + `wallpaper.jpg` into `app/assets/` (app_icon.png,
  app_icon.ico, wallpaper.jpg), which the build bundles. The folder is committed (public artwork and generated
  content). Without the assets the app falls back to a plain waveform icon and no wallpaper.
- Background: the wallpaper sits only behind the filter panel, scaled to the panel height so its left part fills
  the panel, under a light veil. Settings -> Background toggles it and sets its strength (default 10 %).
- Loading feedback stays where the work is, never modal: a spinner inside the waveform being loaded (sound or
  replacement file, decoded off the GUI thread), next to the Apply/Revert buttons while encoding/writing, and in
  the status bar for game loading and bulk changes.
  All filter selectors (including Character) are plain, non-editable drop-downs so they look and behave alike.