Version 3.1.0: voice lines linked across all languages from the game data

NOTES.md

@@ -20,7 +20,7 @@
      `ackit/bnk.py` (bank parsing: DIDX/DATA/HIRC sources). Docs: formats/upk.md, banks.md. Findings: pure-Python
      LZO ~47 MB/s (== decompress.exe on every block); 15,800 media IDs in 96,011 copies, 396 are prefetch heads
      of WAD sounds (2,101 copies) -> 15,404 in-memory sounds; copies identical except 1C898731.
-  2. [x] Catalog archive `BANK` (15,404 sounds, 6,416 named by events) + `BANK_locations.csv.gz` (93,910 copies,
+  2. [x] Catalog archive `BANK` (15,404 sounds, 6,416 named by events, now 6,456) + `BANK_locations.csv.gz` (93,910 copies,
      2,101 prefetch heads of 396 speech lines); `ackit verify --catalog` (full: catalog OK; fixed 2 duplicate
      names per language: variation shown as NAME#i); categories for bank prefixes.
   3. [x] Workspace writes BANK sounds into every copy and keeps speech-line prefetch heads in sync (bank files
@@ -29,7 +29,7 @@
      audible check not possible without input: findings 18).
   4. [x] Subtitles = RDialogueEvent objects in packages (9,016 lines x 11 languages, 168k copies; parser
      `ackit/subtitles.py`, catalog SUB_<LANG>.csv + SUB_locations.csv.gz, 92 % linked to speech sounds via
-     events); menu text = Localization/<LANG>/*.int (`ackit/loctext.py`, ~7,000 strings/language).
+     events; now ~99 %); menu text = Localization/<LANG>/*.int (`ackit/loctext.py`, ~7,000 strings/language).
      `Workspace.texts` (ackit/texts.py): states, apply/revert, conflicts, mod packages; CLI `ackit text`.
      Subtitles in place (NUL-padded, limit = original length); decision + reasons in decisions.md 4c.
   5. [x] Editor: tabs Sounds / Subtitles / Menu text (app/textpanel.py: search, filters, edit with room
@@ -50,6 +50,27 @@
   gets a language column), verified 955/955 against dialogue speaker fields; RSubtitle + RSubtitleLookup
   (key = FNV-1 of the line name without Speech_) added; editor redesigned: Listen | Replace | Subtitle columns,
   editor spans the full window width (no scrolling at 1000x700 and up).
+- 2026-10-10 final check: build_catalog refuses to run on a modified game (`--accept-changes` after an update);
+  every test tool reverts in `finally`; mods from another language are not matched to position-named lines;
+  `tools/e2e_features.py` (8 real-game round trips) PASS; `--ingame --texts` also finds 23/23 Startup.upk
+  subtitle markers in memory; `--ingame`, `--banks`, e2e_editor PASS; `verify --catalog` OK; 124 tests; game
+  original after. Docs brought up to date.
+- 2026-10-10 (user: "make it complete for every language, fact check everything"): voice lines are identified
+  by the game data alone (`ackit/lines.py`, catalog column `line`): per-language BKPK tables (they were merged
+  by media ID, and 657 shared IDs let ESN overwrite the others), prefetch sources in the Sound-object map,
+  recordings stored twice written together, multi-language in-memory media, subtitle objects naming events.
+  12,434 lines, each with exactly one recording in all 5 voice languages; 3,366 speaker labels in every language;
+  subtitles link every version of a line; mods move between languages by line identity. `ackit verify
+  --catalog` re-derives all relations and prints facts (independent checks: bank language of every source,
+  prefetch heads = start of their recording, copies identical, speaker field of the dialogue objects, menu keys
+  per language). `tools/e2e_features.py --lang all` 78/78; `--ingame`, `--banks`, `--texts`, e2e_editor PASS;
+  127 tests; game original after. Speaker labels vs the dialogue speaker field: 2,766 agree, 19 differ (listed in
+  categories.md, for the user to check by listening; the source lists are never edited). The user's guide files
+  (`ArkhamCityModding/Audio/AbdAshes' ...`) hold no relation data: the hashlist equals ackit/data/hashlist.txt,
+  the scripts only unpack (wemscan = position names). `data/catalog/source/ita/Checkpoint.txt` was found deleted
+  (no code touches source/; cause unknown) and restored from git. Version 3.1.0 (minor):
+  dist/UnheardCity-3.1.0-windows-amd64.zip smoke PASS. e2e_features now deletes its temporary stores (GBs of
+  full copies) when everything reverted. NEXT: UI fixes the user will name.
 - NEXT (needs a person): play a mission with a replaced footstep/foley sound and an edited subtitle (REPORT.md).
 - OPEN: channel order for >2 channels unverified; speech and in-memory sounds *heard* in-game (need gameplay;
   posted keys do not drive the menu).
@@ -64,7 +85,7 @@
   `%USERPROFILE%\Desktop\ArkhamCityModding\Audio\49D84887_LOC_ITA\WAD\` (sha1 1238172C...). The game is clean now
   (fingerprints match the old `_backup/`, deleted on 2026-10-08).
 - Dev tools: `.venv` (Python 3.14.6, PySide6 6.10.3, soundfile 0.14.0 / libsndfile 1.2.2, soxr 1.1.0, numpy,
-  pytest, PyInstaller 6.22.3, soundcard 0.4.6). git at `C:\Program Files\Git\cmd`. WinDbg Store (path changes on
+  pytest, pyflakes, PyInstaller 6.22.3, soundcard 0.4.6). git at `C:\Program Files\Git\cmd`. WinDbg Store (path changes on
   update; `ackit/verify/debugger.py: find_cdb`). No WSL distro, no Docker.
 - User data (this PC): `%APPDATA%\UnheardCity\` (settings.json, stores\<id>\), cache `%LOCALAPPDATA%\UnheardCity\cache`.
 - The user's own material: `%USERPROFILE%\Desktop\ArkhamCityModding` (old mod work, Ghidra project, dump);
@@ -74,13 +95,16 @@
 ## Where to look
 - Findings, failed hypotheses, in-game experiment log E0-E14: `docs/reverse-engineering/findings.md`.
 - Formats: `docs/formats/`. Decoder addresses: `docs/reverse-engineering/decoder.md`.
-- Decisions (stack, deps, licenses, originals store, size rule): `docs/decisions.md`.
+- Decisions (stack, deps, licenses, originals store, size rule, catalog, verification): `docs/decisions.md`.
+- Voice lines across languages (one recording per language, from the game data): `docs/formats/banks.md`
+  (*Lines*), `ackit/lines.py`; checked by `ackit verify --catalog` (facts).
 
 ## Session log
 - 2026-10-08: user added `my_catalog_for_ita/` (per-character lists, wemscan names) -> moved to
   `data/catalog/source/ita/`; 3364 names labelled (2 wemscan name collisions label both candidates; one line had a
   `.wem.wav` typo). Built catalogs for ITA/INT/FRA/DEU/ESN/SFX (identical names and counts across languages).
-  Assumption: speakers are the same in every language, so ITA labels apply to all.
+  Assumption: speakers are the same in every language, so ITA labels apply to all. (Wrong for multi-variation
+  entries, whose WEM order differs per language: since 2026-10-10 labels go through the game's line names.)
 - 2026-10-08: libsndfile's Vorbis writer crashed (C stack overflow) on a 30 s x 6 ch single write -> block writes.
 - 2026-10-08: wad_fingerprint made structural (IDs + non-pack sizes) so music edits don't invalidate the catalog;
   scan mode never borrows checksums from the shipped CSV (other builds / synthetic tests).
@@ -89,8 +113,10 @@
   SoundFilter.refilter uses begin/endFilterChange when available.
 - 2026-10-08: search haystack must not contain the variation index (":2" matched "2"); IDs are matched separately
   (words of >= 6 chars against the target id).
-- Verification commands: `pytest -q`; `ackit verify`; `ackit verify --ingame`; `python tools/e2e_editor.py`
-  (needs Steam logged in); packaged: `dist/UnheardCity/UnheardCity.exe --smoke-test --report f.txt`.
+- Verification commands: `pytest -q`; `ackit verify`; `ackit verify --catalog`; `python tools/e2e_features.py
+  [--lang all]`;
+  `python tools/e2e_editor.py`; `ackit verify --ingame [--banks | --texts]` (needs Steam logged in); packaged:
+  `dist/UnheardCity/UnheardCity.exe --smoke-test --report f.txt`.
 - 2026-10-08: UI restyle on request: vector icon set (app/icons.py, no emoji), "Arkham" theme (app/theme.py: QSS +
   palette, translucent panels over a wallpaper backdrop).
 - 2026-10-08: user rejected the procedural icon/backdrop and supplied images (~/Downloads/Arkham City:
@@ -130,3 +156,12 @@
   stored in media-ID order, not _NN order): links come only from events. Numbered event names (Speech_X_04)
   live in export tables, not in name tables: the bank scan now collects export names (+40 named BANK sounds,
   links 71 % -> 92 %). Version*.int (build IDs) are not treated as UI text.
+- 2026-10-10 INCIDENT: a crashed e2e_features run (exception before its revert) left ITA:8108E77E:0 modified,
+  and a catalog rebuild then recorded the modified bytes. Recovered: reverted from the run's temporary store,
+  ITA WAD sha1 1238172C... again, every other full copy in that store equal to the game, ITA.csv/SUB_ITA.csv
+  restored from git and an identical rebuild. Prevention: build_catalog stops when a sound differs from the
+  catalog (`changed_since_catalog`), all test tools revert in `finally`, e2e_features writes only ORIGINAL sounds.
+- 2026-10-10: source lists of another language (`source/<lang>/`) are resolved through THAT language's WAD
+  positions; ~1,270 speech sounds per language have no game line name (efforts 519, radio/TV 139, cinematics
+  134) and keep position names (`Sound.is_position_name`): their labels stay per language and mod packages from
+  another language are not matched to them.