# NOTES — Arkham City audio toolkit & editor ## Current status / Next steps (rewrite as you go) - Mission 1 (2026-10-07): DONE. Format reversed, encoder + injector, verified in-game (no exe patch). - Mission 2 Part 1 (2026-10-08): repository cleanup DONE — tag `pre-cleanup` holds the old state. New layout (README "Where things are"); `ackit` refactored: portable encoder (libsndfile, no ffmpeg/vgmstream), `Game` (Steam/Linux/Proton discovery), catalog from `data/catalog/*.csv` (user's character lists + rules), originals store + `Workspace` (apply/revert/conflicts), mod packages, CLI. 63 tests pass; `ackit verify` PASS; `ackit verify --ingame` PASS (E14: ncc 1.0, revert byte-identical). - Part 2 (2026-10-08): Unheard City DONE (app/, PySide6): browser (search + group/category/character/ archive/tag/modified/favorites filters, sortable, multi-select, notes/tags/favorites), editor (original/installed/ replacement waveforms, A/B, loop, volume, file or recording, trim/gain/normalize/match loudness, apply/revert, conflict + changed-underneath handling), mod export/import, settings. 72 tests incl. offscreen GUI tests. Packaging: `python tools/build_app.py` -> dist/UnheardCity (self-contained, licenses); packaged exe passes `--smoke-test` with PATH = C:\Windows only. Real e2e through the editor code path (tools/e2e_editor.py): PASS (music heard in game ncc 1.0, reverts byte-identical, speech line apply/revert byte-identical). - Mission 3 (2026-10-09, brief: `docs/history/mission3_audio_text_prompt.md`): full-game audio, then subtitle and menu text editing. Plan (one commit per stage; a stage = code + tests + docs + this status): 1. [x] `ackit/lzo.py`, `ackit/upk.py` (UE3 package summary, compressed chunks, block-level read/patch), `ackit/bnk.py` (bank parsing: DIDX/DATA/HIRC sources). Docs: formats/upk.md, banks.md. Findings: pure-Python LZO ~47 MB/s (== decompress.exe on every block); 15,800 media IDs in 96,011 copies, 396 are prefetch heads of WAD sounds (2,101 copies) -> 15,404 in-memory sounds; copies identical except 1C898731. 2. [x] Catalog archive `BANK` (15,404 sounds, 6,416 named by `Play_` events; now 15,355, 13,501 named by events) + `BANK_locations.csv.gz` (93,910 copies, 2,101 prefetch heads of 396 speech lines); `ackit verify --catalog` (full: catalog OK; fixed 2 duplicate names per language: variation shown as NAME#i); categories for bank prefixes. 3. [x] Workspace writes BANK sounds into every copy and keeps speech-line prefetch heads in sync (bank files rebuilt from a mandatory full original copy); editor shows bank + copies. `ackit verify` PASS (300/300 BANK WEMs rebuilt identically); `ackit verify --ingame --banks` PASS (loaded, no crash, revert identical; audible check not possible without input: findings 18). 4. [x] Subtitles = RDialogueEvent objects in packages (9,016 lines x 11 languages, 168k copies; parser `ackit/subtitles.py`, catalog SUB_.csv + SUB_locations.csv.gz, 92 % linked to speech sounds via events; now ~99 %); menu text = Localization//*.int (`ackit/loctext.py`, ~7,000 strings/language). `Workspace.texts` (ackit/texts.py): states, apply/revert, conflicts, mod packages; CLI `ackit text`. Subtitles in place (NUL-padded, limit = original length); decision + reasons in decisions.md 4c. 5. [x] Editor: tabs Sounds / Subtitles / Menu text (app/textpanel.py: search, filters, edit with room counter, apply/revert/revert all, find & replace on the filtered rows, go to sound); the sound editor shows and edits the sound's subtitle (SubtitleBox); texts in mod export/import and the changed-outside banner; smoke test loads texts. 15 GUI tests. 6. [x] Final pass 2026-10-09: `ackit verify` PASS, `--catalog` OK, `--ingame` PASS (ncc 1.0), `--ingame --banks` PASS, `--ingame --texts` PASS (12/12 menu markers in memory), tools/e2e_editor.py PASS; game original after. Version 3.0.0; dist/UnheardCity-3.0.0-windows-amd64.zip (86 MB) smoke + GUI start PASS with PATH = C:\Windows. 121 tests. REPORT.md rewritten (unresolved items). - 2026-10-09 (after user test): in-memory speech is localized (one AkBank export = language list + one bank per language; ~1,460 lines per voice language): BANK.csv has a `language` column, the catalog shows shared sounds + the loaded language, names unique per language, subtitles of in-memory lines link to them (links 92 % -> 99 %). Editor: one Language setting at the top (always starts at the game's language; languages without voices use INT voices), text tabs follow it. - 2026-10-10 (user asked "is it really all in order?"): variation positions differ per language (75 % of lines) -> speech sounds named after the game's line (speech event), curated labels mapped through it (characters.csv gets a language column), verified 955/955 against dialogue speaker fields; RSubtitle + RSubtitleLookup (key = FNV-1 of the line name without Speech_) added; editor redesigned: Listen | Replace | Subtitle columns, editor spans the full window width (no scrolling at 1000x700 and up). - 2026-10-10 final check: build_catalog refuses to run on a modified game (`--accept-changes` after an update); every test tool reverts in `finally`; mods from another language are not matched to position-named lines; `tools/e2e_features.py` (8 real-game round trips) PASS; `--ingame --texts` also finds 23/23 Startup.upk subtitle markers in memory; `--ingame`, `--banks`, e2e_editor PASS; `verify --catalog` OK; 124 tests; game original after. Docs brought up to date. - 2026-10-10 (user: "make it complete for every language, fact check everything"): voice lines are identified by the game data alone (`ackit/lines.py`, catalog column `line`): per-language BKPK tables (they were merged by media ID, and 657 shared IDs let ESN overwrite the others), prefetch sources in the Sound-object map, recordings stored twice written together, multi-language in-memory media, subtitle objects naming events. 12,434 lines, each with exactly one recording in all 5 voice languages; 3,366 speaker labels in every language; subtitles link every version of a line; mods move between languages by line identity. `ackit verify --catalog` re-derives all relations and prints facts (independent checks: bank language of every source, prefetch heads = start of their recording, copies identical, speaker field of the dialogue objects, menu keys per language). `tools/e2e_features.py --lang all` 78/78; `--ingame`, `--banks`, `--texts`, e2e_editor PASS; 127 tests; game original after. Speaker labels vs the dialogue speaker field: 2,766 agree, 19 differ (listed in categories.md, for the user to check by listening; the source lists are never edited). The user's guide files (`ArkhamCityModding/Audio/AbdAshes' ...`) hold no relation data: their hashlist was ackit/data/hashlist.txt, the scripts only unpack (wemscan = position names). `data/catalog/source/ita/Checkpoint.txt` was found deleted (no code touches source/; cause unknown) and restored from git. Version 3.1.0 (minor): dist/UnheardCity-3.1.0-windows-amd64.zip smoke PASS. e2e_features now deletes its temporary stores (GBs of full copies) when everything reverted. - 2026-10-10 (user: order in the browser "messed up" after filter changes; redundant files): the proxy had dynamic sorting off, so rows a filter showed again were appended at the end -> dynamic sort on (+ GUI test). ackit/data/hashlist.txt removed: its 2,544 names the catalog uses are exactly the game's CookedBanks.txt bank names (WAD entry ID = FNV-1 of the bank name), so names now come from the game (`catalog.bank_names`); rebuilt catalogs identical. - 2026-10-10 (user: filtering/sorting choppy, add spinners like the audio preview): QSortFilterProxyModel replaced by `app/models.py: RowView` (Python filter + one sort over cached keys): sound list 1.3-1.5 s -> 10-20 ms per sort/filter, text lists 0.26-0.7 s -> ~10 ms. Spinner overlay (`spinner.BusyOverlay`) over the sound list while it loads and over the subtitle/menu lists, which now load as background jobs (a selection asked for meanwhile is applied when the rows arrive). GUI tests for order, selection and the overlays. - 2026-10-10 version 3.2.0: docs use INT (the game's English) for examples and figures; facts measured on this PC's Italian install stay labelled as such. User-guide screenshots in INT (`tools/screenshots.py --lang`, default INT; the app itself still starts in the game's language). Every WEM of the 5 speech WADs rebuilt from the rules: 54,079 / 54,080 identical (INT FE9963E8:74: avg-bytes/s field one lower, informational). Removed unused code (helpers/constants nothing called, the key-posting debugger helper, a double refilter after changes); old dist zips, logs/ and caches deleted. - 2026-10-10 final check (user: "is everything fact checked ... functioning ... documentation correct?"): every number in the docs recomputed from the catalog and a fresh game scan; stale ones fixed (positions differ for 81.2-81.6 %, not 75 %; 8,092 neutral BANK sounds; 24 unlinked subtitle names; 637 position-named lines; 14,802 package media; subtitle coverage 9 % combat barks, 23 % cinematics; speaker coverage by line). Bugs found and fixed (findings 23): 49 WAD recordings also stored whole in banks were separate BANK sounds (now bank copies of the line, written with it); 8 BANK/SFX name clashes; name lookup preferred no name over an alias (CLI "ambiguous" for e.g. `CHR_RadioScanner#1` in FRA). `ackit verify --lang` now applies (catalog: that language only; in-game: must be the game's language). Subtitle error message says Windows-1252, not Latin-1. Catalog rebuilt from the original files; `verify --catalog` OK (2,185 / 2,185 bank copies of WAD recordings identical), e2e_features --lang all 89/89, e2e_editor, --ingame/--banks/--texts PASS, 132 tests; game original after. Version 3.2.1: dist/UnheardCity-3.2.1-windows-amd64.zip smoke + GUI start PASS. - 2026-10-10 re-check (user: "fact check everything again"): the catalog rebuilt from the game is identical to the shipped one (every CSV and location index), and `verify --catalog` gives the same facts. Figures in the docs are now checked by `tools/check_docs.py` (catalog, index.json, menu files; `--scan`: bank survey + catalog-check facts; `--wems`: every WEM rebuilt); the build records the scan-only figures in index.json (names by event, dialogue objects, WAD lines named after a recording). It found what hand-copied figures had wrong: 61,510 "recordings" counted the 25 second places (61,485 recordings; the stat was fixed too), 81.2-81.6 % (not 81-82), the loose/embedded bank-name overlap, REPORT's test count, wem.md's SFX claim (626 / 651 rebuild, not "all but 4 looping tracks": 14 music seek tables, 5 music sample counts, 6 ambience `smpl` loops) and its extra-chunk count (722 WEMs, not 37). New test: cross-references between the shipped files. - 2026-10-10 user gameplay test: replaced voice lines and subtitles WORK in game. Batman footsteps do NOT: all 568 BANK sounds of the CHR_BatmanFootsteps* banks were replaced with Downloads\boom.mp3 (trimmed per sound, still APPLIED, with the user's own test subtitle SUB:ITA:Speech_CHURCH_B1_CH2_HQ_INTRO2_08; revert with `ackit revert --all` when the user says so). In game the boom was heard once, briefly; otherwise the original steps. Checked: every copy (40 files) holds the new bytes; replacements are loud (-8.8 dB RMS vs -20..-33); no other media ID has the same original audio; no game data outside CookedPCConsole. The named footstep events (Play_BM_Footstep_Walk/Run/Land/...) reach only the 46 media of bank CHR_BatmanFootsteps (replaced); the 522 surface media (CHR_BatmanFootsteps__Media, loose SFX\*.bnk) are reached by no named event (switch containers not followed by event_objects?). NEXT: find where the game gets the step audio it plays (switch/random containers and their children, banks loaded at runtime, a memory capture of a step in game). - 2026-10-10 footsteps solved (findings 24): in-memory replacements were zero-padded in their slot and the game plays that as silence; ackit now fills each in-memory copy exactly with a LIST/adtl/labl chunk (`wem.fill`, `Workspace.fits/fill_sizes/prepare`). Verified by the user in game (a beep on every step), e2e_features 89/89. Version 3.2.2. The footstep test was reverted (game original). - 2026-10-11 version 3.2.3: editor layout (user: controls "placed in line and go to new line, not pretty"): aligned grid for the edit fields, every button row fills its column, each column ends with a full-width action bar level with the others; A/B and Loop are icons next to the volume. Fits a 1000-px window without horizontal scrolling (minimum 984 px). Docs: recordings the game shares between languages (user saw English lines "leaking" into other languages: the game's own data, banks.md *Lines*). - 2026-10-13 (user: "full check of the source code for correctness, also against the documentation"): figures all matched (`check_docs --scan`), the formats code matched the docs; fixed: editor **Revert all** reverted sounds only (docs and uninstaller promised everything; the uninstaller also counted only sound records and could offer to delete the store while texts were modified) -> sounds + texts, uninstaller counts texts; concurrent background writes to one package lost an edit (reproduced) -> `Workspace.lock` serializes every write; editor mod import forced over conflicts silently -> asks; a second originals store (Settings / `--store`) copied a package modified by the first as its "original" (reproduced: a replaced sound survived both reverts) -> `Workspace.unoriginal_files` (catalog checksums, heads, subtitles; 1,447 files, no false alarm, ~15 s for all) makes it a conflict and the editor refuses a store change while changes exist; `ackit text --lang JPN` (text-only languages) failed -> `Game.voice_archive`; menu lookups reloaded all ~37 files per call (find & replace of "the" in INT: ~4 min frozen) -> one cached file per lookup (0.04 s for 1,193 keys); build_catalog's safety check ignored subtitles -> also compares them; `ackit restore-backup` added (decisions named a restore that nothing exposed; it also forgot WAD sounds without a full copy); recorder handles Int32/UInt8 input and split frames; stale docstrings, `ackit -h` examples, conflict message, case-insensitive Localization path in `verify --catalog`. 146 tests. During the checks an e2e_editor game launch failed ("code never decrypted") and its revert found the WAD still locked by the dying game (left Music_393BC53A modified; reverted by hand, SFX catalog OK): `debugger.kill_game` now waits until the game has exited. - 2026-10-13 second full check (user: "do this again to make sure"): codec (wwvorbis, codebooks, allocsize, ogg), classify vs categories.md, verify harness, tools and widgets re-read; fixed: `tools/e2e_editor.py` saved its own settings over the user's settings.json (lost store/device/background settings; it now uses the user's settings and never saves); `Game.language()` ignored languages without voices (a JPN game opened the editor and `ackit text` in INT; now the game's language, `Game.voice_archive` for its speech archive) and the tests read this PC's UserEngine.ini (conftest isolates it); a mod exported while another language was loaded lost name + line of that language's changes (`Workspace.sound_for_key`); down-mix dropped the extra channels (a 5.1 centre-only voice became silent stereo; now folded in at -3 dB, peak-limited); `Texts.for_sound` default for shared in-memory lines; package block cache locked (concurrent readers); `catalog._whole_recordings` -> `whole_recordings` in docs/comments; the user guide no longer sends app users to the CLI-only restore. 148 tests. - 2026-10-13 third check (property checks on the real game data): every menu file rewrites byte-identically, no ambiguous menu keys, every subtitle copy has the catalog's room; 2 of 103,658 subtitle strings could not be written back as stored: one line (`Speech_TAPE_J_Answer_1_00`) is UTF-8 bytes in a cp1252 string in KOR, POL, POR, RUS (the game's data; KOR's damaged), and the RUS/KOR ones contain the 5 bytes Python's cp1252 leaves undefined (decoded as U+FFFD, lossy) -> cp1252 as Windows maps it (lossless; all 103,658 reproduce), catalog rebuilt from the original game (`--accept-changes`: exactly those 2 rows changed), subtitles.md documents it (claim derived by check_docs). User guide: Space plays the replacement once one is loaded (not always the original); tooltip too. App smoke test on the real game PASS. - DONE by the user in game: replaced voice lines, edited subtitles and replaced footsteps (after the fill fix). - OPEN: channel order for >2 channels unverified; 19 speaker labels to check by listening; the Linux build was never built or run (no Linux here). ## Environment facts - Game: Steam AppID 200260 "Batman: Arkham City GOTY", `C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham City GOTY`, exe `Binaries\Win32\BatmanAC.exe` (32-bit, SteamStub). buildid 3718300. Game language: Italian (`Documents\WB Games\...\UserEngine.ini` is UTF-16, `Language=Ita`). - Steam must be running and logged in for the in-game test (`steam.exe -silent`, wait for HKCU\Software\Valve\Steam\ActiveProcess ActiveUser != 0). appmanifest AutoUpdateBehavior=0 (no updates expected). - History: the ITA WAD in the game folder was once the user's modded copy; the clean original came from `%USERPROFILE%\Desktop\ArkhamCityModding\Audio\49D84887_LOC_ITA\WAD\` (sha1 1238172C...). The game is clean now (fingerprints match the old `_backup/`, deleted on 2026-10-08). - Dev tools: `.venv` (Python 3.14.6, PySide6 6.10.3, soundfile 0.14.0 / libsndfile 1.2.2, soxr 1.1.0, numpy, pytest, pyflakes, PyInstaller 6.22.3, soundcard 0.4.6). git at `C:\Program Files\Git\cmd`. WinDbg Store (path changes on update; `ackit/verify/debugger.py: find_cdb`). No WSL distro, no Docker. - User data (this PC): `%APPDATA%\UnheardCity\` (settings.json, stores\\), cache `%LOCALAPPDATA%\UnheardCity\cache`. - The user's own material: `%USERPROFILE%\Desktop\ArkhamCityModding` (old mod work, Ghidra project, dump); their third-party downloads were moved from `tools/`, `repositories/` to `external/` (git-ignored). - Ghidra project BAC: `.text` Ghidra address = runtime + 0xC00 (docs use runtime addresses). ## Where to look - Findings, failed hypotheses, in-game experiment log E0-E14: `docs/reverse-engineering/findings.md`. - Formats: `docs/formats/`. Decoder addresses: `docs/reverse-engineering/decoder.md`. - Decisions (stack, deps, licenses, originals store, size rule, catalog, verification): `docs/decisions.md`. - Voice lines across languages (one recording per language, from the game data): `docs/formats/banks.md` (*Lines*), `ackit/lines.py`; checked by `ackit verify --catalog` (facts). ## Session log - 2026-10-08: user added `my_catalog_for_ita/` (per-character lists, wemscan names) -> moved to `data/catalog/source/ita/`; 3364 names labelled (2 wemscan name collisions label both candidates; one line had a `.wem.wav` typo). Built catalogs for ITA/INT/FRA/DEU/ESN/SFX (identical names and counts across languages). Assumption: speakers are the same in every language, so ITA labels apply to all. (Wrong for multi-variation entries, whose WEM order differs per language: since 2026-10-10 labels go through the game's line names.) - 2026-10-08: libsndfile's Vorbis writer crashed (C stack overflow) on a 30 s x 6 ch single write -> block writes. - 2026-10-08: wad_fingerprint made structural (IDs + non-pack sizes) so music edits don't invalidate the catalog; scan mode never borrows checksums from the shipped CSV (other builds / synthetic tests). - 2026-10-08: editor async loading looked stuck: Job objects were garbage-collected before their queued result signal arrived -> Runner keeps references until done. Qt 6.10 deprecates invalidateFilter/invalidateRowsFilter -> SoundFilter.refilter uses begin/endFilterChange when available. - 2026-10-08: search haystack must not contain the variation index (":2" matched "2"); IDs are matched separately (words of >= 6 chars against the target id). - Verification commands: `pytest -q`; `ackit verify`; `ackit verify --catalog`; `python tools/e2e_features.py [--lang all]`; `python tools/e2e_editor.py`; `ackit verify --ingame [--banks | --texts]` (needs Steam logged in); packaged: `dist/UnheardCity/UnheardCity.exe --smoke-test --report f.txt`. - 2026-10-08: UI restyle on request: vector icon set (app/icons.py, no emoji), "Arkham" theme (app/theme.py: QSS + palette, translucent panels over a wallpaper backdrop). - 2026-10-08: user rejected the procedural icon/backdrop and supplied images (~/Downloads/Arkham City: bac_logo_full.png, wallpaper.jpg). tools/make_assets.py builds git-ignored app/assets/ (icon = logo bat over a red waveform; wallpaper copied); app/art.py, make_icon.py and the old icon files removed. Settings -> Background toggle (background_enabled). Character filter is now a plain combobox like the others. - 2026-10-08: theme selector and custom background image removed (one look); the wallpaper is painted only in the filter panel (theme.Sidebar: scaled to panel height, left part shown). - 2026-10-08: build simplified to one offline command: build_app.py runs make_assets when app/assets is missing (--art FOLDER), and license texts live in tools/licenses/ (www.gnu.org was down; no more downloads). - 2026-10-08: neutral dark theme (no blue), editor header icon removed, new icon: flat BAC bat with echolocation arcs. build_app.py also regenerates app/assets when tools/make_assets.py is newer than the icon. - 2026-10-08: background strength default 10 %; obsolete settings keys (theme, background_image) dropped on load. Spinners (app/spinner.py): waveform overlay while loading, editor action row, status bar for global jobs. Replacement files now decode in a background job. - 2026-10-08: accent is now dark red (#b0242c; icons.ACCENT / theme.ACCENT, was AMBER). Icon: white rounded tile, red waveform bars, black logo bat on top. Filter panel opens at its maximum width (320 px, can only shrink). Sidebar veil alpha follows the background strength (at 10 % the wallpaper stays visible). - 2026-10-09: editor renamed "Unheard City" (exe/zip UnheardCity, launcher unheard-city, user data %APPDATA%\UnheardCity). Icon: 9 wider/taller bars (5 at <=24 px), bat 78% of the tile. Panel background #0c0c0c. - 2026-10-09: editor panel opens fitted to its contents (MainWindow.fit_editor; refits on layout changes until the divider is dragged). Windows installer: tools/installer (Install.cmd -> install.ps1/uninstall.ps1, per-user, HKCU uninstall entry), copied into the build by build_app.py; install/upgrade/uninstall cycle verified. User chose no new software (no Inno Setup). - 2026-10-09: app/assets is committed (user: public artwork and generated content, not Warner Bros. pictures). build_app.py regenerates it only when missing or with --art; after changing make_assets.py run it by hand. - 2026-10-09: SFX coverage check (read-only): the catalog covers every WAD sound, but ~15,800 in-memory WEMs in soundbanks are missing (docs/formats/banks.md). Packages were unpacked with Gildor's decompress.exe from `%USERPROFILE%\Desktop\ArkhamCityModding\Audio\decompress\` (outside the repo). User: document now, implement editing later. - 2026-10-09 (mission 3): BANK = one row per media ID, written into every copy in place (assumption: the game may load any copy; a partial write would be inconsistent). Package writes store changed blocks as LZO literal runs (no compressor needed). Bank-file conflicts use the whole-file SHA-1 (the cheap guid+chunk-table fingerprint missed same-size edits in tests). Name collisions in the WAD catalogs (2 per language) are shown as NAME#i. `soundcard` comes from the `verify` extra (was missing from the venv). - 2026-10-09 (mission 3, stage 4): subtitle links by name were wrong for most multi-variation entries (WEMs are stored in media-ID order, not _NN order): links come only from events. Numbered event names (Speech_X_04) live in export tables, not in name tables: the bank scan now collects export names (+40 named BANK sounds, links 71 % -> 92 %). Version*.int (build IDs) are not treated as UI text. - 2026-10-10 INCIDENT: a crashed e2e_features run (exception before its revert) left ITA:8108E77E:0 modified, and a catalog rebuild then recorded the modified bytes. Recovered: reverted from the run's temporary store, ITA WAD sha1 1238172C... again, every other full copy in that store equal to the game, ITA.csv/SUB_ITA.csv restored from git and an identical rebuild. Prevention: build_catalog stops when a sound differs from the catalog (`changed_since_catalog`), all test tools revert in `finally`, e2e_features writes only ORIGINAL sounds. - 2026-10-10: source lists of another language (`source//`) are resolved through THAT language's WAD positions; ~1,270 speech sounds per language have no game line name (efforts 519, radio/TV 139, cinematics 134) and keep position names (`Sound.is_position_name`): their labels stay per language and mod packages from another language are not matched to them. (Superseded the same day by the line identities of `ackit/lines.py`; `is_position_name` was removed.)