gora/UnheardCity
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f29af40b375ab4c675104c880c3e6a8a374c8da7
UnheardCity/NOTES.md

12 KiB
Raw Blame History

NOTES — Arkham City audio toolkit & editor

Current status / Next steps (rewrite as you go)

  • Mission 1 (2026-10-07): DONE. Format reversed, encoder + injector, verified in-game (no exe patch).
  • Mission 2 Part 1 (2026-10-08): repository cleanup DONE — tag pre-cleanup holds the old state. New layout (README "Where things are"); ackit refactored: portable encoder (libsndfile, no ffmpeg/vgmstream), Game (Steam/Linux/Proton discovery), catalog from data/catalog/*.csv (user's character lists + rules), originals store + Workspace (apply/revert/conflicts), mod packages, CLI. 63 tests pass; ackit verify PASS; ackit verify --ingame PASS (E14: ncc 1.0, revert byte-identical).
  • Part 2 (2026-10-08): Unheard City DONE (app/, PySide6): browser (search + group/category/character/ archive/tag/modified/favorites filters, sortable, multi-select, notes/tags/favorites), editor (original/installed/ replacement waveforms, A/B, loop, volume, file or recording, trim/gain/normalize/match loudness, apply/revert, conflict + changed-underneath handling), mod export/import, settings. 72 tests incl. offscreen GUI tests. Packaging: python tools/build_app.py -> dist/UnheardCity (self-contained, licenses); packaged exe passes --smoke-test with PATH = C:\Windows only. Real e2e through the editor code path (tools/e2e_editor.py): PASS (music heard in game ncc 1.0, reverts byte-identical, speech line apply/revert byte-identical).
  • Mission 3 (2026-10-09, brief: docs/history/mission3_audio_text_prompt.md): full-game audio, then subtitle and menu text editing. Plan (one commit per stage; a stage = code + tests + docs + this status):
    1. ackit/lzo.py, ackit/upk.py (UE3 package summary, compressed chunks, block-level read/patch), ackit/bnk.py (bank parsing: DIDX/DATA/HIRC sources). Docs: formats/upk.md, banks.md. Findings: pure-Python LZO ~47 MB/s (== decompress.exe on every block); 15,800 media IDs in 96,011 copies, 396 are prefetch heads of WAD sounds (2,101 copies) -> 15,404 in-memory sounds; copies identical except 1C898731.
    2. Catalog archive BANK (15,404 sounds, 6,416 named by events) + BANK_locations.csv.gz (93,910 copies, 2,101 prefetch heads of 396 speech lines); ackit verify --catalog (full: catalog OK; fixed 2 duplicate names per language: variation shown as NAME#i); categories for bank prefixes.
    3. Workspace writes BANK sounds into every copy and keeps speech-line prefetch heads in sync (bank files rebuilt from a mandatory full original copy); editor shows bank + copies. ackit verify PASS (300/300 BANK WEMs rebuilt identically); ackit verify --ingame --banks PASS (loaded, no crash, revert identical; audible check not possible without input: findings 18).
    4. Subtitles = RDialogueEvent objects in packages (9,016 lines x 11 languages, 168k copies; parser ackit/subtitles.py, catalog SUB_.csv + SUB_locations.csv.gz, 92 % linked to speech sounds via events); menu text = Localization//*.int (ackit/loctext.py, ~7,000 strings/language). Workspace.texts (ackit/texts.py): states, apply/revert, conflicts, mod packages; CLI ackit text. Subtitles in place (NUL-padded, limit = original length); decision + reasons in decisions.md 4c.
    5. Editor: tabs Sounds / Subtitles / Menu text (app/textpanel.py: search, filters, edit with room counter, apply/revert/revert all, find & replace on the filtered rows, go to sound); the sound editor shows and edits the sound's subtitle (SubtitleBox); texts in mod export/import and the changed-outside banner; smoke test loads texts. 15 GUI tests.
    6. Final pass 2026-10-09: ackit verify PASS, --catalog OK, --ingame PASS (ncc 1.0), --ingame --banks PASS, --ingame --texts PASS (12/12 menu markers in memory), tools/e2e_editor.py PASS; game original after. Version 3.0.0; dist/UnheardCity-3.0.0-windows-amd64.zip (86 MB) smoke + GUI start PASS with PATH = C:\Windows. 121 tests. REPORT.md rewritten (unresolved items).
  • 2026-10-09 (after user test): in-memory speech is localized (one AkBank export = language list + one bank per language; ~1,460 lines per voice language): BANK.csv has a language column, the catalog shows shared sounds + the loaded language, names unique per language, subtitles of in-memory lines link to them (links 92 % -> 99 %). Editor: one Language setting at the top (always starts at the game's language; languages without voices use INT voices), text tabs follow it.
  • NEXT (needs a person): play a mission with a replaced footstep/foley sound and an edited subtitle (REPORT.md).
  • OPEN: channel order for >2 channels unverified; speech and in-memory sounds heard in-game (need gameplay; posted keys do not drive the menu).

Environment facts

  • Game: Steam AppID 200260 "Batman: Arkham City GOTY", C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham City GOTY, exe Binaries\Win32\BatmanAC.exe (32-bit, SteamStub). buildid 3718300. Game language: Italian (Documents\WB Games\...\UserEngine.ini is UTF-16, Language=Ita).
  • Steam must be running and logged in for the in-game test (steam.exe -silent, wait for HKCU\Software\Valve\Steam\ActiveProcess ActiveUser != 0). appmanifest AutoUpdateBehavior=0 (no updates expected).
  • History: the ITA WAD in the game folder was once the user's modded copy; the clean original came from %USERPROFILE%\Desktop\ArkhamCityModding\Audio\49D84887_LOC_ITA\WAD\ (sha1 1238172C...). The game is clean now (fingerprints match the old _backup/, deleted on 2026-10-08).
  • Dev tools: .venv (Python 3.14.6, PySide6 6.10.3, soundfile 0.14.0 / libsndfile 1.2.2, soxr 1.1.0, numpy, pytest, PyInstaller 6.22.3, soundcard 0.4.6). git at C:\Program Files\Git\cmd. WinDbg Store (path changes on update; ackit/verify/debugger.py: find_cdb). No WSL distro, no Docker.
  • User data (this PC): %APPDATA%\UnheardCity\ (settings.json, stores<id>), cache %LOCALAPPDATA%\UnheardCity\cache.
  • The user's own material: %USERPROFILE%\Desktop\ArkhamCityModding (old mod work, Ghidra project, dump); their third-party downloads were moved from tools/, repositories/ to external/ (git-ignored).
  • Ghidra project BAC: .text Ghidra address = runtime + 0xC00 (docs use runtime addresses).

Where to look

  • Findings, failed hypotheses, in-game experiment log E0-E14: docs/reverse-engineering/findings.md.
  • Formats: docs/formats/. Decoder addresses: docs/reverse-engineering/decoder.md.
  • Decisions (stack, deps, licenses, originals store, size rule): docs/decisions.md.

Session log

  • 2026-10-08: user added my_catalog_for_ita/ (per-character lists, wemscan names) -> moved to data/catalog/source/ita/; 3364 names labelled (2 wemscan name collisions label both candidates; one line had a .wem.wav typo). Built catalogs for ITA/INT/FRA/DEU/ESN/SFX (identical names and counts across languages). Assumption: speakers are the same in every language, so ITA labels apply to all.
  • 2026-10-08: libsndfile's Vorbis writer crashed (C stack overflow) on a 30 s x 6 ch single write -> block writes.
  • 2026-10-08: wad_fingerprint made structural (IDs + non-pack sizes) so music edits don't invalidate the catalog; scan mode never borrows checksums from the shipped CSV (other builds / synthetic tests).
  • 2026-10-08: editor async loading looked stuck: Job objects were garbage-collected before their queued result signal arrived -> Runner keeps references until done. Qt 6.10 deprecates invalidateFilter/invalidateRowsFilter -> SoundFilter.refilter uses begin/endFilterChange when available.
  • 2026-10-08: search haystack must not contain the variation index (":2" matched "2"); IDs are matched separately (words of >= 6 chars against the target id).
  • Verification commands: pytest -q; ackit verify; ackit verify --ingame; python tools/e2e_editor.py (needs Steam logged in); packaged: dist/UnheardCity/UnheardCity.exe --smoke-test --report f.txt.
  • 2026-10-08: UI restyle on request: vector icon set (app/icons.py, no emoji), "Arkham" theme (app/theme.py: QSS + palette, translucent panels over a wallpaper backdrop).
  • 2026-10-08: user rejected the procedural icon/backdrop and supplied images (~/Downloads/Arkham City: bac_logo_full.png, wallpaper.jpg). tools/make_assets.py builds git-ignored app/assets/ (icon = logo bat over a red waveform; wallpaper copied); app/art.py, make_icon.py and the old icon files removed. Settings -> Background toggle (background_enabled). Character filter is now a plain combobox like the others.
  • 2026-10-08: theme selector and custom background image removed (one look); the wallpaper is painted only in the filter panel (theme.Sidebar: scaled to panel height, left part shown).
  • 2026-10-08: build simplified to one offline command: build_app.py runs make_assets when app/assets is missing (--art FOLDER), and license texts live in tools/licenses/ (www.gnu.org was down; no more downloads).
  • 2026-10-08: neutral dark theme (no blue), editor header icon removed, new icon: flat BAC bat with echolocation arcs. build_app.py also regenerates app/assets when tools/make_assets.py is newer than the icon.
  • 2026-10-08: background strength default 10 %; obsolete settings keys (theme, background_image) dropped on load. Spinners (app/spinner.py): waveform overlay while loading, editor action row, status bar for global jobs. Replacement files now decode in a background job.
  • 2026-10-08: accent is now dark red (#b0242c; icons.ACCENT / theme.ACCENT, was AMBER). Icon: white rounded tile, red waveform bars, black logo bat on top. Filter panel opens at its maximum width (320 px, can only shrink). Sidebar veil alpha follows the background strength (at 10 % the wallpaper stays visible).
  • 2026-10-09: editor renamed "Unheard City" (exe/zip UnheardCity, launcher unheard-city, user data %APPDATA%\UnheardCity). Icon: 9 wider/taller bars (5 at <=24 px), bat 78% of the tile. Panel background #0c0c0c.
  • 2026-10-09: editor panel opens fitted to its contents (MainWindow.fit_editor; refits on layout changes until the divider is dragged). Windows installer: tools/installer (Install.cmd -> install.ps1/uninstall.ps1, per-user, HKCU uninstall entry), copied into the build by build_app.py; install/upgrade/uninstall cycle verified. User chose no new software (no Inno Setup).
  • 2026-10-09: app/assets is committed (user: public artwork and generated content, not Warner Bros. pictures). build_app.py regenerates it only when missing or with --art; after changing make_assets.py run it by hand.
  • 2026-10-09: SFX coverage check (read-only): the catalog covers every WAD sound, but ~15,800 in-memory WEMs in soundbanks are missing (docs/formats/banks.md). Packages were unpacked with Gildor's decompress.exe from %USERPROFILE%\Desktop\ArkhamCityModding\Audio\decompress\ (outside the repo). User: document now, implement editing later.
  • 2026-10-09 (mission 3): BANK = one row per media ID, written into every copy in place (assumption: the game may load any copy; a partial write would be inconsistent). Package writes store changed blocks as LZO literal runs (no compressor needed). Bank-file conflicts use the whole-file SHA-1 (the cheap guid+chunk-table fingerprint missed same-size edits in tests). Name collisions in the WAD catalogs (2 per language) are shown as NAME#i. soundcard comes from the verify extra (was missing from the venv).
  • 2026-10-09 (mission 3, stage 4): subtitle links by name were wrong for most multi-variation entries (WEMs are stored in media-ID order, not _NN order): links come only from events. Numbered event names (Speech_X_04) live in export tables, not in name tables: the bank scan now collects export names (+40 named BANK sounds, links 71 % -> 92 %). Version*.int (build IDs) are not treated as UI text.
Reference in New Issue View Git Blame Copy Permalink