Version 3.4.0: Edit tab (edit list, large subtitle, replacements with effects and presets), Sounds tab only browses, ackit fx (pedalboard + own reverb) and replace --fx, first Linux build, catalog without a mod package

This commit is contained in:
2026-10-28 13:54:12 +01:00
parent a706dae419
commit 6de470174e
36 changed files with 2872 additions and 278 deletions

View File

@@ -1,7 +1,8 @@
# Building Unheard City
The build is a PyInstaller **one-folder** bundle: Python, Qt (incl. Qt Multimedia), libsndfile (libvorbis,
FLAC, opus, mpg123) and soxr are all inside; nothing else is installed or needed at runtime. Build on each
FLAC, opus, mpg123), soxr and pedalboard (the effects) are all inside; nothing else is installed or needed at
runtime. Build on each
target OS (no cross-compiling).
## Windows (x86-64)
@@ -42,6 +43,9 @@ QT_QPA_PLATFORM=offscreen pytest -q
python tools/build_app.py # -> dist/UnheardCity-<ver>-linux-x86_64.tar.gz
QT_QPA_PLATFORM=offscreen dist/UnheardCity/UnheardCity --smoke-test --game /path/to/game
```
First built with version 3.4.0 (Debian 13 in a container, Python 3.14.8 built from source because Debian has no
3.14 package): ~124 MB tar.gz, ~330 MB unpacked; the smoke test passes with `PATH=/usr/bin:/bin` and the GUI
starts.
Notes:
- The `soundfile` and `soxr` wheels for manylinux bundle libsndfile/libsoxr; the PySide6 wheels bundle Qt and
Qt Multimedia's FFmpeg backend (PulseAudio/PipeWire output). The target desktop only needs the usual X11/

View File

@@ -22,6 +22,13 @@ The first toolkit used ffmpeg (decoding inputs and libvorbis encoding) and vgmst
ffmpeg: an ffmpeg found on the system is used if present, otherwise the user gets a clear "convert to WAV/MP3"
message. Bundling ffmpeg (~80-100 MB, GPL/LGPL build care) for rare formats was judged not worth it.
- **Resampling:** `soxr` (LGPL-2.1).
- **Effects for replacements (any sound):** `pedalboard` (Spotify, GPL-3.0, compatible with this project's GPL-3.0; it
statically contains JUCE and the Rubber Band library): filters, distortion, codec, delay and pitch/formant
shifting (`ackit/fx.py`). The reverb is not pedalboard's (Freeverb sounded metallic): a generated room response
(early reflections + a 4-band noise tail, the highs decaying first) convolved with numpy's FFT. Chosen over
writing the DSP ourselves (user: "just use some libraries") and over scipy (no ready pitch shifter). Its
high/low-pass filters are first order only, so the voice-band cuts are Butterworth magnitudes applied with
numpy's FFT. Wheels for Python 3.14 on Windows and Linux (0.9.26); it needs only numpy.
So there are **no vendor binaries to fetch** (no `tools/fetch_vendor` script was needed): every native
dependency comes from version-pinned PyPI wheels (which embed libsndfile, libvorbis, libogg, FLAC, opus, mpg123,
Qt and Qt Multimedia's FFmpeg backend) that PyInstaller bundles, resolved relative to the app (never `PATH`).
@@ -30,7 +37,7 @@ plus official texts kept in `tools/licenses/` (sources in SOURCES.txt; no networ
its decode/encode/inject smoke test with `PATH` reduced to `C:\Windows` (no ffmpeg, vgmstream or Python).
**Licenses:** PySide6/Qt LGPL-3.0 (dynamic linking, unmodified Qt shared libraries in the one-folder build,
source at https://code.qt.io), libsndfile LGPL-2.1, mpg123 LGPL-2.1, soxr LGPL-2.1, libvorbis/libogg/FLAC/opus
BSD, numpy BSD, Python PSF. One-folder (not one-file) builds keep the LGPL libraries replaceable.
BSD, numpy BSD, pedalboard GPL-3.0, Python PSF. One-folder (not one-file) builds keep the LGPL libraries replaceable.
The only Windows-only part is the in-game test harness (`ackit verify --ingame`, uses WinDbg/cdb and WASAPI
loopback); it is a developer tool and not part of the editor.
@@ -162,15 +169,22 @@ copy per voice language) is filtered the same way, so the sound list never mixes
store does not need it.
## 7. Editor UX choices
- One window with three tabs. *Sounds*: filters (left) and sound table on top, the editor of the selected
sound below across the full width, in three resizable columns (Listen | Replace | Subtitle). Inside each
column the controls are aligned, not packed in rows that wrap: Replace's edit fields in a grid with one
label column, and every column's actions in one full-width bar at its bottom, level across the three
columns (Listen: Original | Installed | Stop; Replace: Apply to game, twice as wide, | Revert | Save WAV;
Subtitle: Apply | Revert). Every row of buttons fills its column's width (Load file / device / Record; Trim
silence / Replacement; Show in list). A/B and Loop are icon buttons with tooltips next to the volume. The editor fits a 1000-pixel-wide window without horizontal scrolling (measured:
minimum 984 px with the window's vertical scroll bar). *Subtitles* and *Menu text*: search, filters, table, editor. The primary flow
(search -> select -> Space to listen -> Load file/Record -> Apply) needs no menus; power features (tags,
- One window with four tabs; browsing and editing are separate (user, mission 4: "the browser for browsing, the
edit tab for editing, not both for both"). *Sounds*: filters (left), sound table, and below it a compact
preview (details, subtitle as text, original/installed waveforms with play buttons, **Edit** / **Add to edit
list**); no editing tools: details | player (Play original, Play installed, Stop together above the
waveforms) | Edit, Add to edit list. *Edit*: the edit list (sounds sent from the browser, kept in the user
data folder with each sound's replacement audio, trim/level values and effects; `app/edit.py`) | the editor
(`app/editor.py`, top to bottom: SUBTITLE large, with the original and the English line, room counter and
buttons on one row; LISTEN = original, installed and replacement, one row each with its play button, the
waveforms take the free height, transport row; REPLACEMENT = Record / input / level / Load file, trim and
level in an aligned grid) with its action bar (Previous | Apply to game | Revert | Save WAV | Next) pinned
below | the effects (`app/effects.py`: a preset list, the effect chain = one line per effect, tick = on, click
= its settings shown below: preset buttons and sliders with readable values, logarithmic for frequencies and
times). No scrolling at 1400 x 900 and 1280 x 800 (tested). Wording is generic (replacement, effects), not
dubbing-specific: the tab edits every kind of sound. *Subtitles* and *Menu text*:
search, filters, table, editor. The primary flow
(search -> select -> Space to listen -> Edit -> Record/Load file -> Apply -> Next) needs no menus; power features (tags,
favorites, multi-select revert, mod packages, settings) are in the toolbar and context menu.
- Long work (loading, encoding, writing 0.5 GB WADs) runs in background jobs with status messages; the
window stays responsive. Errors are shown as plain sentences (game running/locked file, wrong folder,

View File

@@ -10,11 +10,11 @@ Survey of the installed game (buildid 3718300), 2026-10-09.
| where | banks | unique in-memory WEMs |
|-------|-------|-----------------------|
| loose `SFX\*.bnk` (Batman/Catwoman footsteps, 32 surfaces each, + `TST_Prep_Media`) | 65 | 1,117 |
| banks embedded in `.upk` packages (in 1,432 of the 2,650 packages) | 3,340 | 14,802 |
| banks embedded in `.upk` packages (in 1,431 of the 2,648 packages) | 3,340 | 14,802 |
All of them are in the catalog (archive `BANK`).
The two sets share 119 media IDs: 15,800 unique media IDs in 96,011 places (bank copies). 396 of them are
The two sets share 119 media IDs: 15,800 unique media IDs in 95,887 places (bank copies). 396 of them are
**prefetch heads** of streamed WAD sounds and 49 are **whole copies** of WAD recordings (below), so there are
**15,355 in-memory sounds**. All copies of a
media ID are byte-identical except `1C898731` (two variants). The WADs, by comparison, are fully covered: the
@@ -47,7 +47,7 @@ HIRC Sound object (type 2) payload: u32 object id, u32 plugin id (`0x00040001` =
(0 = in memory, 1 = streamed, 2 = prefetch), u32 media id, u32 bank id of the media; for in-memory sources
then u32 offset, u32 size, where offset = `DATA` payload position (from the start of the bank) + DIDX offset
and size = DIDX size (checked on every in-memory source of the footstep banks and of `BmGame.upk`).
Source kinds over all bank copies: 115,861 in-memory sources of the bank's own media; 2,233 in-memory sources
Source kinds over all bank copies: 115,737 in-memory sources of the bank's own media; 2,233 in-memory sources
with bank id 0 whose media is not in the same bank (loaded by another bank); 133,075 streamed sources;
2,101 **prefetch** sources. Every streamed or prefetched media ID is in a BKPK table, i.e. a WAD sound
(`wad.md`). A prefetch source's media is also in the bank's DIDX: that in-memory copy is the **head** of the
@@ -106,8 +106,8 @@ and the game ties the recordings together itself:
- The packages are LZO-compressed UE3 packages (`upk.md`); ackit decompresses them itself and searches the
uncompressed data for `BKHD` (banks) and `BKPK` (WAD indexes). (The first survey used Gildor's
decompress.exe; `ackit/upk.py` gives identical data.)
- Bank blobs in packages: 41,798 for 3,340 bank IDs. Many banks are copied into several packages (e.g.
`CHR_CatwomanFoleySuit` 31 times, `CHR_BatmanFoleyBelt` 26). 302 bank IDs occur as several blobs with
- Bank blobs in packages: 41,797 for 3,340 bank IDs. Many banks are copied into several packages (e.g.
`CHR_CatwomanFoleySuit` 31 times, `CHR_BatmanFoleyBelt` 25). 302 bank IDs occur as several blobs with
*different* media subsets (often one WEM each, several per package), so a bank's media set is the union
of all its blobs.
- Largest: `CHR_RobinFoleyCape` 300 WEMs, `CHR_BatmanFoleyCape` 282, `CHR_Clayface` 263,

View File

@@ -1,6 +1,6 @@
# UE3 packages (`*.upk`)
Arkham City's 2,650 packages in `BmGame\CookedPCConsole` are Unreal Engine 3 packages, version 805,
Arkham City's 2,648 packages in `BmGame\CookedPCConsole` are Unreal Engine 3 packages, version 805,
licensee 101. 2,647 are fully LZO-compressed, 2 are stored uncompressed, and `UnrealX.upk` (512 bytes)
is not a readable package. ackit reads and patches them with `ackit/upk.py` and `ackit/lzo.py` (no external
tools). Soundbanks inside them: `banks.md`.

View File

@@ -0,0 +1,284 @@
# Plan: dubbing voice lines, with the game's voice effects
Status (2026-10-27): stage 0 done (section 9); the scope changed after it (section 10: manual effect controls,
no automatic profiles); stages 1-2 of section 10 done (effects engine, Dubbing tab). The user's answers are in section 8. Sections 1-7 are the original proposal, written
from the code, the docs and the shipped catalog CSVs; the steps marked **(game)** read the game files.
The game is also installed on the Linux machine (Steam Flatpak, mounted into the podman container at
`/home/gora/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Batman Arkham City GOTY`;
the container's HOME differs, so it is passed with `--game`), so the read-only stages can run there; in-game
checks still need the Windows PC.
Sections 3-7 name modules and files that were proposed but not built after the scope change (`ackit/fxdetect.py`,
`ackit/dub.py`, `data/catalog/FX.csv`, `fx_profiles.json`, `docs/formats/effects.md`); what was built is in
sections 10-11 and `NOTES.md`: `ackit/fx.py`, `app/edit.py`, `app/editor.py`, `app/effects.py`, `app/preview.py`.
## 1. The problem
A dub replaces a character's lines with new recordings. Some original lines are not dry: they sound like a
telephone or radio (band-limited EQ), a PA announcement over the city (echo, several speakers), a room (reverb),
a suit or a machine (Mr. Freeze). A dry dub next to them sounds wrong. Two questions:
1. **Where is the effect applied?** This decides everything:
- **Baked into the WEM** (applied in the studio, or by Wwise when the bank was built, *rendered* effect):
the dry dub loses it, so the dub must get the effect before it is written.
- **Applied by the game at run time** (a Wwise effect on the Sound, on a parent container or on a bus, an
environmental/reverb zone, attenuation, obstruction): the game also applies it to the replacement. The
dub must stay **dry**, or it gets the effect twice.
The user heard effects "in game and here": the ones heard *here* (the editor plays the WEM) are baked; the ones
heard only in game are run-time and need nothing.
2. **Which effects exist?** Unknown, so they have to be found in the game data (section 3).
## 2. What the catalog already shows (INT, measured on the shipped `data/catalog/INT.csv`)
Cheap signals that are already in the catalog (`rate`, `channels`) and point to baked effects:
| signal | lines | groups (examples) | likely effect |
|--------|-------|-------------------|---------------|
| 3 or more channels | 858 WAD speech rows (mostly 5 ch, then 4 and 3 ch), plus 86 in-memory 5 ch | Predator chatter 251, story 211, Riddler 90: `PRED_RIDDLER`, `PRED_J`, `PRED_STRANGE`, `PRED_MAYOR`, `RIDDLER_R`, `GAME_OVER`, `COURT_B`, `UNDER_S` | PA / hacked-speaker broadcasts: one voice spread over several channels, probably with delays (echo) between them |
| 32,000 Hz | 110 (102 are `CHAR_FREEZE`) | Mr. Freeze | suit voice filter |
| 11,025 / 12,000 Hz | 91 | `CHR_RadioScanner_*` | police radio (band-limited) |
| 6,000 Hz | 25 | `PSO_CarRadio_Chatter_*` | car radio |
| category *Radio & TV* (`Speech_RADIO`, `Speech_TV`, `Speech_CAMERAS`, `DIA_`) | see `categories.md` | Vicki Vale reports, TV, cameras | radio/TV EQ, maybe noise |
These are hypotheses until someone listens and the metadata scan (3a) confirms them. Two things follow already:
- A mono dub of a 5-channel line is copied to every channel today (`audio.to_channels`): the spatial echo between
the channels is lost. Multichannel lines need a per-channel layout (gain + delay per channel) in their effect.
- Low-rate lines limit the band themselves (6 kHz = nothing above 3 kHz), so part of the "radio" sound is just
the encode rate; the rest (low cut, mid boost, distortion, noise) has to be added.
## 3. Finding every effect the game uses
Three independent sources, from exact to fuzzy. Every result goes into the catalog, so the editor can say
for each line: *baked: Radio EQ (from metadata)*, *baked?: echo, 3 channels (analysis, 0.8)*, *game adds: room
reverb (run time, record dry)*.
### 3a. Wwise metadata (exact, where it exists) **(game)**
The banks (version 56, Wwise 2011.1) describe effects in HIRC; `ackit/bnk.py` reads only the Sound sources today.
To add (`ackit/bnk.py` + a survey tool, like the bank survey of mission 3):
- Parse the **node base parameters** of Sounds, containers, actor-mixers and buses: the effect list (slot,
effect object ID, and a **"rendered"** flag in this bank version, per wwiser's v56 layout, to confirm on the
game's banks), the parent ID (effects are inherited unless the child overrides them), the output bus.
- Parse the **effect objects** (HIRC effect / share-set type; the type number for v56 to confirm): plugin ID
(Audiokinetic Parametric EQ, Delay, RoomVerb, Matrix Reverb, Peak Limiter, Compressor, Expander, Pitch
Shifter, Flanger, Guitar Distortion, Tremolo, ...; plugin ID = company + type + number) and its **parameters**
(EQ band frequencies/gains/Q, delay time and feedback, reverb time ...). The `FXPR` chunk the parser already
recognizes and `ENVS` in `Init.bnk` (environments, i.e. run-time reverb zones) belong here too.
- Walk each line's Sound object up the parent chain to the master bus: list of `(effect, plugin, params,
where: sound/parent/bus, rendered or run time)`.
- Line identity helps: Sound objects are the same in every language (`banks.md` *Lines*), so a line's chain is
the same in all 5 languages.
Expected result: run-time effects (bus reverbs, environments) are fully known, with parameters. Rendered
effects are known **with their exact parameters**, which is the best case for a dub. Effects applied in the
studio before the files reached Wwise leave **no metadata at all**: likely many of the radio/PA lines. Those need 3b.
### 3b. Signal analysis of the WEMs (no metadata needed) **(game, ~minutes over all speech)**
`ackit/fxdetect.py` (numpy only), per recording, a small feature vector:
- **Band limits**: frequency below which / above which 1 % of the energy lies (telephone ~300-3,400 Hz), steepness.
- **Effect EQ curve**: long-term average spectrum (LTAS) of the line divided by the LTAS of the **same speaker's
dry lines** (speaker from `characters.csv` / the line's speaker). The voice cancels out and what is left is the
effect's frequency response: directly usable as the dub's EQ.
- **Reverb**: energy decay after speech offsets (Schroeder-style decay fit -> RT60 estimate), direct-to-
reverberant ratio, tail length after the last word.
- **Echo / delay**: peaks in the autocorrelation of the onset envelope, or in the cepstrum, at lags 30-800 ms;
for multichannel lines the cross-correlation **between channels** (delay and gain per channel).
- **Distortion / noise**: crest factor, clipping, spectral flatness of the gaps between words (static, hiss),
harmonics added to a narrow band.
- **Modulation / pitch**: envelope modulation 2-20 Hz (tremolo), pitch and formant shift against the speaker's
dry lines (masks, Freeze, Clayface, synthesized voices).
Then **cluster** the vectors (e.g. k-means on normalized features, or simple thresholds first) and **listen to a
few lines per cluster** (the user names it: "Radio scanner", "Freeze suit", "Strange PA", "Riddler hacked
speakers", "Interview tape", ...). The names and cluster labels go into a profile table (3d). Cross-language
check: the 5 recordings of one line must land in the same cluster; disagreements are lines to listen to.
### 3c. Comparing with the game's output (optional, run-time effects only) **(game)**
The in-game harness can already play sounds and record the loopback (`ackit/verify/ingame.py`, `loopback.py`).
For a sample of lines with a run-time effect, the same features computed on the game's output vs on the WEM
show what the game adds. **Never save raw captures** (CLAUDE.md: they can contain voice chat): compute the
features in memory and keep only the numbers. Useful to confirm, not needed for the dub itself.
### 3d. What gets stored
- `data/catalog/FX.csv`: per line (`line` identity, language-independent where the metadata says so): `profile`,
`baked` (yes / no / unknown), `source` (metadata / analysis / manual), `confidence`, `runtime` (what the game
adds, for the "record dry" hint). Built by `tools/build_catalog.py` from the original files only, like the rest.
- `data/catalog/fx_profiles.json`: the profiles (section 4), with where each parameter came from.
- `docs/formats/effects.md` (format of the HIRC effect data, survey figures) and claims in `tools/check_docs.py`
for every figure; `ackit verify --catalog` re-derives the metadata part.
- Manual labels of the user live in user data (like notes/tags), never in `source/`.
## 4. Re-creating the effects on a dub
An **effect profile** = a chain of simple, deterministic processors with parameters (data, not code), rendered by
`ackit/fx.py`:
| block | parameters | source of the numbers |
|-------|------------|-----------------------|
| high-pass / low-pass / peaking / shelf (biquads) | freq, gain, Q | metadata EQ, or fitted to the 3b EQ curve |
| matched EQ (FIR) | the 3b curve itself | analysis (generic fallback for any baked EQ) |
| delay / echo | time, feedback, mix, per-channel delays/gains | metadata, or 3b autocorrelation / inter-channel lags |
| reverb | RT60, pre-delay, damping, mix | metadata (RoomVerb), or 3b decay fit; convolution with a generated IR (filtered, exponentially decaying noise) keeps it deterministic |
| distortion / bit-crush / noise bed | drive, bits, noise level and colour | 3b, tuned by ear |
| compressor / limiter | threshold, ratio | metadata, or by ear |
| channel layout | per-channel gain + delay (replaces the plain mono copy for 3-5 channel lines) | 3b inter-channel analysis |
The rendered dub then goes through the existing chain: loudness matched to the original, encoded at the
original's rate/channels, fitted to the size limit (`Workspace.prepare`). A reverb or echo tail makes the audio
longer: the editor's length warning and size fitting already cover that, but a dub UI should show the room.
**DSP dependency** (user: "just use some libraries", no own DSP, no VST hosting):
- Effects: **pedalboard** (Spotify, GPL-3.0, compatible with this project's GPL-3.0): ready filters (high/low-
pass, peak, shelf), reverb, delay, chorus, phaser, distortion, bit-crush, GSM compression (telephone),
compressor/limiter, pitch shift. A profile is a list of pedalboard plugins with parameters; the per-channel
layout (gain + delay per channel) is a few numpy lines. To check before stage 1: wheels for the project's Python
(3.14) on Windows and Linux, and that PyInstaller bundles it.
- Analysis (3b): numpy (FFT, correlations), scipy only if a step really needs it.
- Decision goes into `docs/decisions.md` like the other dependencies, licenses into `THIRD_PARTY_LICENSES.txt`.
## 5. Where to build it: new app or the existing editor?
| | Separate dubbing app on `ackit` | Inside Unheard City (new **Dubbing** tab) |
|-|-------------------------------|------------------------------------------|
| Write safety | `Workspace.lock` is a `threading.RLock`: it serializes writes **inside one process** only. Two apps open on the same game and store can write one package at the same time (the lost-edit bug of 2026-10-13, across processes). Needs a cross-process file lock + "changed outside" handling in both apps. | Same process, same lock, same store, same changed-outside banner: nothing new. |
| Reuse | Re-implements catalog loading (background jobs, spinners), language setting, line identity, subtitles, waveform, player/recorder, encode/fit, apply/revert, mod export/import, settings, packaging. | All of that exists (`app/editor.py`, `audioio.py`, `waveform.py`, `textpanel.py`, `modpack.py`). |
| Users | Two programs, two installs, two settings files, two originals stores to keep apart. | One program; a dub is just another kind of change (and a mod package). |
| Complexity | The editor stays as it is. | `mainwindow.py` (774 lines) and `editor.py` (590) must not grow into one blob: the tab is its own package. |
| UX | Free to design a session-based workflow. | Same freedom inside a tab; the Sounds tab stays for one-off replacements. |
**Recommendation: build it into Unheard City, with the engine in `ackit`.**
- `ackit/fx.py` (profiles + rendering), `ackit/fxdetect.py` (features, clustering), `ackit/dub.py` (dub
sessions/takes: files, states; no Qt), CLI `ackit fx survey | scan | render` and `ackit dub ...`: all headless,
testable, usable from scripts.
- `app/dubbing/` (own package): the **Dubbing** tab.
- The **Sounds** editor gets one small row: *Effect: [profile ▾] (detected: Radio EQ, metadata)* and a *game
adds ... record dry* hint, so single replacements benefit too.
- A separate program makes sense only later, if voice actors record **without the game** (remote dubbing): a
small recorder that reads a script export and writes a take pack (WAVs + JSON), which Unheard City imports.
It never touches game files, so the lock problem does not arise.
## 6. The Dubbing tab (workflow)
1. **Session**: language + character (or a filter: category, a list of lines). Lists the character's lines via the
line identity; shows the script (subtitle of the loaded language, the INT text as reference), the original's
length, the effect profile and baked/run-time status, and a state per line: *to do / recorded / approved /
applied*. Sessions and takes are stored in user data (`%APPDATA%\UnheardCity\dubs\...`), never in the game.
2. **Record**: play the original as a cue, count-in, record, auto-trim silence, several takes per line, pick one;
length vs the original (and optional time-stretch to fit, if pedalboard or a WSOLA stretch is added).
3. **Process**: clean-up (high-pass, gate), loudness match to the original (exists), the effect profile
(suggested from FX.csv, editable, A/B against the original with and without the effect).
4. **Apply** the approved lines in one `Workspace.apply` call (it takes a list); **export** the session as a mod
package (`modpack.export_mod`, with edited subtitles if the dub's wording changed, within their room).
5. Non-destructive: raw takes + parameters are kept, so a changed profile re-renders every line.
Warnings the tab must show (from the existing data):
- Lines the game shares between languages (167 in-memory voice lines stored once for every language; 616-651 WAD
lines per language identical to INT, `banks.md` *Lines*): dubbing one changes it for every language that uses it.
- Size limit (speech in place, never larger than the original WEM): long takes plus reverb tails get squeezed or
refused; show the room.
- Subtitle room (NUL-padded in place).
- Lines without a speaker label or with a disputed one (19 listed in `categories.md`).
## 7. Stages (one commit each: code + tests + docs + NOTES status)
0. **Evidence (game)**: HIRC effect survey (3a) and a first feature scan (3b) over all speech; listen to samples
per cluster with the user; write `docs/formats/effects.md` with figures (+ `check_docs` claims). Outcome: how
many lines are baked vs run time, and the list of profiles. **This decides the size of everything after it.**
1. `ackit/fx.py` on pedalboard: profiles JSON, render; tests with synthetic signals (an EQ's
measured response, a delay's lag, an RT60 measured back from the rendered reverb).
2. `ackit/fxdetect.py` + `FX.csv` / `fx_profiles.json` in `build_catalog.py`; `verify --catalog` checks; CLI.
Test: render a profile onto a dry synthetic voice, detect it back.
3. Sounds editor: effect row + run-time hint; GUI tests.
4. `ackit/dub.py` + Dubbing tab (sessions, takes, script, batch apply, mod export); GUI tests.
5. Real-game round trips (`tools/e2e_features.py` / `e2e_editor.py` cases for a dubbed line with an effect,
reverted in `finally`), the user listens in game, packaging (`build_app.py`, licenses), version 3.4.0 or 4.0.0.
## 8. Decisions (user, 2026-10-27)
- **All characters** can be dubbed, from the existing catalogs (no per-character priority): every effect group
found in stage 0 needs a profile.
- **Approximate effects** are enough; profiles are tuned later by ear (the Riddler lines' effect is complex).
- **Libraries, not own DSP**: pedalboard for effects (section 4); no VST hosting, no time stretching for now.
- **Everyone has the game**: no separate recorder or take packs; everything lives in Unheard City (section 5).
- Still open, out of scope unless asked: audio inside video files (pre-rendered movies carry their own sound
track, not in the WADs or banks; to check whether the game has any with speech).
## 9. Stage 0 results (2026-10-27, INT, read-only)
Measured with scratch scripts (HIRC survey of every bank, signal features of 12,430 INT speech recordings),
then the user listened to 77 candidate groups (`external/fx_samples/INDEX.md`, git-ignored). Figures here are a
record of that day, not re-checked by `tools/check_docs.py`.
**Metadata (Wwise, v56).** Layouts from wwiser: node base params = u8 override-parent-fx, u8 count, [u8 bypass,
count x {u8 slot, u32 fx id, u8 share-set, u8 rendered}], u32 output bus, u32 parent; a Sound has them after its
source (+ u8 source bits); a bus (v56) has u8 is-environmental at payload offset 33, ducks of 17 bytes, then its
effects. Effect objects are HIRC types 0x12 (share-set) / 0x13 (custom): u32 plugin, u32 size, parameters.
- Run time (the game adds them; the files are clean, so a dub is recorded **dry**): McDSP FutzBox share-set
`4272216A` (in 1,125 speech banks, on 464 actor-mixer slots, RTPC-driven) -> 2,858 speech rows; Parametric EQ
on a bus (30 rows) or on the sound (3); the ML1 limiter on the top bus (everything); 84 RoomVerb + 6 Matrix
Reverb share-sets in `Init` = the environments the game switches per area.
- Rendered (baked by Wwise): 33 speech rows (car radio 25, radio scanner 3, museum thug-to-Penguin 4, Steel HQ
1); the slot's fx id is zeroed, so the effect type is lost.
- Everything else that is baked was made in the studio and has no metadata.
**Channels.** Speech WEMs with 2-5 channels carry the WAVE channel mask: 2 ch `0x3` (L R), 3 ch `0x7` (L R C),
4 ch `0x33` (L R SL SR), 5 ch `0x37` (L R C SL SR); ackit decodes and encodes them in that order. PA lines put
the dry voice in C and an echo 160-173 ms later in the surrounds (Harley/HQ 121 ms; Freeze's quad rears 73 ms);
Riddler, Ivy and Ra's 5-channel lines are processed differently per channel.
**What the user heard** (per group; "no effects" groups omitted: thug/Ninja/Catwoman/JR efforts, combat barks,
Riddler `RIDDLER_R`, DLC `PDLC_PDLC`/`PDLC_EMOTES`/Robin/Harley, Outcast, Penguin `MUSEUM_B`, Steel/Church Harley,
Bully hobo, Talia `UNDER_B`, `CH_ZERO` Strange, Strange/Black Mask game-over):
| effect heard | groups |
|--------------|--------|
| radio EQ | radio scanner, Alfred/Oracle/Penguin/Strange/Ra's comms, copter, cameras, Gordon radio, car radio |
| phone EQ | Penguin PRED, Zsasz calls, Two-Face court, Joker (PRED, theatre), Strange/Mayor/Harley PA, Riddler (Enigma, cameos), Vicki Vale radio scan, Penguin exhibits |
| phone-mic EQ / TV-speaker EQ | interview tapes (Vicki Vale, Joker), Strange on TV, Joker church message, `UNDER_C` TV |
| radio noise | copter, car radio, radio scanner (Vicki), Vicki Vale radio scan, map radio |
| distortion (slight, speaker) | Penguin exhibits, radio scanner (Vicki), `UNDER_C` TV |
| echo (PA delay) | Strange/Mayor/Harley/Joker/Two-Face PA and court lines, Catwoman side mission (Strange) |
| very short delay (width) | Riddler cameos, radio scanner (Riddler) |
| reverb (small/short room, slight wide room) | Riddler PRED, Harley 5 ch, Strange TV, Joker church, Talia sequence, Harley DLC, game-over lines (barely) |
| reverse reverb (occasional) | Riddler (PRED, cameos, radio scanner), Poison Ivy |
| glitches (parts repeat at another pitch), small repeats | Riddler (PRED, cameos, Enigma, radio scanner) |
| pitch shift, fixed, down | Clayface |
| pitch shift over time + robot + pan over time | Mr. Freeze |
| octaves overlaid (down), formant shift, stereo width | Ra's al Ghul (giant, trials), Poison Ivy |
| formant shift, unintelligible voice | car radio |
## 10. Scope after stage 0 (user, 2026-10-27)
Matching effects automatically per line is too complex: the Dubbing section gets **manual controls for every
effect found** (section 9), which the user sets by ear. Dropped: automatic profiles, `FX.csv` /
`fx_profiles.json`, detection in the catalog build (stage 2 of section 7). Kept: pedalboard, the Dubbing tab in
Unheard City, everything else of sections 5-6.
Effects rack (`ackit/fx.py`, one table of blocks and parameters that the UI is generated from; every block off
by default; fixed order; seeded randomness, so a setting always renders the same audio):
| # | block | controls | built on |
|---|-------|----------|----------|
| 1 | Pitch | shift (semitones), formant shift, drift depth + rate (pitch over time) | `pedalboard.time_stretch` (pitch curve, formants) |
| 2 | Octave layers | two layers: interval + level | `time_stretch` pitch, mixed |
| 3 | Robot | ring modulator frequency + mix, metallic comb (delay, feedback) | numpy |
| 4 | Glitch | events per 10 s, length, repeats, pitch of the repeats, seed | numpy + `time_stretch` |
| 5 | Voice band (EQ) | low cut, high cut, slope, mid boost (freq, gain, Q) | `HighpassFilter`, `LowpassFilter`, `PeakFilter` |
| 6 | Distortion | drive, clip, bit depth, phone codec (GSM) | `Distortion`, `Clipping`, `Bitcrush`, `GSMFullRateCompressor` |
| 7 | Noise | level, colour (band), crackle | numpy |
| 8 | Echo | time, feedback, mix | `Delay` |
| 9 | Reverb | room size, damping, wet, width; reverse reverb amount + how often | `Reverb` (reversed for the reverse swell) |
| 10 | Channels (2-5 ch lines) | width (L/R delay), auto-pan rate + depth, surround echo delay + level, dry voice in C | numpy, channel order of section 9 |
Then the existing chain: loudness match / normalize / gain, encode within the size limit.
Stages from here (one commit each; 1 and 2 done on 2026-10-27, see NOTES.md):
1. `ackit/fx.py` + tests (synthetic signals: each block measured back) + pedalboard dependency
(`pyproject.toml`, `docs/decisions.md`, licenses).
2. Dubbing tab (`app/dubbing/`): line list (character/category/search filters, state per line), the existing
sound editor embedded (record/load, trim, A/B, subtitle, apply/revert), the effects rack between the take and
the loudness step, settings saved per line in user data; GUI tests; user guide.
3. Real-game round trip of a dubbed line with effects, packaging (PyInstaller + pedalboard), version bump.
## 11. UI revision (user, 2026-10-27, after trying stages 1-2)
- The subtitle box was too small for dubbing, and the Sounds tab and the Dubbing tab both had an editor:
browsing and editing are now separate. **Sounds** only browses (list + a compact preview with **Edit** / **Add
to edit list**); **Dubbing** became **Edit**, for every kind of sound, with an edit list instead of a second
browser, the subtitle as a large **script**, and Previous / Next.
- The effects panel was "raw": sliders with readable values (log scale for frequencies and times), **looks**
for the voices heard in the game (section 9) and **preset buttons** per effect that set the sliders in place.
- The reverb (pedalboard's Freeverb) sounded poor: replaced by a generated room response (decay, pre-delay,
reflections, damping, width) with presets (small room to church, tiled cell, outdoors).
- Second round: the browser's player keeps Play/Stop together (editing actions apart on the right); the Edit tab
fits the window without scrolling (effects as a chain with one settings area); the app and docs use generic
terms (replacement, effects, subtitle), since the Edit tab is for every kind of sound, not only dubbing.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 215 KiB

After

Width:  |  Height:  |  Size: 358 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 176 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 187 KiB

After

Width:  |  Height:  |  Size: 315 KiB

View File

@@ -23,6 +23,7 @@ game is set to (Steam > game properties > Language) and applies to everything yo
effects). The game has voices in English (INT), French, Italian, German and Spanish; for the other languages
(Latin American Spanish, Polish, Portuguese, Russian, Japanese, Korean) it plays the English voices, so the
editor shows those and says *voices: INT* next to the list.
- **Edit**: the sounds of the edit list in that language (sounds of another language stay on the list, hidden).
- **Subtitles** and **Menu text**: the texts of that language.
Changing it reloads the lists; the next start is again in the game's language.
Some voice lines sound English in every language: the game itself ships the English recording for most
@@ -51,38 +52,81 @@ the same line and are changed and reverted together.
- Click a column header to sort (length, size, status, ...). Ctrl/Shift-click selects several rows.
- Archive **BANK** holds the short sounds stored inside soundbanks (footsteps, foley, combat,
weapons, menu sounds, grunts). Many are named after the game event that plays them (`Play_...`), the
rest after their bank (`CHR_BatclawGrapple_3`). The editor shows the bank and how many copies the game keeps of
the sound; a replacement is written into every copy.
rest after their bank (`CHR_BatclawGrapple_3`). The Edit tab shows the bank and how many copies the game keeps
of the sound; a replacement is written into every copy.
![Browsing by category](img/browse.png)
## Listen
The editor under the list shows the selected sound in three columns side by side (drag the dividers to
resize them): **Listen**, **Replace** and **Subtitle**. Select a row and press **Space** to play the original
(once a replacement is loaded, Space plays the replacement; Space again stops), or double-click it. Each column
ends with its own buttons on one line. In **Listen**:
- **Original** / **Installed** play the game's original and what is currently in the game; **Stop** (Esc).
Click on a waveform to play from that point.
- Above them: **A/B** (switch between the original and your replacement at the same position, to compare),
**Loop**, **Volume** and the playback position. The icon buttons show their names when you point at them.
The **Sounds** tab is for finding and listening; it does not change anything. Below the list, the selected sound
is shown with its details and subtitle (left), a player (middle) and the way into the Edit tab (right).
**Play original** (or Space, or double-click the row), **Play installed** (what the game has now, if you
replaced it) and **Stop** (Esc) sit together above the waveforms; click on a waveform to play from that point.
## Replace
![Editor](img/editor.png)
1. **Load file…** (Ctrl+O): WAV, FLAC, OGG, OPUS, MP3 or AIFF. (M4A/WMA work only if ffmpeg is installed.)
Or choose a microphone in the list and press **Record**, then **Stop recording**. Listen with
**Replacement**; record again as often as you like.
2. Adjust: **Keep … to** trims (or right-drag on the replacement waveform), **Trim silence**, **Gain**,
**Normalize**, **Match loudness** (to the original; recommended for voice lines).
3. **Apply to game** (Ctrl+S). The audio is converted to the original's format and written into the game files.
The row turns bold and shows *Modified*.
To change sounds, select them (Ctrl/Shift-click for several) and press **Edit** (or Enter, or right-click >
**Edit**): they go onto the **edit list** and the **Edit** tab opens on the first. **Add to edit list** puts them
there without leaving the list, e.g. to collect all the lines of a character or all the sounds of a weapon
first.
Size rule: voice lines and effects must fit into the original's size (the *Max size* column), because the
game reads exactly that many bytes. Longer audio is encoded at lower quality to fit; if it still does not fit
you are asked to shorten it. Music (`Music_…`) can be any length. A warning appears when your audio is longer
than the original, since the game may cut it off.
## Edit
![Edit tab](img/editor.png)
The **Edit** tab is where sounds are replaced: voice lines, music, ambience, sound effects. Everything fits in
the window: left the **edit list**, middle the sound you are working on, right its effects.
- **Edit list**: filter it, **Remove** sounds you are done with, or **Remove applied**. Columns: a dot = a
replacement is saved, FX = number of effects on, a tick = applied to the game. **Previous** / **Next**
(Alt+Left / Alt+Right) walk the list.
1. **Subtitle** (voice lines only): shown large, with the game's original (and the English line when you edit
another language) under it. You can rewrite it there and **Apply** it (see *Subtitles*).
2. **Listen**: **Original**, **Installed** and **Replacement** each have a play button and a waveform; **A/B**
switches between the original and the replacement at the same position; **Loop**, **Volume**, **Stop**
(Esc).
3. **Replacement**: **Record** (choose the input in the list next to it; press again to stop) or **Load file…**
(Ctrl+O: WAV, FLAC, OGG, OPUS, MP3 or AIFF; M4A/WMA only if ffmpeg is installed), as often as you like.
**Keep … to** trims (or right-drag on the replacement's waveform), **Trim silence**, **Gain**,
**Normalize**, **Match loudness** (to the original; on by default for voice lines). The replacement is saved
at once in your user data folder (`edits`), with its trim, level and effect settings, so you can come back
to it any time; nothing is written to the game until you apply it.
4. **Apply to game** (Ctrl+S). The audio (with its effects) is converted to the original's format and written
into the game files; the list shows a tick and the Sounds tab *Modified*.
Size rule: voice lines and sound effects must fit into the original's size (the *Max size* column), because
the game reads exactly that many bytes. Longer audio is encoded at lower quality to fit; if it still does not
fit you are asked to shorten it. Music (`Music_…`) can be any length. A warning appears when your audio is
longer than the original, since the game may cut it off.
### Effects
Many sounds are not dry in the game (radios, phones, announcements, speakers, Mr. Freeze's suit...); the
replacement gets the effects you turn on, and **Replacement** plays it with them.
- **Start from a preset…** sets several effects at once: Radio, Phone, Tape / phone mic, TV in a room, PA
announcement, Mr. Freeze, Clayface, Riddler broadcast, Giant (Ra's al Ghul), Car radio.
- The **effect chain** lists every effect in the order it runs: tick an effect to turn it on, click its name to
see its settings below. Each effect has preset buttons that set its sliders; then adjust by ear. Moving a
slider turns its effect on; double-click a slider to put it back.
| effect | in the game, for example |
|--------|--------------------------|
| **Pitch** (shift, formant, drift = pitch over time) | Clayface (shift down), Mr. Freeze (drift) |
| **Octave layers** | Ra's al Ghul, Poison Ivy (octaves below the voice) |
| **Robot** (ring modulator, metallic comb) | Mr. Freeze's suit |
| **Glitch** (pieces repeated, at another pitch) | the Riddler's hacked broadcasts |
| **Band-pass EQ** (low/high cut, mid boost) | radio, phone, phone microphone, TV and small speakers |
| **Distortion** (drive, clip, bits, phone codec) | speakers, radios, the car radio |
| **Noise** (level, colour, crackle) | radio static |
| **Echo** | Hugo Strange, the Mayor, Joker and Harley over the city speakers (the *PA announcement* preset: 160 ms) |
| **Reverb** (decay, mix, pre-delay, reflections, damping; reverse swells before phrases) | rooms, TV, halls; reverse: the Riddler, Poison Ivy |
| **Channels** (sounds with 2-5 channels) | stereo width, pan over time, the PA layout: dry sound in the centre, echo in the surrounds |
**Copy** / **Paste** move settings between sounds; **Use these effects for the selected** gives every selected
sound of the edit list the same settings (e.g. all of Mr. Freeze's lines), keeping their replacements.
**Reset** turns everything off.
Some effects are added by the game itself while it plays, not stored in the sound (the radio sound of many
thugs and street conversations, the reverb of rooms and areas): if the original sounds clean here but not in
the game, keep your replacement clean too, or the effect is applied twice. Glitch, echo and reverb make a sound
longer, which counts for the size rule.
## Undo
- **Revert** in the editor puts that sound back exactly.
- **Revert** in the Edit tab puts that sound back exactly (the replacement stays saved for later).
- Right-click rows → **Revert selected**; toolbar **Revert all** puts every original back: all sounds,
subtitles and menu texts you changed, in every language.
- Originals are kept in your user data folder (`%APPDATA%\UnheardCity` on Windows,
@@ -109,9 +153,9 @@ The **Subtitles** tab lists the subtitle of every speech line in the selected la
(Windows-1252: accents are fine, other scripts are not); those stored as Unicode (almost all Russian,
Japanese and Korean ones, most Polish ones) accept any character. The editor says so when a character does
not fit.
- **Go to the sound** opens the speech line in the Sounds tab. In the Sounds tab, the editor of a speech line
shows its subtitle in the **Subtitle** column next to the audio: edit both side by side, **Apply** /
**Revert** the subtitle there, or **Show in list** to jump back to the Subtitles tab.
- **Go to the sound** selects the speech line in the Sounds tab (its subtitle shows in the preview). In the
Edit tab, a voice line's **Subtitle** is shown large above the audio: edit it there, **Apply** / **Revert**
it, or **Show in list** to jump back to the Subtitles tab.
- **Find and replace…** works on the rows the filters show (texts that would not fit are listed and skipped).
- **Revert** / **Revert all texts here** put the original text back (every copy in every game file).
- About 30 subtitles per language have no sound in the game (texts such as the Arkham City Stories, unused
@@ -145,7 +189,7 @@ Game folder, where originals are kept (changeable only while nothing is modified
the playback device. The recording device
is chosen next to the Record button. **Background**: show or hide the Arkham City wallpaper behind the filter
panel; **Background strength**: how visible it is (10 % by default).
The editor panel opens tall enough to show all its controls; drag the divider above it to change that.
Drag the dividers to resize: the preview under the sound list, and the edit list, editor and effects columns of the Edit tab.
## Troubleshooting
| message | what to do |