Version 3.1.0: voice lines linked across all languages from the game data

docs/formats/banks.md

@@ -7,10 +7,12 @@ Survey of the installed game (buildid 3718300), 2026-10-09.
 
 ## Inventory
 
-| where | banks | unique in-memory WEMs | in the catalog |
-|-------|-------|-----------------------|----------------|
-| loose `SFX\*.bnk` (Batman/Catwoman footsteps, 32 surfaces each, + `TST_Prep_Media`) | 65 | 1,117 | 0 |
-| banks embedded in `.upk` packages (in 1,432 of the 2,650 packages) | 3,340 | 15,641 (~276 MB) | 0 |
+| where | banks | unique in-memory WEMs |
+|-------|-------|-----------------------|
+| loose `SFX\*.bnk` (Batman/Catwoman footsteps, 32 surfaces each, + `TST_Prep_Media`) | 65 | 1,117 |
+| banks embedded in `.upk` packages (in 1,432 of the 2,650 packages) | 3,340 | 15,641 (~276 MB) |
+
+All of them are in the catalog (archive `BANK`).
 
 The two sets share 119 media IDs: 15,800 unique media IDs in 96,011 places (bank copies). 396 of them are
 **prefetch heads** of streamed WAD sounds (below), so there are **15,404 in-memory sounds**. All copies of a
@@ -52,11 +54,37 @@ WAD sounds have such heads (2,101 copies); replacing one of them must rewrite th
 
 ## Localized banks
 Speech banks are localized inside the (language-neutral) packages: one `AkBank` export starts with a language
-list (FStrings `INT FRA ITA DEU ESN`) and then holds one complete bank per language, in that order, all with
-the same bank ID; each language's copy has its own media IDs (checked on 215 banks against the WAD languages
-of their streamed lines). ackit labels every in-memory media with its language (`BANK.csv` column `language`):
-8,098 language-neutral sounds, ~1,460 per voice language. A catalog shows the neutral sounds plus the loaded
-language's lines; names are unique per language.
+list (FStrings `INT FRA ITA DEU ESN`) and then holds one block per language, in that order: the BKPK table(s)
+of the speech WAD entry for that language (`wad.md`) and a complete bank, with the same bank ID in every
+language. ackit gives each bank and each BKPK table the language of its position in that list (an export
+whose block count differs from its language count would be reported; none does). Each in-memory media gets
+the languages of the banks that hold it (`BANK.csv` column `language`): 8,098 sounds are language-neutral,
+~1,460 belong to one voice language, 6 to several (a DLC intro line shared by INT/FRA/ITA/ESN while DEU has
+its own); a media stored in the banks of every language is listed as `''` (the same recording everywhere).
+A catalog shows the neutral sounds plus the loaded language's lines; names are unique per language.
+
+## Lines: one recording per language
+Implemented in `ackit/lines.py`, checked by `ackit verify --catalog`. A voice line exists once per language,
+and the game ties the recordings together itself:
+- Every WEM of a speech WAD has exactly one row in its own language's BKPK table (media ID, offset, size):
+  10,816 of 10,816 in each of the 5 languages, no row without a WEM. 657 media IDs are used by several
+  languages (cinematics, emotes, Clayface), each at another position in each language's WAD, so the tables
+  are kept per language and never merged.
+- Every recording, streamed, prefetched or in memory, is the source of a Wwise Sound object; Sound objects,
+  events, actions and containers have the same IDs in every language, only the media differ. A *line* = the
+  recordings that are sources of the same Sound objects: 12,434 lines (WAD and in-memory), each with exactly
+  one recording in each of the 5 voice languages (61,510 recordings), no Sound object with two recordings in
+  one language. Line identity (`line` column) = the smallest Sound object ID.
+- Prefetched sources count too: the 396 lines with a prefetch head (mostly the game-over taunts) are linked
+  only through them.
+- 5 recordings per language are stored twice in that language's WAD (same media ID in two entries, byte-
+  identical, e.g. `Speech_STEEL_A1_CH2_Thug_behinddoormuffled` and `DIA_SteelA1Muffled`): the second place
+  is a copy of the line (same `line`, keeps its position name) and is written and reverted with it, because
+  the game plays whichever place the loaded package's table names.
+- Independent checks (no BKPK involved): every WAD recording is named as a source by a Sound object of a bank
+  of its own language (54,055 of 54,055); every stored prefetch head equals the start of the recording its
+  line gives it (2,101 of 2,101); a line's recordings have similar lengths across languages (11,761 of 12,434
+  within a factor 1.5; the others are different edits, e.g. a 0.55 s ESN line vs 5.46 s elsewhere).
 
 ## Banks inside packages
 - The packages are LZO-compressed UE3 packages (`upk.md`); ackit decompresses them itself and searches the

docs/formats/subtitles.md

@@ -6,7 +6,8 @@ the UE3 packages (`upk.md`), holding the subtitle text of all 11 languages. acki
 
 ## Inventory (buildid 3718300)
 - 14,743 dialogue objects in the packages for 9,016 speech lines (a line's object is copied into every package
-  that uses it; all copies are identical).
+  that uses it; all copies are identical), plus 41 scripted subtitles (below): ~9,400 subtitle strings and
+  168,688 stored copies.
 - 11 languages: INT, FRA, ITA, DEU, ESN, ESM, POL, POR, RUS, JPN, KOR (the speech WADs exist only for INT, FRA,
   ITA, DEU, ESN; the other languages play the INT voices with their own subtitles).
 - ~9,400 subtitle strings per language: 94 % of the subtitles have one line, the rest up to 5 lines.
@@ -29,21 +30,25 @@ or property name from the name table, i32 byte size, i32 array index, then `size
 | field | content |
 |-------|---------|
 | `0x10` @ `0x2C` | the speech event this line plays, e.g. export `Speech_CHURCH_B1_Ch2_POST_Explosion_BM_Cash_04` |
-| `0x0D` @ `0x34` | speaker, e.g. `Batman` |
+| `0x0D` @ `0x34` | speaker, e.g. `Batman` (`ackit verify --catalog` compares it with the curated speaker labels) |
 | `0x09 LocalizedSubtitles` | u32 count, then per language: `0D 00 00 00` FString language code, `09 00 0C 00 Subtitles` struct, `00 00` |
 | `Subtitles` | u32 line count, then per line: `0D 00 00 00` FString text, `04 00 0C 00` 4-byte value, `00 00` |
 
 The sizes add up exactly in all 14,743 objects (checked by the parser: `SubtitleError` otherwise).
 
 ## Link to the speech sound
-The speech event name is hashed (FNV-1) to the Wwise event ID, followed through the soundbanks (event ->
-action -> containers -> Sound) to the streamed media, and that media ID is looked up in the per-language
-BKPK tables (`banks.md`) to find the WAD entry and variation of each language. This is exact; the `_NN`
-suffix of the event name does **not** match the variation order inside the WAD entry (WEMs are stored in
-media-ID order). Lines kept in memory link to the localized BANK sound of their language. Linked: ~99 % of
-the strings (e.g. INT 9,265 / 9,379); the rest have no event that reaches a sound (unused or placeholder
-lines such as `..._TEMP_29`) and stay editable without a sound link.
-Event names with a number (`Speech_X_04`) exist only as export names, not in the name table.
+The speech event name is hashed (FNV-1) to the Wwise event ID and followed through the soundbanks (event ->
+action -> containers -> Sound objects, the same in every language) to that language's recording of the line
+(`banks.md`, *Lines*): a WAD variation or a localized in-memory sound. This is exact; the `_NN` suffix of the
+event name does **not** match the variation order inside the WAD entry (WEMs are stored in media-ID order).
+The names come from the packages' name and export tables (numbered names like `Speech_X_04` exist only as
+export names) and from the subtitle objects themselves: some events are listed in no name table, but their
+dialogue / RSubtitle object names them (Vicki Vale's radio reports, the Hush tape reveal).
+An event that plays two versions of a line (105 subtitled events, e.g. Mr. Freeze's lines) links its subtitle
+to both (`sound` = `key1;key2`). Linked: 103,304 of 103,658 strings in the 11 languages (INT 9,373 / 9,404),
+each to the same line in every language. The rest (29 speech names, ~32 strings per language: Riddler trophy
+texts, Arkham City Stories, radio reports and tapes the game does not ship audio events for) have no event in
+any bank and stay editable without a sound link.
 
 ## How ackit writes them
 - In place, into **every copy** (`data/catalog/SUB_locations.csv.gz`): the new text replaces the FString's

docs/formats/upk.md

@@ -22,6 +22,16 @@ All offsets in the summary (names, exports, ...) refer to the **uncompressed** p
 package = the summary without the chunk table, followed by the chunks' data at their UncompressedOffset
 (the first chunk starts right after the summary, at NameOffset).
 
+## Name, import and export tables (uncompressed package, at NameOffset / ImportOffset / ExportOffset)
+- Name: FString (positive length: ANSI incl. NUL; negative: UTF-16) + u64 flags.
+- Import (28 bytes): ClassPackage (name index, number), ClassName (index, number), i32 OuterIndex,
+  ObjectName (index, number).
+- Export: i32 ClassIndex, SuperIndex, OuterIndex, ObjectName (index, number), ArchetypeIndex, **one extra u32
+  (licensee 101)**, u64 ObjectFlags, i32 SerialSize, SerialOffset, u32 ExportFlags, i32 count + count x i32
+  (net objects), 16-byte guid, u32 package flags: 72 bytes when the net-object list is empty. Checked on every
+  export of the game. A name with number n is shown as `NAME_<n-1>` (e.g. the speech event `..._04`): such full
+  names exist only in the export table. `Package.tables()` returns names, imports and exports.
+
 ## Compressed chunks
 Each chunk (at CompressedOffset): `u32 tag, u32 block size (0x20000), u32 compressed sum, u32 uncompressed
 sum`, then one `{u32 compressed size, u32 uncompressed size}` per block, then the blocks. Every block is an

docs/formats/wad.md

@@ -26,14 +26,19 @@ Rules:
 Payload kinds:
 - **LOC WADs**: each entry is a group of WEMs, the variations of one line (ITA: 10816 WEMs in
   2464 entries; e.g. `CHR_RadioScanner` holds 91). Each WEM starts at the next 0x800 boundary after
-  the previous one (zero padding); 3 entries pack two WEMs back to back. Addressed `HASH:i`.
+  the previous one (zero padding); 3 entries pack two WEMs back to back. Addressed `HASH:i`. The WEMs of an
+  entry are stored in media-ID order and the languages have different media IDs, so variation `i` is a
+  different line in each language (75 % of the lines). Which recording is which line comes from the banks
+  (`banks.md`, *Lines*), not from positions.
 - **SFX WAD**: 79 grouped ambience/effect entries (260 WEMs) plus 4 **stream packs** (below).
 
 Where the game gets the inner offsets: the entry IDs appear next to `BKPK` headers inside the
-LZO-compressed UE3 packages (e.g. `BmGame.upk`, `OW.upk`, `Church_B1_*.upk`). Those embedded BKPK
-tables give each WEM's offset **and size**. ackit does not rewrite `.upk` files, so these WEMs are
-replaced **in place, no larger than the original** (zero-filled to the original size), and the WAD
-layout never changes. The user's earlier quickbms-reimport mod did the same and was loaded by the game.
+LZO-compressed UE3 packages (e.g. `BmGame.upk`, `OW.upk`, `Church_B1_*.upk`). A speech entry has one table
+per language, inside a localized `AkBank` export (its k-th table belongs to the k-th language the export lists;
+the SFX WAD's 79 tables are outside such exports). Those embedded BKPK tables give each WEM's offset **and
+size**; in every language each WEM has exactly one row (`banks.md`, *Lines*). ackit never changes them,
+so these WEMs are replaced **in place, no larger than the original** (zero-filled to the original size), and
+the WAD layout never changes. The user's earlier quickbms-reimport mod did the same and was loaded by the game.
 Only the 4 SFX stream packs, whose BKPK index is a loose `.bnk`, can change size.
 
 ## BKPK stream packs (`SFX\<ID>.bnk` + SFX WAD entry `<ID>`)
@@ -61,9 +66,9 @@ end rounded up to 0x800. Padding is mostly zero (a few non-zero bytes in `EDF036
 original bytes where nothing moved).
 
 Replacing a pack media (`ackit` target `EDF036D6/393BC53A`) rewrites the pack table (`.bnk`), the
-WAD entry payload, and the WAD table/offsets. Verified in-game (NOTES.md, E2).
+WAD entry payload, and the WAD table/offsets. Verified in-game (findings E2, E14).
 
 ## Regular soundbanks
 
 The other `SFX\*.bnk` files and the banks embedded in `.upk` packages hold their WEMs in memory (not in a
-WAD); see `banks.md`. The toolkit does not read or modify them yet.
+WAD); ackit lists and writes them as the catalog archive `BANK` (`banks.md`).