Version 3.2.2: in-memory replacements fill their slots exactly (padded ones played as silence)

This commit is contained in:
2026-10-11 00:10:44 +02:00
parent cac52e7597
commit b6b96307ca
16 changed files with 223 additions and 52 deletions

View File

@@ -107,9 +107,23 @@
the loose/embedded bank-name overlap, REPORT's test count, wem.md's SFX claim (626 / 651 rebuild, not "all but 4 the loose/embedded bank-name overlap, REPORT's test count, wem.md's SFX claim (626 / 651 rebuild, not "all but 4
looping tracks": 14 music seek tables, 5 music sample counts, 6 ambience `smpl` loops) and its extra-chunk count looping tracks": 14 music seek tables, 5 music sample counts, 6 ambience `smpl` loops) and its extra-chunk count
(722 WEMs, not 37). New test: cross-references between the shipped files. (722 WEMs, not 37). New test: cross-references between the shipped files.
- NEXT (needs a person): play a mission with a replaced footstep/foley sound and an edited subtitle (REPORT.md). - 2026-10-10 user gameplay test: replaced voice lines and subtitles WORK in game. Batman footsteps do NOT:
- OPEN: channel order for >2 channels unverified; speech and in-memory sounds *heard* in-game (need gameplay; all 568 BANK sounds of the CHR_BatmanFootsteps* banks were replaced with Downloads\boom.mp3 (trimmed per sound,
posted keys do not drive the menu). still APPLIED, with the user's own test subtitle SUB:ITA:Speech_CHURCH_B1_CH2_HQ_INTRO2_08; revert with
`ackit revert --all` when the user says so). In game the boom was heard once, briefly; otherwise the original
steps. Checked: every copy (40 files) holds the new bytes; replacements are loud (-8.8 dB RMS vs -20..-33);
no other media ID has the same original audio; no game data outside CookedPCConsole. The named footstep
events (Play_BM_Footstep_Walk/Run/Land/...) reach only the 46 media of bank CHR_BatmanFootsteps (replaced);
the 522 surface media (CHR_BatmanFootsteps_<Surface>_Media, loose SFX\*.bnk) are reached by no named event
(switch containers not followed by event_objects?). NEXT: find where the game gets the step audio it plays
(switch/random containers and their children, banks loaded at runtime, a memory capture of a step in game).
- 2026-10-10 footsteps solved (findings 24): in-memory replacements were zero-padded in their slot and the game
plays that as silence; ackit now fills each in-memory copy exactly with a LIST/adtl/labl chunk (`wem.fill`,
`Workspace.fits/fill_sizes/prepare`). Verified by the user in game (a beep on every step), e2e_features 89/89.
Version 3.2.2. The footstep test was reverted (game original).
- DONE by the user in game: replaced voice lines, edited subtitles and replaced footsteps (after the fill fix).
- OPEN: channel order for >2 channels unverified; 19 speaker labels to check by listening; the Linux build was
never built or run (no Linux here).
## Environment facts ## Environment facts
- Game: Steam AppID 200260 "Batman: Arkham City GOTY", `C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham - Game: Steam AppID 200260 "Batman: Arkham City GOTY", `C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham

View File

@@ -42,7 +42,7 @@ editor Unheard City (mission 2).
- **Safety**: every changed file (WAD, bank, package, localization file) is rebuilt from a verified original copy - **Safety**: every changed file (WAD, bank, package, localization file) is rebuilt from a verified original copy
in the store + the edits applied now; reverting the last edit restores it byte-for-byte; files changed by in the store + the edits applied now; reverting the last edit restores it byte-for-byte; files changed by
another tool or Steam are detected (whole-file SHA-1) and not overwritten without confirmation. another tool or Steam are detected (whole-file SHA-1) and not overwritten without confirmation.
- **Tests**: 133 automated tests (unit, synthetic game with packages/banks/dialogues/localization, a - **Tests**: 135 automated tests (unit, synthetic game with packages/banks/dialogues/localization, a
two-language game for the lines (incl. a recording also stored whole in a bank), the shipped catalog's cross-references, real-game read-only (every two-language game for the lines (incl. a recording also stored whole in a bank), the shipped catalog's cross-references, real-game read-only (every
name of every language's catalog resolves to its own sound), offscreen GUI incl. list order, selection and name of every language's catalog resolves to its own sound), offscreen GUI incl. list order, selection and
loading overlays) pass. `ackit verify` PASS (incl. loading overlays) pass. `ackit verify` PASS (incl.
@@ -52,21 +52,21 @@ editor Unheard City (mission 2).
language's localized line, prefetch heads, a line also stored whole in banks, a multi-copy subtitle, a scripted subtitle, a menu text, sound + language's localized line, prefetch heads, a line also stored whole in banks, a multi-copy subtitle, a scripted subtitle, a menu text, sound +
subtitle in one package, a mod moved to another voice language and matched to the same line at another subtitle in one package, a mod moved to another voice language and matched to the same line at another
position); `ackit verify --ingame` PASS (music heard, ncc 1.0); position); `ackit verify --ingame` PASS (music heard, ncc 1.0);
`--banks` PASS (77 in-memory sounds of the front-end packages replaced in 54 files: the game loads them, 68 `--banks` PASS (77 in-memory sounds of the front-end packages replaced in 54 files: the game loads them, 71
probes found in its memory, no crash); `--texts` PASS (12/12 menu texts and 23/23 subtitles of Startup.upk probes found in its memory, no crash); `--texts` PASS (12/12 menu texts and 23/23 subtitles of Startup.upk
found in the game's memory, no crash); `tools/e2e_editor.py` PASS. Game files original at the end found in the game's memory, no crash); `tools/e2e_editor.py` PASS. Game files original at the end
(`ackit status`). The catalog builder refuses to run on a modified game, so the catalog always describes the (`ackit status`). The catalog builder refuses to run on a modified game, so the catalog always describes the
original files. original files.
- **Package**: `dist/UnheardCity-3.2.1-windows-amd64.zip` (86 MB, catalogs included). With `PATH` reduced to - **Package**: `dist/UnheardCity-3.2.2-windows-amd64.zip` (86 MB, catalogs included). With `PATH` reduced to
the Windows folders the packaged exe passes its smoke test (in the game's language - Italian on the test PC: the Windows folders the packaged exe passes its smoke test (in the game's language - Italian on the test PC:
21,015 sounds loaded, encode + inject, 9,377 subtitles and 7,020 menu texts loaded, an in-memory sound decoded from a package) and the GUI starts and 21,015 sounds loaded, encode + inject, 9,377 subtitles and 7,020 menu texts loaded, an in-memory sound decoded from a package) and the GUI starts and
quits cleanly. The installer is unchanged since mission 2 (install/upgrade/uninstall verified then). quits cleanly. The installer is unchanged since mission 2 (install/upgrade/uninstall verified then).
## Unresolved / limitations ## Unresolved / limitations
- **Heard in game, not proven automatically**: in-memory sounds and subtitles. The game loads the patched - **Checked by a person in game** (2026-10-10), not automatically (nothing of this plays or shows on the title
packages (probe data and texts found in its memory, no crash), but nothing of this plays/shows on the title screen without input, findings 18): replaced voice lines and edited subtitles are heard and shown; replaced
screen without input, and keys posted to the game window do not reach the menu (findings 18). Speech lines footsteps play on every step. That test found the one defect the automatic checks could not: in-memory
were already in this situation; the decoder path is the same one proven for music. replacements followed by zero padding in their slot played as silence (fixed in 3.2.2, findings 24).
- **Size limits**: in-memory sounds and speech must fit their original size; subtitles must fit the original - **Size limits**: in-memory sounds and speech must fit their original size; subtitles must fit the original
text's length ("room"); cp1252 subtitles accept cp1252 characters only. Lifting these limits means moving text's length ("room"); cp1252 subtitles accept cp1252 characters only. Lifting these limits means moving
objects inside the packages (new export offsets/sizes, added chunks): designed but not done, because it could objects inside the packages (new export offsets/sizes, added chunks): designed but not done, because it could
@@ -92,8 +92,8 @@ editor Unheard City (mission 2).
cached); it gets the same line identities, but no original checksums. cached); it gets the same line identities, but no original checksums.
## Next steps ## Next steps
1. Gameplay check (a person): a replaced footstep/foley sound and an edited subtitle in a mission. 1. Optional: lift the subtitle length limit by relocating dialogue objects (checked in game by a person).
2. Optional: lift the subtitle length limit by relocating dialogue objects (needs that gameplay check). 2. Build and smoke-test the Linux package once on a Linux machine.
3. Listen to the 19 lines whose speaker label differs from the game's dialogue speaker field 3. Listen to the 19 lines whose speaker label differs from the game's dialogue speaker field
(`docs/categories.md`) and fix the curated lists if needed; extend them (`data/catalog/source/<lang>/*.txt`) (`docs/categories.md`) and fix the curated lists if needed; extend them (`data/catalog/source/<lang>/*.txt`)
and rebuild the catalog. and rebuild the catalog.

View File

@@ -8,4 +8,4 @@ Library layers (low -> high):
inject (WAD writing), originals (store), workspace (apply/revert), modpack (share) inject (WAD writing), originals (store), workspace (apply/revert), modpack (share)
tools cli (`ackit` command), verify/ (offline self-check, in-game test on Windows) tools cli (`ackit` command), verify/ (offline self-check, in-game test on Windows)
""" """
__version__ = "3.2.1" __version__ = "3.2.2"

View File

@@ -140,8 +140,7 @@ def prepare_import(ws, path, items, progress=None):
continue continue
orig = ws.original_bytes(im.sound) orig = ws.original_bytes(im.sound)
wem = z.read(im.item["wem"]) wem = z.read(im.item["wem"])
lim = ws.limit(im.sound) if im.how == "exact" and sha1_16(orig) == im.item.get("original_sha1") and ws.fits(im.sound, wem):
if im.how == "exact" and sha1_16(orig) == im.item.get("original_sha1") and (lim is None or len(wem) <= lim):
im.wem, im.note = wem, "exact copy" im.wem, im.note = wem, "exact copy"
else: else:
x, sr = sf.read(io.BytesIO(z.read(im.item["audio"])), dtype="float32", always_2d=True) x, sr = sf.read(io.BytesIO(z.read(im.item["audio"])), dtype="float32", always_2d=True)

View File

@@ -200,10 +200,10 @@ def bank_probes(ws, sounds):
secs = min(10.0, max(0.4, s.seconds or 1.0)) secs = min(10.0, max(0.4, s.seconds or 1.0))
while secs >= 0.1: while secs >= 0.1:
wem, _ = encode(x[:int(secs * sr)], sr, PROBE_RATE, 1, PROBE_Q) wem, _ = encode(x[:int(secs * sr)], sr, PROBE_RATE, 1, PROBE_Q)
if len(wem) <= lim: if ws.fits(s, wem): # within the slot, and able to fill it exactly (wem.fill)
out.append((s, wem)) out.append((s, wem))
break break
secs *= 0.7 secs *= 0.7 if len(wem) > lim else 0.98
return out return out

View File

@@ -101,6 +101,60 @@ class Wem:
return b"RIFF" + struct.pack("<I", len(body)) + body return b"RIFF" + struct.pack("<I", len(body)) + body
# ---- filling an in-memory slot exactly ----
# The game ships no in-memory WEM with bytes after its RIFF inside its slot, and plays a replacement that is
# followed by zero padding there as silence (findings 24). A smaller replacement is therefore grown to the
# slot's exact size with a label list chunk before `data`, the form the game's own WEMs use for markers:
# LIST{'adtl', labl{u32 cue id 0, zero text}}. Even sizes only (the game's chunks before `data` are even).
FILL_MIN = 8 + 4 + 8 + 4 + 2
def fillable(length, size):
"""True if a WEM of `length` bytes can fill a slot of `size` bytes exactly."""
pad = size - length
return pad == 0 or (pad >= FILL_MIN and pad % 2 == 0)
def fill(wem: bytes, size: int) -> bytes:
"""`wem` grown to exactly `size` bytes (see above). ValueError if `fillable` is False."""
if not fillable(len(wem), size):
raise ValueError("a %d-byte WEM cannot fill a %d-byte slot exactly" % (len(wem), size))
pad = size - len(wem)
if not pad:
return wem
i = _data_chunk(wem)
text = pad - 8 - 4 - 8 - 4
chunk = (b"LIST" + struct.pack("<I", pad - 8) + b"adtl" + b"labl" + struct.pack("<II", text + 4, 0)
+ bytes(text))
out = wem[:i] + chunk + wem[i:]
return out[:4] + struct.pack("<I", len(out) - 8) + out[8:]
def unfill(wem: bytes) -> bytes:
"""The WEM without a filler chunk added by `fill` (a game WEM is returned unchanged)."""
i = _data_chunk(wem)
if i is None:
return wem
for j in range(12, i, 2):
if wem[j:j + 4] == b"LIST" and wem[j + 8:j + 16] == b"adtllabl":
n = 8 + struct.unpack_from("<I", wem, j + 4)[0]
if j + n == i and struct.unpack_from("<I", wem, j + 20)[0] == 0 and not wem[j + 24:i].strip(b"\0"):
out = wem[:j] + wem[i:]
return out[:4] + struct.pack("<I", len(out) - 8) + out[8:]
return wem
def _data_chunk(wem: bytes):
"""Offset of the `data` chunk header (walking the chunks), or None."""
pos = 12
while pos + 8 <= len(wem):
tag, ln = struct.unpack_from("<4sI", wem, pos)
if tag == b"data":
return pos
pos += 8 + ln
return None
SUB_ALIGN = 0x800 SUB_ALIGN = 0x800
_NONZERO = re.compile(rb"[^\x00]") _NONZERO = re.compile(rb"[^\x00]")

View File

@@ -7,7 +7,8 @@ Safety rules (docs/decisions.md):
* if a sound in the game is neither its original nor what we applied (Steam update, "verify * if a sound in the game is neither its original nor what we applied (Steam update, "verify
integrity", another tool), we refuse to write it unless forced, and report it. integrity", another tool), we refuse to write it unless forced, and report it.
In-memory sounds (archive BANK) and the prefetch heads of speech lines live in loose banks and .upk In-memory sounds (archive BANK) and the prefetch heads of speech lines live in loose banks and .upk
packages (ackit/banks.py). Before the first change to such a file a verified full copy is kept, and the packages (ackit/banks.py); a replacement fills each in-memory slot exactly (wem.fill). Before the first
change to such a file a verified full copy is kept, and the
file is always rebuilt as "original copy + every replacement currently applied", so reverting the last file is always rebuilt as "original copy + every replacement currently applied", so reverting the last
change restores it byte-for-byte and any later operation repairs an interrupted one. change restores it byte-for-byte and any later operation repairs an interrupted one.
""" """
@@ -17,7 +18,7 @@ from collections import defaultdict
from . import audio, banks from . import audio, banks
from .catalog import Catalog from .catalog import Catalog
from .encoder import encode_fit from .encoder import EncodeError, encode_fit
from .game import Game, GameError from .game import Game, GameError
from .inject import inject, read_target, parse_target, pack_bnk_path, SizeLimitError from .inject import inject, read_target, parse_target, pack_bnk_path, SizeLimitError
from .originals import OriginalsStore, sha256, sha1_16 from .originals import OriginalsStore, sha256, sha1_16
@@ -25,6 +26,7 @@ from .pack import Pack
from .texts import Texts from .texts import Texts
from .paths import user_dir from .paths import user_dir
from .wad import Wad from .wad import Wad
from .wem import FILL_MIN, fill, fillable, unfill
# sound states # sound states
ORIGINAL = "original" # untouched ORIGINAL = "original" # untouched
@@ -93,7 +95,7 @@ class Workspace:
if not locs: if not locs:
raise GameError("no copy of %s was found in the game's banks" % s.name) raise GameError("no copy of %s was found in the game's banks" % s.name)
l = locs[0] l = locs[0]
return banks.riff_trim(banks.open_file(self.bank_path(l.file)).read(l.offset, l.size)) return unfill(banks.riff_trim(banks.open_file(self.bank_path(l.file)).read(l.offset, l.size)))
return read_target(self.wad_path(s), s.target) return read_target(self.wad_path(s), s.target)
def original_bytes(self, s): def original_bytes(self, s):
@@ -133,11 +135,34 @@ class Workspace:
def original_audio(self, s): def original_audio(self, s):
return audio.decode_wem(self.original_bytes(s)) return audio.decode_wem(self.original_bytes(s))
def fill_sizes(self, s):
"""Slot sizes a replacement of `s` must fill exactly: its in-memory copies, and for a WAD recording the
bank copies that hold it whole (wem.fill: the game plays a padded in-memory WEM as silence)."""
if s.archive == banks.ARCHIVE:
return sorted({l.size for l in self.locations(s) if l.kind == banks.MEMORY})
orig = self.limit(s)
return sorted({l.size for l in self.locations(s) if orig is not None and l.size >= orig})
def fits(self, s, wem):
"""True if `wem` can be written for `s`: within its size limit and able to fill its in-memory slots."""
lim = self.limit(s)
return (lim is None or len(wem) <= lim) and all(fillable(len(wem), n) for n in self.fill_sizes(s))
# encoding ---------------------------------------------------------------------------------- # encoding ----------------------------------------------------------------------------------
def prepare(self, s, x, src_rate, quality="auto", progress=None): def prepare(self, s, x, src_rate, quality="auto", progress=None):
"""Encode PCM for sound `s` (its rate/channels, fitted to its size limit) -> (wem, info).""" """Encode PCM for sound `s` (its rate/channels, fitted to its size limit) -> (wem, info). A sound with
return encode_fit(x, src_rate, s.rate or src_rate, s.channels or x.shape[1], self.limit(s), in-memory slots gets a WEM that fills them exactly (`fits`): at least FILL_MIN bytes smaller, and an
quality, progress) even number of bytes smaller; the end is shortened by a few samples until the size allows that."""
sizes = self.fill_sizes(s)
lim = self.limit(s)
if sizes:
lim = min(([lim] if lim is not None else []) + sizes) - FILL_MIN
rate, ch = s.rate or src_rate, s.channels or x.shape[1]
for k in range(32):
wem, info = encode_fit(x[:len(x) - 16 * k] if k else x, src_rate, rate, ch, lim, quality, progress)
if self.fits(s, wem):
return wem, info
raise EncodeError("could not encode a size that fills this sound's slots exactly; shorten the audio")
# writing ----------------------------------------------------------------------------------- # writing -----------------------------------------------------------------------------------
def _backup(self, wad, pack_hashes): def _backup(self, wad, pack_hashes):
@@ -225,11 +250,14 @@ class Workspace:
locs = [l for l in self.locations(key) if l.file == rel] locs = [l for l in self.locations(key) if l.file == rel]
if not locs: if not locs:
continue continue
data = self.store.get_blob(e["applied"]) data = unfill(self.store.get_blob(e["applied"]))
for l in locs: for l in locs:
if l.kind == banks.MEMORY: whole = l.kind == banks.MEMORY or l.size >= e["orig_size"] # a whole WEM in memory
if whole and fillable(len(data), l.size):
payload = fill(data, l.size)
elif whole: # recorded by a version before 3.2.2 (zero padding: the game plays silence)
payload = data + bytes(l.size - len(data)) payload = data + bytes(l.size - len(data))
else: # head of a WAD line, which is stored zero-padded to its original size else: # prefetch head: the start of the WAD recording as it is stored there
full = data + bytes(max(0, e["orig_size"] - len(data))) full = data + bytes(max(0, e["orig_size"] - len(data)))
payload = (full + bytes(l.size))[:l.size] payload = (full + bytes(l.size))[:l.size]
out.append((l.offset, payload)) out.append((l.offset, payload))
@@ -286,6 +314,12 @@ class Workspace:
"""changes: [(sound, wem_bytes, meta dict)]. One atomic rewrite per WAD / bank file. A recording stored """changes: [(sound, wem_bytes, meta dict)]. One atomic rewrite per WAD / bank file. A recording stored
in two places of a WAD gets the same bytes in both (Workspace.copies).""" in two places of a WAD gets the same bytes in both (Workspace.copies)."""
keys = {s.key for s, _, _ in changes} keys = {s.key for s, _, _ in changes}
changes = [(s, unfill(data), meta) for s, data, meta in changes]
for s, data, _ in changes:
if not self.fits(s, data) and len(data) <= (self.limit(s) or len(data)):
raise GameError("%s: a %d-byte WEM cannot fill this sound's in-memory slots (%s bytes) exactly; "
"encode it for this sound (Workspace.prepare / the editor / ackit replace)"
% (s.name, len(data), ", ".join(map(str, self.fill_sizes(s)))))
changes = list(changes) + [(c, data, meta) for s, data, meta in changes for c in self.copies(s) changes = list(changes) + [(c, data, meta) for s, data, meta in changes for c in self.copies(s)
if c.key not in keys] if c.key not in keys]
conflicts = [] conflicts = []

View File

@@ -65,8 +65,11 @@ tables): high risk for little gain, not done.
- **Model:** one catalog row per Wwise media ID (archive `BANK`, target = media ID), not per bank: the same - **Model:** one catalog row per Wwise media ID (archive `BANK`, target = media ID), not per bank: the same
media is stored in up to dozens of bank copies (banks are cooked into every package that uses them), and the media is stored in up to dozens of bank copies (banks are cooked into every package that uses them), and the
game may load any of them. A replacement is written into **every** copy (`BANK_locations.csv.gz`). game may load any of them. A replacement is written into **every** copy (`BANK_locations.csv.gz`).
- **In place:** each copy keeps its size (zero-padded, like WAD speech); DIDX/HIRC offsets and the package - **In place:** each copy keeps its size; DIDX/HIRC offsets and the package layout never change, so nothing
layout never change, so nothing else in the engine data has to be rewritten. else in the engine data has to be rewritten. Unlike WAD speech, an in-memory copy is never zero-padded:
the replacement fills its slot exactly with a label list chunk (the game plays a padded in-memory WEM as
silence; `docs/formats/banks.md`). Shrinking the declared sizes (DIDX and every Sound object's in-memory
size) would also work, but needs every HIRC source of the media in every bank copy located and rewritten.
- **Packages:** ackit reads the LZO packages itself (pure Python, ~47 MB/s, block-level access) and writes - **Packages:** ackit reads the LZO packages itself (pure Python, ~47 MB/s, block-level access) and writes
only the changed 128 KiB blocks, stored as a single LZO literal run (valid LZO1X without matches), then only the changed 128 KiB blocks, stored as a single LZO literal run (valid LZO1X without matches), then
fixes the chunk table. No compressor and no external tool are needed; untouched blocks stay byte-identical. fixes the chunk table. No compressor and no external tool are needed; untouched blocks stay byte-identical.

View File

@@ -111,10 +111,16 @@ and the game ties the recordings together itself:
- Catalog archive `BANK` (one row per media ID) + `data/catalog/BANK_locations.csv.gz` (every copy, and the - Catalog archive `BANK` (one row per media ID) + `data/catalog/BANK_locations.csv.gz` (every copy, and the
heads and whole copies of WAD speech recordings); built by `tools/build_catalog.py`, checked by `ackit verify heads and whole copies of WAD speech recordings); built by `tools/build_catalog.py`, checked by `ackit verify
--catalog`. --catalog`.
- A replacement is written **in place into every copy** (no larger than the smallest copy, zero-padded), so - A replacement is written **in place into every copy** (no larger than the smallest copy), so DIDX/HIRC
DIDX/HIRC offsets and sizes and the package layout never change. Loose banks are patched directly; in offsets and sizes and the package layout never change. Loose banks are patched directly; in packages only
packages only the touched blocks are re-encoded (`upk.md`). A sound left in one package only would let the the touched blocks are re-encoded (`upk.md`). A sound left in one package only would let the game load the
game load the original from another one. original from another one.
- Replacing a speech line that has prefetch heads or whole copies rewrites them (the first bytes of the new, - **Each copy is filled exactly**, never zero-padded: every in-memory WEM the game ships ends exactly at its
zero-padded line; for a whole copy, all of it). slot's end, and the game plays a replacement followed by zero padding there as **silence** (findings 24:
footsteps replaced that way were silent in game; filled, they play). A smaller replacement is grown to the
slot size with a label list chunk before `data` (`LIST{adtl, labl}`, the form the game's own WEMs use for
markers; `ackit/wem.py: fill`), so the encoder leaves at least 26 spare bytes, an even number.
- Replacing a speech line that has prefetch heads or whole copies rewrites them: a head gets the first bytes
of the new, zero-padded line (the start of the streamed WEM as the WAD stores it); a whole copy is filled
like an in-memory sound.
- Design and revert rules: `docs/decisions.md` (4b). The same WEM rules apply (`wem.md`). - Design and revert rules: `docs/decisions.md` (4b). The same WEM rules apply (`wem.md`).

View File

@@ -125,6 +125,15 @@ Conclusion: approach A works end-to-end in-game. Use >= 30 s probes for the intr
catalog had the same `<Bank>_<n>` name as an SFX WAD sound of the same bank (`CHR_RasBossFight_1`, catalog had the same `<Bank>_<n>` name as an SFX WAD sound of the same bank (`CHR_RasBossFight_1`,
`SEQ_DemonTrials_1`); they now take `<Bank>_<MEDIA>`, and a line's own name wins over another recording's `SEQ_DemonTrials_1`); they now take `<Bank>_<MEDIA>`, and a line's own name wins over another recording's
position alias (`CHR_RadioScanner#1` in FRA). Every name of every catalog resolves to its own sound. position alias (`CHR_RadioScanner#1` in FRA). Every name of every catalog resolves to its own sound.
24. (2026-10-10, gameplay test by the user) **Padded in-memory replacements play as silence.** All 568 Batman
footsteps were replaced; in game the original steps were heard and the replaced cape-open sound was silent.
Memory scan while walking: no original footstep WEM loaded, 137 replacements loaded; debugger: the
replacements' Vorbis setups are created and parsed OK (ret 0). Every untouched in-memory WEM fills its slot
exactly (RIFF size = DIDX size, 500 / 500 sampled), while the replacements were zero-padded. Filled with a
`LIST{adtl, labl}` chunk before `data` (55 in-memory originals carry cue+LIST chunks), every step plays the
replacement. ackit now fills every in-memory copy and every whole bank copy of a WAD recording
(`wem.fill`); WAD entries and prefetch heads keep zero padding (streamed playback is unaffected). The
remaining sound after each step is Batman's gear and cape foley (`CHR_BatmanFoleyBelt`, `..Cape`).
## Old path -> new path ## Old path -> new path
`scripts/harness.py` -> `ackit verify --ingame` (ackit/verify/ingame.py); `scripts/validate_builder.py` -> `scripts/harness.py` -> `ackit verify --ingame` (ackit/verify/ingame.py); `scripts/validate_builder.py` ->

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "ackit" name = "ackit"
version = "3.2.1" version = "3.2.2"
description = "Batman: Arkham City (PC) audio toolkit and editor" description = "Batman: Arkham City (PC) audio toolkit and editor"
readme = "README.md" readme = "README.md"
requires-python = ">=3.10" requires-python = ">=3.10"

View File

@@ -25,6 +25,16 @@ def wem(seconds, freq, rate=24000, ch=1):
return encoder.encode(x, sr, rate, ch)[0] return encoder.encode(x, sr, rate, ch)[0]
def fitting_wem(ws, s, seconds, freq):
"""A tone WEM that `ws` accepts for `s` (Workspace.fits: in-memory slots are filled exactly, which needs
an even number of spare bytes): the tone is shortened a little until the size allows it."""
for k in range(40):
w = wem(seconds * (1 - 0.01 * k), freq)
if ws.fits(s, w):
return w
raise AssertionError("no fitting WEM for %s" % s.key)
def _a(x, n): def _a(x, n):
return (x + n - 1) // n * n return (x + n - 1) // n * n

View File

@@ -9,6 +9,7 @@ from ackit import banks
from ackit.game import Game from ackit.game import Game
from ackit.inject import SizeLimitError from ackit.inject import SizeLimitError
from ackit.upk import Package from ackit.upk import Package
from ackit.wem import fill, unfill
from ackit.workspace import Workspace, ConflictError, MODIFIED, ORIGINAL from ackit.workspace import Workspace, ConflictError, MODIFIED, ORIGINAL
@@ -65,12 +66,13 @@ def test_bank_apply_every_copy_and_revert_byte_identical(synth_game, tmp_path):
files = [os.path.join(cooked, f) for f in ("Synth_A.upk", "Synth_B.upk")] files = [os.path.join(cooked, f) for f in ("Synth_A.upk", "Synth_B.upk")]
before = {f: _sha(f) for f in files} before = {f: _sha(f) for f in files}
s = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[0]] s = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[0]]
new = synth.wem(0.2, 1500) new = synth.fitting_wem(ws, s, 0.2, 1500)
assert len(new) < ws.limit(s) assert len(new) < ws.limit(s)
ws.apply([(s, new, {"source": "test"})]) ws.apply([(s, new, {"source": "test"})])
assert ws.state(s) == MODIFIED assert ws.state(s) == MODIFIED and ws.current_bytes(s) == new
for l in ws.locations(s): for l in ws.locations(s): # every copy filled exactly: no zero padding after the WEM
assert _copy(ws, l) == new + bytes(l.size - len(new)) assert _copy(ws, l) == fill(new, l.size) and len(_copy(ws, l)) == l.size
assert unfill(_copy(ws, l)) == new
other = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[1]] other = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[1]]
assert all(_copy(ws, l) == originals[other.target] for l in ws.locations(other)) assert all(_copy(ws, l) == originals[other.target] for l in ws.locations(other))
assert all(_sha(f) != before[f] for f in files) assert all(_sha(f) != before[f] for f in files)
@@ -79,13 +81,49 @@ def test_bank_apply_every_copy_and_revert_byte_identical(synth_game, tmp_path):
assert ws.state(s) == ORIGINAL and {f: _sha(f) for f in files} == before assert ws.state(s) == ORIGINAL and {f: _sha(f) for f in files} == before
def test_fill_slot_exactly():
"""The game plays an in-memory WEM followed by zero padding as silence: a smaller replacement is grown to
its slot with a label list chunk (wem.fill), which the game's own WEMs use too."""
from ackit import audio
from ackit.wem import FILL_MIN, Wem, fillable
w = synth.wem(0.2, 1500)
for size in (len(w), len(w) + FILL_MIN, len(w) + 1000):
f = fill(w, size)
assert len(f) == size and 8 + int.from_bytes(f[4:8], "little") == size # RIFF covers the slot
tags = [b"fmt ", b"data"] if size == len(w) else [b"fmt ", b"LIST", b"data"]
assert unfill(f) == w and [c.tag for c in Wem.parse(f).chunks] == tags
assert (audio.decode_wem(f)[0] == audio.decode_wem(w)[0]).all()
assert not fillable(len(w), len(w) + 3) and not fillable(len(w), len(w) + FILL_MIN - 2)
with pytest.raises(ValueError):
fill(w, len(w) + 1)
f = fill(w, len(w) + 40)
a = f.index(b"adtllabl")
game_like = f[:a + 12] + (1).to_bytes(4, "little") + b"END\0" + f[a + 20:] # a real marker label: kept
assert len(game_like) == len(f) and unfill(game_like) == game_like
def test_bank_apply_refuses_unfillable_and_prepare_fits(synth_game, tmp_path):
from ackit.game import GameError
root, _ = synth_game
ws = _ws(root, tmp_path)
s = ws.catalog.by_key["BANK:%08X" % synth.FOOT[1]]
bad = next(w for w in (synth.wem(0.1 * (1 - 0.01 * k), 2000) for k in range(40)) if not ws.fits(s, w)
and len(w) <= ws.limit(s))
with pytest.raises(GameError):
ws.apply([(s, bad, {})])
assert ws.store.entries == {}
x, sr = synth.tone(0.1, 24000, 2000, 1)
wem, _ = ws.prepare(s, x, sr)
assert ws.fits(s, wem)
def test_loose_bank_in_place(synth_game, tmp_path): def test_loose_bank_in_place(synth_game, tmp_path):
root, originals = synth_game root, originals = synth_game
ws = _ws(root, tmp_path) ws = _ws(root, tmp_path)
s = ws.catalog.by_key["BANK:%08X" % synth.FOOT[1]] s = ws.catalog.by_key["BANK:%08X" % synth.FOOT[1]]
path = ws.files(s)[0] path = ws.files(s)[0]
before, size = _sha(path), os.path.getsize(path) before, size = _sha(path), os.path.getsize(path)
ws.apply([(s, synth.wem(0.1, 2000), {})]) ws.apply([(s, synth.fitting_wem(ws, s, 0.1, 2000), {})])
assert os.path.getsize(path) == size and _sha(path) != before assert os.path.getsize(path) == size and _sha(path) != before
ws.revert_all() ws.revert_all()
assert _sha(path) == before assert _sha(path) == before
@@ -98,11 +136,11 @@ def test_partial_revert_keeps_other_changes(synth_game, tmp_path):
before = _sha(path) before = _sha(path)
a = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[0]] a = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[0]]
b = ws.catalog.by_key["BANK:%08X" % synth.GUI[0]] b = ws.catalog.by_key["BANK:%08X" % synth.GUI[0]]
na, nb = synth.wem(0.2, 1500), synth.wem(0.2, 1700) na, nb = synth.fitting_wem(ws, a, 0.2, 1500), synth.fitting_wem(ws, b, 0.2, 1700)
ws.apply([(a, na, {}), (b, nb, {})]) ws.apply([(a, na, {}), (b, nb, {})])
ws.revert([a]) ws.revert([a])
assert ws.current_bytes(a) == originals[a.target] assert ws.current_bytes(a) == originals[a.target]
assert ws.current_bytes(b).startswith(nb) and ws.state(b) == MODIFIED assert ws.current_bytes(b) == nb and ws.state(b) == MODIFIED
ws.revert([b]) ws.revert([b])
assert _sha(path) == before assert _sha(path) == before
@@ -136,15 +174,15 @@ def test_bank_file_changed_underneath(synth_game, tmp_path):
root, originals = synth_game root, originals = synth_game
ws = _ws(root, tmp_path) ws = _ws(root, tmp_path)
s = ws.catalog.by_key["BANK:%08X" % synth.GUI[0]] s = ws.catalog.by_key["BANK:%08X" % synth.GUI[0]]
ws.apply([(s, synth.wem(0.2, 1500), {})]) ws.apply([(s, synth.fitting_wem(ws, s, 0.2, 1500), {})])
path = ws.files(s)[0] path = ws.files(s)[0]
# another tool rewrites the package (different bytes elsewhere in it) # another tool rewrites the package (different bytes elsewhere in it)
pk = Package(path) pk = Package(path)
other = pk.patched([(pk.header_size + 10, b"OTHER-TOOL")]) other = pk.patched([(pk.header_size + 10, b"OTHER-TOOL")])
open(path, "wb").write(other) open(path, "wb").write(other)
with pytest.raises(ConflictError): with pytest.raises(ConflictError):
ws.apply([(s, synth.wem(0.2, 1600), {})]) ws.apply([(s, synth.fitting_wem(ws, s, 0.2, 1600), {})])
ws.apply([(s, synth.wem(0.2, 1600), {})], force=True) ws.apply([(s, synth.fitting_wem(ws, s, 0.2, 1600), {})], force=True)
ws.revert([s]) ws.revert([s])
assert _sha(path) == _sha(ws.store.full_copy(ws.game.rel(path))) assert _sha(path) == _sha(ws.store.full_copy(ws.game.rel(path)))

View File

@@ -9,6 +9,7 @@ from ackit import banks
from ackit.bnk import fnv1 from ackit.bnk import fnv1
from ackit.catalog import build_lines from ackit.catalog import build_lines
from ackit.game import Game from ackit.game import Game
from ackit.wem import fill
from ackit.workspace import ORIGINAL, MODIFIED, Workspace from ackit.workspace import ORIGINAL, MODIFIED, Workspace
import synth import synth
@@ -138,8 +139,9 @@ def test_whole_recording_in_a_bank_is_written_with_its_line(tmp_path):
assert l.kind == banks.PREFETCH_HEAD and l.size == len(a["INT"]) and l.file == "Lines.upk" assert l.kind == banks.PREFETCH_HEAD and l.size == len(a["INT"]) and l.file == "Lines.upk"
pkg = os.path.join(ws.game.cooked, "Lines.upk") pkg = os.path.join(ws.game.cooked, "Lines.upk")
before = hashlib.sha1(open(pkg, "rb").read()).hexdigest() before = hashlib.sha1(open(pkg, "rb").read()).hexdigest()
new = synth.wem(0.3, 900) new = synth.fitting_wem(ws, s, 0.3, 900)
ws.apply([(s, new, {})]) ws.apply([(s, new, {})])
assert banks.open_file(pkg).read(l.offset, l.size) == new + bytes(l.size - len(new)) assert banks.open_file(pkg).read(l.offset, l.size) == fill(new, l.size) # filled exactly, not padded
assert ws.current_bytes(s) == new
ws.revert([s]) ws.revert([s])
assert hashlib.sha1(open(pkg, "rb").read()).hexdigest() == before assert hashlib.sha1(open(pkg, "rb").read()).hexdigest() == before

View File

@@ -94,7 +94,7 @@ def test_subtitle_and_sound_in_one_package(synth_game, tmp_path):
ws = _ws(root, tmp_path) ws = _ws(root, tmp_path)
before = _files(root) before = _files(root)
s = ws.catalog.by_key["BANK:%08X" % synth.DLG_MEDIA] s = ws.catalog.by_key["BANK:%08X" % synth.DLG_MEDIA]
ws.apply([(s, synth.wem(0.1, 1900), {})]) ws.apply([(s, synth.fitting_wem(ws, s, 0.1, 1900), {})])
ws.texts.apply({SUB_KEY: "Both."}) ws.texts.apply({SUB_KEY: "Both."})
assert ws.texts.current(SUB_KEY) == "Both." and ws.state(s) == "modified" assert ws.texts.current(SUB_KEY) == "Both." and ws.state(s) == "modified"
ws.revert([s]) ws.revert([s])

View File

@@ -35,6 +35,7 @@ import numpy as np # noqa: E402
from ackit import banks, settings, subtitles # noqa: E402 from ackit import banks, settings, subtitles # noqa: E402
from ackit.game import Game, default_game_dir # noqa: E402 from ackit.game import Game, default_game_dir # noqa: E402
from ackit.modpack import export_mod, import_mod, match_mod # noqa: E402 from ackit.modpack import export_mod, import_mod, match_mod # noqa: E402
from ackit.wem import fill # noqa: E402
from ackit.workspace import Workspace, MODIFIED, ORIGINAL # noqa: E402 from ackit.workspace import Workspace, MODIFIED, ORIGINAL # noqa: E402
RESULTS = [] RESULTS = []
@@ -80,11 +81,12 @@ def sound_round_trip(ws, s, label):
before = snapshot(files) before = snapshot(files)
ws.apply([(s, wem, {"source": "e2e"})]) ws.apply([(s, wem, {"source": "e2e"})])
ok = ws.state(s) == MODIFIED and ws.current_bytes(s) == wem ok = ws.state(s) == MODIFIED and ws.current_bytes(s) == wem
orig_size = ws.store.get(s.key)["orig_size"]
for l, b in copies_of(ws, s): for l, b in copies_of(ws, s):
if l.kind == banks.MEMORY: if l.kind == banks.MEMORY or l.size >= orig_size: # a whole WEM in memory: fills its slot exactly
ok &= b == wem + bytes(l.size - len(wem)) ok &= b == fill(wem, l.size)
else: # head or whole copy of a WAD line (zero-padded to its size) else: # prefetch head: the start of the WAD recording (zero-padded)
full = wem + bytes(max(0, ws.store.get(s.key)["orig_size"] - len(wem))) full = wem + bytes(max(0, orig_size - len(wem)))
ok &= b == (full + bytes(l.size))[:l.size] ok &= b == (full + bytes(l.size))[:l.size]
ws.revert([s]) ws.revert([s])
ok &= ws.state(s) == ORIGINAL and snapshot(files) == before and not ws.store.entries ok &= ws.state(s) == ORIGINAL and snapshot(files) == before and not ws.store.entries