Full correctness check against the docs: safer writes, reverts and texts

Safety
- Writes are serialized (Workspace.lock): two background jobs rebuilding one
  package (a sound and its subtitle applied side by side) lost an edit.
- A store copies a bank file or package as "original" only if it matches the
  shipped catalog (Workspace.unoriginal_files: in-memory sounds, bank copies of
  voice lines, subtitles); the editor refuses to switch stores while changes
  are applied. A second store could otherwise resurrect a reverted sound.
- Mod import in the editor asks before overwriting files changed outside the
  tool (it forced silently).
- Revert all in the editor reverts texts too; the uninstaller counts text
  edits before offering to delete the store.
- ackit restore-backup: the full-backup restore decisions.md described; it
  keeps the sounds of WADs without a full copy recorded.
- build_catalog also stops when a subtitle differs from the catalog.

Languages and texts
- Game.language() returns languages without voices (JPN, RUS, ...) and
  Game.voice_archive() their INT voices; ackit text --lang JPN works.
- Menu lookups parse one cached file (find & replace: minutes -> ms).
- Subtitles decode cp1252 as Windows does: the UTF-8-in-cp1252 line
  Speech_TAPE_J_Answer_1_00 (KOR/POL/POR/RUS) is written back byte-for-byte;
  SUB_KOR/SUB_RUS rebuilt from the original game (only those rows changed).
- Mods exported while another language is loaded keep name and line.

Other fixes
- Down-mix folds extra channels in at -3 dB (a 5.1 centre voice was dropped).
- e2e_editor no longer overwrites the user's settings.json; kill_game waits
  for the game to exit before reverts.
- Recorder handles Int32/UInt8 input and split frames; package block cache
  locked; conflict messages, CLI help examples, stale docstrings and doc
  references fixed; tests no longer read this PC's game language.

Docs (decisions, user guide, README, subtitles.md, banks.md, catalog README,
REPORT, NOTES, CLAUDE.md) updated; check_docs: 102 claims match. 148 tests;
e2e_features 9/9, e2e_editor and verify PASS; game original afterwards.
This commit is contained in:
2026-10-13 21:34:13 +02:00
parent a6ef3b9458
commit 1ccaa8b398
41 changed files with 846 additions and 186 deletions

View File

@@ -38,19 +38,32 @@ loopback); it is a developer tool and not part of the editor.
- **Originals store** per game folder: `<user data>/UnheardCity/stores/<id>/` (Windows `%APPDATA%`, Linux
`~/.local/share/unheardcity`), id = hash of the game path; configurable (`--store`, editor settings).
Before a sound is first changed, its original WEM bytes (SHA-256-named blob) and metadata are saved;
the applied replacement is saved too. **Revert** re-injects the stored original; **Revert all** does this for
every record. Stream-pack padding bytes are stored so pack reverts are byte-identical.
the applied replacement is saved too. **Revert** re-injects the stored original; **Revert all** (editor toolbar,
`ackit revert --all`) does this for every record of the store, sounds and texts, in every language. Stream-pack
padding bytes are stored so pack reverts are byte-identical. The uninstaller counts sound and text records and
does not offer to delete the store while any is left.
- **One store per game at a time:** the changes are recorded only in the store that made them. The editor
refuses to switch to another store folder while changes are applied (Revert all first). A store takes the
full copy of a soundbank or package only from a file that matches the shipped catalog (every in-memory
sound's checksum, every bank copy of a voice recording, every subtitle; `Workspace.unoriginal_files`), so a
second store (`--store`) or another tool's change is a conflict, never a new "original". (Menu files and other
game builds have no catalog checksums to check.)
- **Full-file fallback:** before the first change to a WAD (and its pack index), one verified full copy is kept
in the store (`full/`), default on, can be disabled. Used by "Restore from full backup" if the store is damaged.
in the store (`full/`), default on, can be disabled. If the store is damaged, `ackit restore-backup --yes`
copies the WADs back (their sounds are forgotten) and rebuilds the soundbanks, packages and localization files
from their copies, keeping the sounds of WADs without a full copy recorded and revertible.
Soundbank files, packages and localization files always get this copy (they are rebuilt from it, 4b/4c).
- **Why the user data folder:** it survives app updates and reinstalls, needs no write access next to the game,
and works the same on Linux/Proton. Steam never touches it.
- **Safe writes:** every WAD is rebuilt into a temp file and atomically renamed over the original, pack indexes
first. Several sounds of one WAD are written in one pass. A locked WAD (game running) gives a clear message.
Writes are serialized (`Workspace.lock`): the editor runs them as background jobs, and two rebuilds of one
package at once (e.g. a sound and its subtitle applied side by side) would otherwise lose one edit.
- **Changed underneath:** each sound's state is computed from its bytes: original / modified (ours) / lost
(ours was replaced by the original, e.g. Steam "verify") / changed (neither: update or another tool) /
foreign (never touched by us, but not the catalog's original checksum). Writing over changed/foreign sounds
requires explicit confirmation; the catalog is used only when the WAD fingerprint matches.
or texts requires explicit confirmation (editor: a question, also when installing a mod; CLI: `--force`); the
catalog is used only when the game's fingerprint matches.
## 4. Size rule
Speech/ambience variations are addressed by offset+size from BKPK tables inside compressed `.upk` packages,
@@ -77,7 +90,7 @@ tables): high risk for little gain, not done.
optional like the WAD copies: the file is rebuilt from it). The file content is always "original copy +
replacements applied now", so reverting the last one restores the original bytes exactly, and an interrupted
operation is repaired by the next one. Conflicts: the whole-file SHA-1 must be the original's or the last one
ackit wrote.
ackit wrote, and before the copy is taken the file must match the catalog (section 3).
- **Prefetch heads and whole copies:** 396 voice recordings (81 lines, 78-80 recordings per language) have
their first bytes copied into banks, and 49 more are stored whole in banks (same media ID, identical bytes);
a replacement of such a recording rewrites those copies too, otherwise the game could play the old line.
@@ -130,8 +143,9 @@ copy per voice language) is filtered the same way, so the sound list never mixes
WAD voice lines), numbered `NAME#2` in the event's own order when it plays two versions; the other 637 keep
the name of their English (INT) recording's position. The position in each language stays an alias (`NAME#i`).
- A recording stored twice in a WAD (5 per language) is written and reverted in both places.
- The catalog records the ORIGINAL bytes: `tools/build_catalog.py` stops before writing anything if a sound
differs from the current catalog (a modified game), unless `--accept-changes` (a game update).
- The catalog records the ORIGINAL bytes and texts: `tools/build_catalog.py` stops before writing anything if a
sound or a subtitle differs from the current catalog (a modified game), unless `--accept-changes` (a game
update).
- Category scheme: `docs/categories.md`.
## 6. Repository cleanup choices

View File

@@ -57,7 +57,7 @@ WAD sounds have such heads (2,101 copies); replacing one of them must rewrite th
story lines) are
also stored **whole** in banks: an in-memory source with the WAD recording's own media ID, byte-identical to
the WAD WEM (84 copies). They are handled like heads: part of the WAD line, written and reverted with it, not
sounds of their own (`catalog._whole_recordings`). Heads and whole copies together: 445 recordings, 2,185
sounds of their own (`catalog.whole_recordings`). Heads and whole copies together: 445 recordings, 2,185
copies (`BANK_locations.csv.gz` kind `p`).
## Localized banks

View File

@@ -14,6 +14,10 @@ the UE3 packages (`upk.md`), holding the subtitle text of all 11 languages. acki
to 5 lines.
- Strings are stored as cp1252 (positive FString length) or UTF-16 (negative length): almost all RUS, JPN and
KOR strings and 85 % of POL are UTF-16; the other languages have 17-56 UTF-16 strings each.
4 cp1252 strings hold UTF-8 bytes instead (the same line in KOR, POL, POR, RUS: `Speech_TAPE_J_Answer_1_00`;
the KOR one with a damaged byte; the game's own data). ackit reads cp1252 as Windows maps it (every byte, also
the 5 that Python's cp1252 leaves undefined, which the RUS and KOR ones contain), so they are shown as stored
and every string is written back byte-for-byte.
## Scripted subtitles (`RSubtitle`)
41 more subtitles (radio and encryption-card lines) are `RSubtitle` objects in `Startup.upk`: the same layout,

View File

@@ -58,8 +58,9 @@ the same line and are changed and reverted together.
## Listen
The editor under the list shows the selected sound in three columns side by side (drag the dividers to
resize them): **Listen**, **Replace** and **Subtitle**. Select a row and press **Space** (or double-click) to
play the original. Each column ends with its own buttons on one line. In **Listen**:
resize them): **Listen**, **Replace** and **Subtitle**. Select a row and press **Space** to play the original
(once a replacement is loaded, Space plays the replacement; Space again stops), or double-click it. Each column
ends with its own buttons on one line. In **Listen**:
- **Original** / **Installed** play the game's original and what is currently in the game; **Stop** (Esc).
Click on a waveform to play from that point.
- Above them: **A/B** (switch between the original and your replacement at the same position, to compare),
@@ -82,14 +83,22 @@ than the original, since the game may cut it off.
## Undo
- **Revert** in the editor puts that sound back exactly.
- Right-click rows → **Revert selected**; toolbar **Revert all** puts every original back.
- Right-click rows → **Revert selected**; toolbar **Revert all** puts every original back: all sounds,
subtitles and menu texts you changed, in every language.
- Originals are kept in your user data folder (`%APPDATA%\UnheardCity` on Windows,
`~/.local/share/unheardcity` on Linux), so reverting works after updating or reinstalling the app too.
Uninstalling keeps this folder; use **Revert all** before uninstalling if you want the game back to original.
Before the first change to a game file, a full copy of it is kept there too: optional for the WAD archives
(Settings), always for soundbanks, packages and text files (they are rebuilt from that copy).
(Settings), always for soundbanks, packages and text files (they are rebuilt from that copy). Only files the
game ships unchanged are copied; a file another tool (or another originals store) has changed is reported as
*changed outside this tool* first.
- The originals of your changes are only in the store that made them, so the store folder (Settings) cannot be
changed while changes are applied: use **Revert all** first.
- If Steam "Verify integrity of game files" or an update undoes your changes, a bar offers **Re-apply my
changes** or **Forget them**. Do not use "verify" to undo mods; use Revert all.
- If the store folder itself is damaged, the full copies can still be put back with the command-line toolkit
that Unheard City is built on (`ackit restore-backup`, see the project README; not part of the editor
download).
## Subtitles
The **Subtitles** tab lists the subtitle of every speech line in the selected language (see *Language*).
@@ -123,6 +132,7 @@ Click the **★** column to mark favorites. These are stored in your user data f
- **Export mod…**: choose which changed sounds and texts to include and save a `.acmod` file (the exact game
data plus the audio, so it also works for another language whose original has a different size).
- **Import mod…**: shows which sounds match this game, then installs them. You can revert them like any change.
If a sound or text the mod replaces was changed outside the editor, you are asked before it is overwritten.
- Changed subtitles and menu texts are exported and imported with the sounds (listed as *Text*); a text stays
in its language (a French subtitle edit changes the French subtitle).
- In another language, voice lines are matched by the game's own identity of the line, so the same line is
@@ -131,7 +141,8 @@ Click the **★** column to mark favorites. These are stored in your user data f
## Settings
![Settings](img/settings.png)
Game folder, where originals are kept, the full-backup option and the playback device. The recording device
Game folder, where originals are kept (changeable only while nothing is modified), the full-backup option and
the playback device. The recording device
is chosen next to the Record button. **Background**: show or hide the Arkham City wallpaper behind the filter
panel; **Background strength**: how visible it is (10 % by default).
The editor panel opens tall enough to show all its controls; drag the divider above it to change that.