Version 3.4.0: Edit tab (edit list, large subtitle, replacements with effects and presets), Sounds tab only browses, ackit fx (pedalboard + own reverb) and replace --fx, first Linux build, catalog without a mod package

dist/UnheardCity-3.3.0-windows-amd64.zip (86 MB): smoke test PASS with PATH reduced to the Windows folders (21,015 sounds, 9,377 subtitles, 7,020 menu texts), GUI start/quit PASS. REPORT and NOTES updated.

.claudio

@@ -0,0 +1,7 @@
+MOUNTS=(
+    "$HOME/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Batman Arkham City GOTY"
+)
+GUI=1
+AUDIO=1
+ROOT=1
+PERSIST=1

CLAUDE.md

@@ -4,13 +4,26 @@ Batman: Arkham City (PC) audio toolkit + editor. Read `NOTES.md` first (status,
 
 - Layout: `ackit/` library + CLI (`python -m ackit -h`), `app/` PySide6 editor, `tests/` pytest, `data/catalog/`
   CSVs (+ user's original lists in `data/catalog/source/`, never edit them), `docs/`, `tools/` build scripts.
-- Dev env: `.venv` (Python 3.14, `pip install -e .[app,dev]`). Run tests with `.venv/Scripts/python -m pytest -q`.
+- Dev env: `.venv` (Python 3.14, `pip install -e .[app,dev,verify]`). Run tests with `.venv/Scripts/python -m pytest -q`.
-- Writing game files goes through `ackit.workspace.Workspace` only (originals store, atomic writes, conflict checks).
+- Writing game files goes through `ackit.workspace.Workspace` only (originals store, atomic writes, conflict checks,
+  one write at a time: `Workspace.lock`, held by every apply/revert/rebuild incl. `Texts`).
   Speech/ambience: in place, never larger than the original WEM; music packs: any size (`docs/formats/wad.md`).
+  In-memory sounds (archive BANK, loose `.bnk` + `.upk` packages): in place into every copy; a bank file is always
+  rebuilt from the store's original copy + the applied replacements (`docs/formats/banks.md`, `upk.md`).
+  Texts go through `Workspace.texts` (subtitles in packages, in place and NUL-padded; menu `.int` files).
 - Never use Steam "verify integrity". Keep game files original after tests (`ackit status`, `ackit revert --all`).
-- Real-game checks: `ackit verify` (offline), `ackit verify --ingame` (Windows; needs Steam running and logged in,
+- Real-game checks: `ackit verify` (offline), `ackit verify --catalog [--quick]` (catalog vs game files),
-  WinDbg; ~1.5 min; reverts and checks byte-identity itself).
+  `ackit verify --ingame [--banks | --texts]` (Windows; needs Steam running and logged in, WinDbg; ~1.5 min; reverts and
-- Rebuild catalogs only from original game files: `python tools/build_catalog.py`.
+  checks byte-identity itself).
+- Rebuild catalogs only from original game files: `python tools/build_catalog.py` (~5 min, scans all packages;
+  stops if a sound or subtitle differs from the current catalog). Voice lines are identified by the game's Wwise Sound
+  objects (`ackit/lines.py`, column `line`; one recording per language), never by WAD position or name:
+  positions differ per language (`docs/decisions.md` 5, `docs/formats/banks.md` *Lines*).
+- Figures in the docs are checked, not copied: after changing data, code or docs run `python tools/check_docs.py`
+  (`--scan`: also the bank survey and the catalog check's facts, ~10 min; `--wems`: every WEM rebuilt, ~10 min);
+  add a claim there for every new figure.
+- Real-game write round trips: `python tools/e2e_features.py`, `python tools/e2e_editor.py` (they revert in a
+  `finally` block; still check `ackit status` after any interrupted run).
 - Shell: git at `C:\Program Files\Git\cmd` (prepend to PATH). Write Python with the Write/Edit tools; avoid
   PowerShell here-strings and heredoc string surgery with backslash escapes (they produced NUL bytes before).
 - The speaker loopback can contain other apps' audio (voice chat): never save raw captures.

LICENSE

@@ -0,0 +1,674 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+the GNU General Public License is intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights.  Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received.  You must make sure that they, too, receive
+or can get the source code.  And you must show them these terms so they
+know their rights.
+
+  Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+
+  For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software.  For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Use with the GNU Affero General Public License.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+    <program>  Copyright (C) <year>  <name of author>
+    This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU GPL, see
+<https://www.gnu.org/licenses/>.
+
+  The GNU General Public License does not permit incorporating your program
+into proprietary programs.  If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.  But first, please read
+<https://www.gnu.org/licenses/why-not-lgpl.html>.

NOTES.md

@@ -14,10 +14,237 @@
   Packaging: `python tools/build_app.py` -> dist/UnheardCity (self-contained, licenses); packaged exe passes
   `--smoke-test` with PATH = C:\Windows only. Real e2e through the editor code path (tools/e2e_editor.py): PASS
   (music heard in game ncc 1.0, reverts byte-identical, speech line apply/revert byte-identical).
-- NEXT: editing of the ~15,800 in-memory sounds in soundbanks (loose `SFX\*.bnk` footsteps first, then banks
+- Mission 3 (2026-10-09, brief: `docs/history/mission3_audio_text_prompt.md`): full-game audio, then subtitle
-  embedded in `.upk` packages); not listed or editable yet, survey in `docs/formats/banks.md`.
+  and menu text editing. Plan (one commit per stage; a stage = code + tests + docs + this status):
-- NEXT (optional): verify >2-channel channel order in game; speech heard in-game (needs gameplay).
+  1. [x] `ackit/lzo.py`, `ackit/upk.py` (UE3 package summary, compressed chunks, block-level read/patch),
-- OPEN: channel order for >2 channels unverified.
+     `ackit/bnk.py` (bank parsing: DIDX/DATA/HIRC sources). Docs: formats/upk.md, banks.md. Findings: pure-Python
+     LZO ~47 MB/s (== decompress.exe on every block); 15,800 media IDs in 96,011 copies, 396 are prefetch heads
+     of WAD sounds (2,101 copies) -> 15,404 in-memory sounds; copies identical except 1C898731.
+  2. [x] Catalog archive `BANK` (15,404 sounds, 6,416 named by `Play_` events; now 15,355, 13,501 named by events) + `BANK_locations.csv.gz` (93,910 copies,
+     2,101 prefetch heads of 396 speech lines); `ackit verify --catalog` (full: catalog OK; fixed 2 duplicate
+     names per language: variation shown as NAME#i); categories for bank prefixes.
+  3. [x] Workspace writes BANK sounds into every copy and keeps speech-line prefetch heads in sync (bank files
+     rebuilt from a mandatory full original copy); editor shows bank + copies. `ackit verify` PASS (300/300
+     BANK WEMs rebuilt identically); `ackit verify --ingame --banks` PASS (loaded, no crash, revert identical;
+     audible check not possible without input: findings 18).
+  4. [x] Subtitles = RDialogueEvent objects in packages (9,016 lines x 11 languages, 168k copies; parser
+     `ackit/subtitles.py`, catalog SUB_<LANG>.csv + SUB_locations.csv.gz, 92 % linked to speech sounds via
+     events; now ~99 %); menu text = Localization/<LANG>/*.int (`ackit/loctext.py`, ~7,000 strings/language).
+     `Workspace.texts` (ackit/texts.py): states, apply/revert, conflicts, mod packages; CLI `ackit text`.
+     Subtitles in place (NUL-padded, limit = original length); decision + reasons in decisions.md 4c.
+  5. [x] Editor: tabs Sounds / Subtitles / Menu text (app/textpanel.py: search, filters, edit with room
+     counter, apply/revert/revert all, find & replace on the filtered rows, go to sound); the sound editor
+     shows and edits the sound's subtitle (SubtitleBox); texts in mod export/import and the changed-outside
+     banner; smoke test loads texts. 15 GUI tests.
+  6. [x] Final pass 2026-10-09: `ackit verify` PASS, `--catalog` OK, `--ingame` PASS (ncc 1.0), `--ingame
+     --banks` PASS, `--ingame --texts` PASS (12/12 menu markers in memory), tools/e2e_editor.py PASS; game
+     original after. Version 3.0.0; dist/UnheardCity-3.0.0-windows-amd64.zip (86 MB) smoke + GUI start PASS
+     with PATH = C:\Windows. 121 tests. REPORT.md rewritten (unresolved items).
+- 2026-10-09 (after user test): in-memory speech is localized (one AkBank export = language list + one bank
+  per language; ~1,460 lines per voice language): BANK.csv has a `language` column, the catalog shows shared
+  sounds + the loaded language, names unique per language, subtitles of in-memory lines link to them (links
+  92 % -> 99 %). Editor: one Language setting at the top (always starts at the game's language; languages
+  without voices use INT voices), text tabs follow it.
+- 2026-10-10 (user asked "is it really all in order?"): variation positions differ per language (75 % of lines)
+  -> speech sounds named after the game's line (speech event), curated labels mapped through it (characters.csv
+  gets a language column), verified 955/955 against dialogue speaker fields; RSubtitle + RSubtitleLookup
+  (key = FNV-1 of the line name without Speech_) added; editor redesigned: Listen | Replace | Subtitle columns,
+  editor spans the full window width (no scrolling at 1000x700 and up).
+- 2026-10-10 final check: build_catalog refuses to run on a modified game (`--accept-changes` after an update);
+  every test tool reverts in `finally`; mods from another language are not matched to position-named lines;
+  `tools/e2e_features.py` (8 real-game round trips) PASS; `--ingame --texts` also finds 23/23 Startup.upk
+  subtitle markers in memory; `--ingame`, `--banks`, e2e_editor PASS; `verify --catalog` OK; 124 tests; game
+  original after. Docs brought up to date.
+- 2026-10-10 (user: "make it complete for every language, fact check everything"): voice lines are identified
+  by the game data alone (`ackit/lines.py`, catalog column `line`): per-language BKPK tables (they were merged
+  by media ID, and 657 shared IDs let ESN overwrite the others), prefetch sources in the Sound-object map,
+  recordings stored twice written together, multi-language in-memory media, subtitle objects naming events.
+  12,434 lines, each with exactly one recording in all 5 voice languages; 3,366 speaker labels in every language;
+  subtitles link every version of a line; mods move between languages by line identity. `ackit verify
+  --catalog` re-derives all relations and prints facts (independent checks: bank language of every source,
+  prefetch heads = start of their recording, copies identical, speaker field of the dialogue objects, menu keys
+  per language). `tools/e2e_features.py --lang all` 78/78; `--ingame`, `--banks`, `--texts`, e2e_editor PASS;
+  127 tests; game original after. Speaker labels vs the dialogue speaker field: 2,766 agree, 19 differ (listed in
+  categories.md, for the user to check by listening; the source lists are never edited). The user's guide files
+  (`ArkhamCityModding/Audio/AbdAshes' ...`) hold no relation data: their hashlist was ackit/data/hashlist.txt,
+  the scripts only unpack (wemscan = position names). `data/catalog/source/ita/Checkpoint.txt` was found deleted
+  (no code touches source/; cause unknown) and restored from git. Version 3.1.0 (minor):
+  dist/UnheardCity-3.1.0-windows-amd64.zip smoke PASS. e2e_features now deletes its temporary stores (GBs of
+  full copies) when everything reverted.
+- 2026-10-10 (user: order in the browser "messed up" after filter changes; redundant files): the proxy had
+  dynamic sorting off, so rows a filter showed again were appended at the end -> dynamic sort on (+ GUI test).
+  ackit/data/hashlist.txt removed: its 2,544 names the catalog uses are exactly the game's CookedBanks.txt bank
+  names (WAD entry ID = FNV-1 of the bank name), so names now come from the game (`catalog.bank_names`);
+  rebuilt catalogs identical.
+- 2026-10-10 (user: filtering/sorting choppy, add spinners like the audio preview): QSortFilterProxyModel
+  replaced by `app/models.py: RowView` (Python filter + one sort over cached keys): sound list 1.3-1.5 s ->
+  10-20 ms per sort/filter, text lists 0.26-0.7 s -> ~10 ms. Spinner overlay (`spinner.BusyOverlay`) over the
+  sound list while it loads and over the subtitle/menu lists, which now load as background jobs (a selection
+  asked for meanwhile is applied when the rows arrive). GUI tests for order, selection and the overlays.
+- 2026-10-10 version 3.2.0: docs use INT (the game's English) for examples and figures; facts measured on this
+  PC's Italian install stay labelled as such. User-guide screenshots in INT (`tools/screenshots.py --lang`,
+  default INT; the app itself still starts in the game's language). Every WEM of the 5 speech WADs rebuilt
+  from the rules: 54,079 / 54,080 identical (INT FE9963E8:74: avg-bytes/s field one lower, informational).
+  Removed unused code (helpers/constants nothing called, the key-posting debugger helper, a double refilter
+  after changes); old dist zips, logs/ and caches deleted.
+- 2026-10-10 final check (user: "is everything fact checked ... functioning ... documentation correct?"):
+  every number in the docs recomputed from the catalog and a fresh game scan; stale ones fixed (positions differ
+  for 81.2-81.6 %, not 75 %; 8,092 neutral BANK sounds; 24 unlinked subtitle names; 637 position-named lines;
+  14,802 package media; subtitle coverage 9 % combat barks, 23 % cinematics; speaker coverage by line). Bugs
+  found and fixed (findings 23): 49 WAD recordings also stored whole in banks were separate BANK sounds (now
+  bank copies of the line, written with it); 8 BANK/SFX name clashes; name lookup preferred no name over an
+  alias (CLI "ambiguous" for e.g. `CHR_RadioScanner#1` in FRA). `ackit verify --lang` now applies (catalog: that
+  language only; in-game: must be the game's language). Subtitle error message says Windows-1252, not Latin-1.
+  Catalog rebuilt from the original files; `verify --catalog` OK (2,185 / 2,185 bank copies of WAD recordings
+  identical), e2e_features --lang all 89/89, e2e_editor, --ingame/--banks/--texts PASS, 132 tests; game original
+  after. Version 3.2.1: dist/UnheardCity-3.2.1-windows-amd64.zip smoke + GUI start PASS.
+- 2026-10-10 re-check (user: "fact check everything again"): the catalog rebuilt from the game is identical to the
+  shipped one (every CSV and location index), and `verify --catalog` gives the same facts. Figures in the docs are
+  now checked by `tools/check_docs.py` (catalog, index.json, menu files; `--scan`: bank survey + catalog-check
+  facts; `--wems`: every WEM rebuilt); the build records the scan-only figures in index.json (names by event,
+  dialogue objects, WAD lines named after a recording). It found what hand-copied figures had wrong: 61,510
+  "recordings" counted the 25 second places (61,485 recordings; the stat was fixed too), 81.2-81.6 % (not 81-82),
+  the loose/embedded bank-name overlap, REPORT's test count, wem.md's SFX claim (626 / 651 rebuild, not "all but 4
+  looping tracks": 14 music seek tables, 5 music sample counts, 6 ambience `smpl` loops) and its extra-chunk count
+  (722 WEMs, not 37). New test: cross-references between the shipped files.
+- 2026-10-10 user gameplay test: replaced voice lines and subtitles WORK in game. Batman footsteps do NOT:
+  all 568 BANK sounds of the CHR_BatmanFootsteps* banks were replaced with Downloads\boom.mp3 (trimmed per sound,
+  still APPLIED, with the user's own test subtitle SUB:ITA:Speech_CHURCH_B1_CH2_HQ_INTRO2_08; revert with
+  `ackit revert --all` when the user says so). In game the boom was heard once, briefly; otherwise the original
+  steps. Checked: every copy (40 files) holds the new bytes; replacements are loud (-8.8 dB RMS vs -20..-33);
+  no other media ID has the same original audio; no game data outside CookedPCConsole. The named footstep
+  events (Play_BM_Footstep_Walk/Run/Land/...) reach only the 46 media of bank CHR_BatmanFootsteps (replaced);
+  the 522 surface media (CHR_BatmanFootsteps_<Surface>_Media, loose SFX\*.bnk) are reached by no named event
+  (switch containers not followed by event_objects?). NEXT: find where the game gets the step audio it plays
+  (switch/random containers and their children, banks loaded at runtime, a memory capture of a step in game).
+- 2026-10-10 footsteps solved (findings 24): in-memory replacements were zero-padded in their slot and the game
+  plays that as silence; ackit now fills each in-memory copy exactly with a LIST/adtl/labl chunk (`wem.fill`,
+  `Workspace.fits/fill_sizes/prepare`). Verified by the user in game (a beep on every step), e2e_features 89/89.
+  Version 3.2.2. The footstep test was reverted (game original).
+- 2026-10-11 version 3.2.3: editor layout (user: controls "placed in line and go to new line, not pretty"):
+  aligned grid for the edit fields, every button row fills its column, each column ends with a full-width
+  action bar level with the others; A/B and Loop are icons next to the volume. Fits a 1000-px window
+  without horizontal scrolling (minimum 984 px). Docs: recordings the game shares between languages (user saw
+  English lines "leaking" into other languages: the game's own data, banks.md *Lines*).
+- 2026-10-13 (user: "full check of the source code for correctness, also against the documentation"): figures
+  all matched (`check_docs --scan`), the formats code matched the docs; fixed: editor **Revert all** reverted
+  sounds only (docs and uninstaller promised everything; the uninstaller also counted only sound records and
+  could offer to delete the store while texts were modified) -> sounds + texts, uninstaller counts texts;
+  concurrent background writes to one package lost an edit (reproduced) -> `Workspace.lock` serializes every
+  write; editor mod import forced over conflicts silently -> asks; a second originals store (Settings / `--store`)
+  copied a package modified by the first as its "original" (reproduced: a replaced sound survived both reverts)
+  -> `Workspace.unoriginal_files` (catalog checksums, heads, subtitles; 1,447 files, no false alarm, ~15 s for
+  all) makes it a conflict and the editor refuses a store change while changes exist; `ackit text --lang JPN`
+  (text-only languages) failed -> `Game.voice_archive`; menu lookups reloaded all ~37 files per call (find &
+  replace of "the" in INT: ~4 min frozen) -> one cached file per lookup (0.04 s for 1,193 keys);
+  build_catalog's safety check ignored subtitles -> also compares them; `ackit restore-backup` added (decisions
+  named a restore that nothing exposed; it also forgot WAD sounds without a full copy); recorder handles
+  Int32/UInt8 input and split frames; stale docstrings, `ackit -h` examples, conflict message, case-insensitive
+  Localization path in `verify --catalog`. 146 tests. During the checks an e2e_editor game launch failed
+  ("code never decrypted") and its revert found the WAD still locked by the dying game (left Music_393BC53A
+  modified; reverted by hand, SFX catalog OK): `debugger.kill_game` now waits until the game has exited.
+- 2026-10-13 second full check (user: "do this again to make sure"): codec (wwvorbis, codebooks, allocsize, ogg),
+  classify vs categories.md, verify harness, tools and widgets re-read; fixed: `tools/e2e_editor.py` saved its own
+  settings over the user's settings.json (lost store/device/background settings; it now uses the user's
+  settings and never saves); `Game.language()` ignored languages without voices (a JPN game opened the editor
+  and `ackit text` in INT; now the game's language, `Game.voice_archive` for its speech archive) and the tests
+  read this PC's UserEngine.ini (conftest isolates it); a mod exported while another language was loaded lost
+  name + line of that language's changes (`Workspace.sound_for_key`); down-mix dropped the extra channels (a
+  5.1 centre-only voice became silent stereo; now folded in at -3 dB, peak-limited); `Texts.for_sound` default
+  for shared in-memory lines; package block cache locked (concurrent readers); `catalog._whole_recordings` ->
+  `whole_recordings` in docs/comments; the user guide no longer sends app users to the CLI-only restore.
+  148 tests.
+- 2026-10-13 third check (property checks on the real game data): every menu file rewrites byte-identically,
+  no ambiguous menu keys, every subtitle copy has the catalog's room; 2 of 103,658 subtitle strings could not be
+  written back as stored: one line (`Speech_TAPE_J_Answer_1_00`) is UTF-8 bytes in a cp1252 string in KOR, POL,
+  POR, RUS (the game's data; KOR's damaged), and the RUS/KOR ones contain the 5 bytes Python's cp1252 leaves
+  undefined (decoded as U+FFFD, lossy) -> cp1252 as Windows maps it (lossless; all 103,658 reproduce), catalog
+  rebuilt from the original game (`--accept-changes`: exactly those 2 rows changed), subtitles.md documents it
+  (claim derived by check_docs). User guide: Space plays the replacement once one is loaded (not always the
+  original); tooltip too. App smoke test on the real game PASS.
+- 2026-10-13 version 3.3.0 (minor: `ackit restore-backup`; Revert all includes texts; the editor opens in a
+  text-only game language; mod import asks; store folder locked while changes exist):
+  dist/UnheardCity-3.3.0-windows-amd64.zip (86 MB) smoke test PASS with PATH = C:\Windows only (21,015 sounds,
+  9,377 subtitles, 7,020 menu texts, ITA), GUI start/quit PASS.
+- DONE by the user in game: replaced voice lines, edited subtitles and replaced footsteps (after the fill fix).
+- Mission 4 (2026-10-27, plan: `docs/history/mission4_dubbing_plan.md`): dubbing every character's lines with
+  the game's voice effects, inside Unheard City (Dubbing tab), engine in ackit, effects from pedalboard.
+  Stage 0 (evidence, read-only) IN PROGRESS on the Linux container: HIRC effect survey (effect slots of sounds,
+  containers, buses; effect objects) + signal features of every speech recording, then listening per group.
+  2026-10-27 first results (scratch scripts, INT; not yet in docs/check_docs): v56 layouts from wwiser (node
+  base params: u8 override, u8 count, [u8 bypass, count x {u8 index, u32 fx id, u8 shareset, u8 rendered}],
+  u32 bus, u32 parent; Sound: after the source + u8 source bits; bus: u8 isEnv at +33, ducks 17 B, then fx).
+  Effect objects (type 0x12/0x13): RoomVerb 84 + Matrix Reverb 6 + Delay/Meter/ML1/EQ all in Init (env
+  reverbs = run time); McDSP FutzBox share-set 4272216A in 1,125 speech banks, on 464 actor-mixer slots, run
+  time with RTPC curves -> 2,858 speech rows (2,358 WAD + 500 in-memory) are clean in the files (wide band;
+  record dry). Rendered (baked) slots: 33 speech rows (car radio 25, radio scanner 3, museum thug-to-Penguin 4,
+  Steel HQ 1); their fx id is 0 (type lost). Every speech row goes through the ML1 limiter on the top bus.
+  Most baked effects are studio-made (no metadata): signal features vs each speaker's widest-band lines AND vs
+  all speakers (a speaker whose every line is processed has no dry baseline: Mr. Freeze was missed by the
+  speaker-relative test alone) + a "dark" flag (< 3 kHz, Clayface) give 77 groups of >= 6 lines (169 in all,
+  3,273 recordings; filters: Freeze, Strange/Mayor PA, Joker PRED, Two-Face court, radio scanner, Penguin,
+  Zsasz, copter, Oracle, tapes; echo: Penguin exhibit, Riddler cameo, Strange TV; multichannel layouts).
+  Not measured yet: pitch/formant shift, distortion, modulation (tremolo/flanger), noise beds; every flagged
+  line is listed in `flagged_lines.csv` so small groups are not lost.
+  Stage 0 DONE (2026-10-27): the user listened to the 77 groups (results + scope change in the plan, sections
+  9-10): manual controls for every effect heard, no automatic profiles/FX.csv.
+- Mission 4 stages 1-2 DONE (2026-10-27, Linux container): `ackit/fx.py` (pedalboard 0.9.26 + numpy; 10 blocks:
+  pitch/formant/drift, octave layers, robot, glitch, voice band EQ (Butterworth via FFT: pedalboard's cuts are
+  first order and cost 3.7 dB in the pass band when stacked), distortion/GSM, noise, echo, reverb + reverse
+  swells, channel layout in WAVE mask order; seeded; `BLOCKS` drives the UI), CLI `ackit fx-template` +
+  `replace --fx`; Dubbing tab (`app/dubbing.py`, replaced the same day by the Edit tab, below: speech list with character/category/dub-state filters, a second
+  EditorPanel with the effects rack, takes + settings per sound key in user data `dubs/`, "use these effects for
+  the selected lines"); EditorPanel renders effects in the background (cached) between trim and loudness, apply
+  meta records the effects; group-box indicators styled like checkboxes. 170 tests (19 fx, 2 dubbing GUI, 1 CLI);
+  `tools/e2e_features.py --lang INT` 10/10 incl. new check 8 (5-ch dubbed line: decoded from the game file, C dry,
+  SL echo 0.160 s later; reverted byte-identical); game original after, `verify --catalog --quick` OK;
+  check_docs all match; smoke test renders effects; first Linux build (`dist/UnheardCity-3.3.0-linux-x86_64.tar.gz`,
+  124 MB) smoke + GUI start PASS with PATH=/usr/bin:/bin. User guide: Dubbing section + docs/img/dubbing.png
+  (`tools/screenshots.py --only dubbing`; the other images were not re-rendered on Linux).
+  NEXT: user tries the tab and tunes the effect defaults by ear; Windows build + in-game listening of a dubbed
+  line (needs the Windows PC); version bump (3.4.0) when released. Not done: hint per line that the game adds an
+  effect at run time (FutzBox lines; data known from stage 0, would need the HIRC survey in the catalog build).
+- 2026-10-27 UI revision (user: subtitle box too small, Sounds and Dubbing redundant, effects panel "raw",
+  reverb quality): browsing and editing split. Sounds = browser + compact preview (app/preview.py: details,
+  subtitle text, original/installed waveforms, Edit / Add to edit list; Enter = Edit); Dubbing renamed Edit and
+  made universal (app/edit.py: edit list of the sounds sent from the browser, Previous/Next, Alt+Left/Right,
+  takes + settings in user data `edits/`, old `dubs/` moved there once); editor relaid out for the Edit tab
+  (SCRIPT = ScriptBox, large 3-line text + original + English; LISTEN rows with play buttons; YOUR TAKE; action
+  bar pinned below the scroll area); effects panel app/effects.py (looks, per-block preset buttons, sliders with
+  readable values, log for Hz/ms/s, double-click reset, wheel only when focused); fx: own reverb (generated room
+  response, 4 bands, early reflections; RT60 measured back within 15 %) instead of Freeverb, PRESETS + LOOKS.
+  `MainWindow.editor` = the Edit tab's editor. 175 tests (with the real game); check_docs match; smoke + Linux
+  package PASS; docs/img main/browse/editor re-rendered on Linux (dubbing.png removed); e2e_editor uses a
+  temporary edit store.
+- 2026-10-27 second UI round (user: preview player buttons scattered; no scrolling in the Edit tab; generic
+  terms, not dubbing ones): preview = details | player (Play original, Play installed, Stop together above the
+  waveforms) | Edit, Add to edit list. Effects panel = preset list + effect chain (QListWidget: tick = on, click
+  = settings) + one settings page (QStackedWidget); moving a slider turns its effect on. Editor compacted
+  (2-line subtitle, room counter in the button row, 44 px waveforms that grow with the window, tighter
+  spacing, edit list 250 px): no scroll bars at 1400 x 900 and 1280 x 800 (GUI test). Wording: replacement
+  (not take), effects (not voice effects), Subtitle (not script), Band-pass EQ; code too (EditStore
+  has/load/save_replacement, item field `source`, folder `edits/replacements`; earlier `take`/`takes` data
+  moved on load, tested; SubtitlePanel/SubtitleEdit). Theme: list check boxes styled like the others.
+- 2026-10-27 version 3.4.0 (minor: Edit tab, effects, `ackit fx-template` / `replace --fx`): every Markdown
+  document checked (links, images, named files; REPORT.md rewritten for mission 4; check_docs gained effect
+  claims: presets, effect names, the pedalboard pin, the no-scroll window sizes); generated files removed
+  (listening WAVs, 3.3.0 builds, scratch survey data). `dist/UnheardCity-3.4.0-linux-x86_64.tar.gz` (124 MB)
+  smoke + GUI start PASS with PATH=/usr/bin:/bin; tested by the user.
+  Catalog: `FEChar_Batman_AKV8_SF.upk` was a mod the user once installed, not a game file (absent from the clean
+  Linux install); rebuilt from that install (`tools/build_catalog.py`, no sound or subtitle differs): its 124 bank
+  copies left BANK_locations.csv.gz (every sound in it is also in original packages: same 15,355 sounds, 93,702
+  copies in 1,498 files), docs figures updated.
+  Reverb (user: "no mix slider"): the level was "Amount" and never fully wet (dry 1 - wet/2) -> "Mix" (key `mix`,
+  2nd slider), equal-power dry/wet (0 = dry only, 1 = reverb only); `wet` of older settings is read as `mix`
+  (fx.RENAMED); presets scaled x0.73 to sound as before. Effect chain: the selected row is grey, so its red tick
+  shows. Final check: 177 tests (with the game), `ackit verify` PASS, `verify --catalog` OK, e2e_features INT
+  10/10, check_docs all match, 22 Markdown files without broken links, package smoke + GUI start PASS. Channel order
+  of the 3-5 ch WEMs = fmt channel mask order (0x37 = FL FR C SL SR, 0x33 quad): the centre holds the dry voice,
+  PA surrounds echo it 160-173 ms later (Harley/HQ 121 ms, Freeze rears 73 ms); Riddler/Ivy/Ra's 5 ch are
+  decorrelated per channel. Baked reverb is rare (blind RT60 < 0.2 s for 90 %). Listening samples:
+  `external/fx_samples/INDEX.md` (git-ignored; the WAVs were deleted with version 3.4.0, the user's notes kept). NEXT: user listens and names the groups; then the survey goes
+  into ackit/bnk.py + a tool, docs/formats/effects.md with check_docs claims, and the other 4 voice languages.
+- OPEN: 19 speaker labels to check by listening.
 
 ## Environment facts
 - Game: Steam AppID 200260 "Batman: Arkham City GOTY", `C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham
@@ -29,23 +256,34 @@
   `%USERPROFILE%\Desktop\ArkhamCityModding\Audio\49D84887_LOC_ITA\WAD\` (sha1 1238172C...). The game is clean now
   (fingerprints match the old `_backup/`, deleted on 2026-10-08).
 - Dev tools: `.venv` (Python 3.14.6, PySide6 6.10.3, soundfile 0.14.0 / libsndfile 1.2.2, soxr 1.1.0, numpy,
-  pytest, PyInstaller 6.22.3, soundcard 0.4.6). git at `C:\Program Files\Git\cmd`. WinDbg Store (path changes on
+  pytest, pyflakes, PyInstaller 6.22.3, soundcard 0.4.6). git at `C:\Program Files\Git\cmd`. WinDbg Store (path changes on
   update; `ackit/verify/debugger.py: find_cdb`). No WSL distro, no Docker.
 - User data (this PC): `%APPDATA%\UnheardCity\` (settings.json, stores\<id>\), cache `%LOCALAPPDATA%\UnheardCity\cache`.
 - The user's own material: `%USERPROFILE%\Desktop\ArkhamCityModding` (old mod work, Ghidra project, dump);
   their third-party downloads were moved from `tools/`, `repositories/` to `external/` (git-ignored).
 - Ghidra project BAC: `.text` Ghidra address = runtime + 0xC00 (docs use runtime addresses).
 
+- Linux dev machine (2026-10-27): podman container (Debian 13, user `claude`, HOME=/home/claude, sudo) on an
+  Arch host. The game is the host's Steam Flatpak install, mounted read-write at
+  `/home/gora/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Batman Arkham City GOTY`
+  (only that folder: no libraryfolders.vdf, no Proton prefix). Steam discovery looks under HOME, so pass
+  `--game DIR` (CLI) / `ACKIT_TEST_GAME=DIR` (tests) and `--lang` (no UserEngine.ini; `Game.language()` -> INT).
+  Python 3.14.8 built from source into /usr (`/usr/bin/python3.14`, the host's path too); `.venv` regenerated
+  with it (`.venv/bin/python -m pytest -q`; GUI tests need `QT_QPA_PLATFORM=offscreen`). No in-game checks here.
+
 ## Where to look
 - Findings, failed hypotheses, in-game experiment log E0-E14: `docs/reverse-engineering/findings.md`.
 - Formats: `docs/formats/`. Decoder addresses: `docs/reverse-engineering/decoder.md`.
-- Decisions (stack, deps, licenses, originals store, size rule): `docs/decisions.md`.
+- Decisions (stack, deps, licenses, originals store, size rule, catalog, verification): `docs/decisions.md`.
+- Voice lines across languages (one recording per language, from the game data): `docs/formats/banks.md`
+  (*Lines*), `ackit/lines.py`; checked by `ackit verify --catalog` (facts).
 
 ## Session log
 - 2026-10-08: user added `my_catalog_for_ita/` (per-character lists, wemscan names) -> moved to
   `data/catalog/source/ita/`; 3364 names labelled (2 wemscan name collisions label both candidates; one line had a
   `.wem.wav` typo). Built catalogs for ITA/INT/FRA/DEU/ESN/SFX (identical names and counts across languages).
-  Assumption: speakers are the same in every language, so ITA labels apply to all.
+  Assumption: speakers are the same in every language, so ITA labels apply to all. (Wrong for multi-variation
+  entries, whose WEM order differs per language: since 2026-10-10 labels go through the game's line names.)
 - 2026-10-08: libsndfile's Vorbis writer crashed (C stack overflow) on a 30 s x 6 ch single write -> block writes.
 - 2026-10-08: wad_fingerprint made structural (IDs + non-pack sizes) so music edits don't invalidate the catalog;
   scan mode never borrows checksums from the shipped CSV (other builds / synthetic tests).
@@ -54,8 +292,10 @@
   SoundFilter.refilter uses begin/endFilterChange when available.
 - 2026-10-08: search haystack must not contain the variation index (":2" matched "2"); IDs are matched separately
   (words of >= 6 chars against the target id).
-- Verification commands: `pytest -q`; `ackit verify`; `ackit verify --ingame`; `python tools/e2e_editor.py`
+- Verification commands: `pytest -q`; `ackit verify`; `ackit verify --catalog`; `python tools/e2e_features.py
-  (needs Steam logged in); packaged: `dist/UnheardCity/UnheardCity.exe --smoke-test --report f.txt`.
+  [--lang all]`;
+  `python tools/e2e_editor.py`; `ackit verify --ingame [--banks | --texts]` (needs Steam logged in); packaged:
+  `dist/UnheardCity/UnheardCity.exe --smoke-test --report f.txt`.
 - 2026-10-08: UI restyle on request: vector icon set (app/icons.py, no emoji), "Arkham" theme (app/theme.py: QSS +
   palette, translucent panels over a wallpaper backdrop).
 - 2026-10-08: user rejected the procedural icon/backdrop and supplied images (~/Downloads/Arkham City:
@@ -86,3 +326,22 @@
   soundbanks are missing (docs/formats/banks.md). Packages were unpacked with Gildor's decompress.exe from
   `%USERPROFILE%\Desktop\ArkhamCityModding\Audio\decompress\` (outside the repo). User: document now, implement
   editing later.
+- 2026-10-09 (mission 3): BANK = one row per media ID, written into every copy in place (assumption: the game
+  may load any copy; a partial write would be inconsistent). Package writes store changed blocks as LZO literal
+  runs (no compressor needed). Bank-file conflicts use the whole-file SHA-1 (the cheap guid+chunk-table
+  fingerprint missed same-size edits in tests). Name collisions in the WAD catalogs (2 per language) are shown as
+  NAME#i. `soundcard` comes from the `verify` extra (was missing from the venv).
+- 2026-10-09 (mission 3, stage 4): subtitle links by name were wrong for most multi-variation entries (WEMs are
+  stored in media-ID order, not _NN order): links come only from events. Numbered event names (Speech_X_04)
+  live in export tables, not in name tables: the bank scan now collects export names (+40 named BANK sounds,
+  links 71 % -> 92 %). Version*.int (build IDs) are not treated as UI text.
+- 2026-10-10 INCIDENT: a crashed e2e_features run (exception before its revert) left ITA:8108E77E:0 modified,
+  and a catalog rebuild then recorded the modified bytes. Recovered: reverted from the run's temporary store,
+  ITA WAD sha1 1238172C... again, every other full copy in that store equal to the game, ITA.csv/SUB_ITA.csv
+  restored from git and an identical rebuild. Prevention: build_catalog stops when a sound differs from the
+  catalog (`changed_since_catalog`), all test tools revert in `finally`, e2e_features writes only ORIGINAL sounds.
+- 2026-10-10: source lists of another language (`source/<lang>/`) are resolved through THAT language's WAD
+  positions; ~1,270 speech sounds per language have no game line name (efforts 519, radio/TV 139, cinematics
+  134) and keep position names (`Sound.is_position_name`): their labels stay per language and mod packages from
+  another language are not matched to them. (Superseded the same day by the line identities of `ackit/lines.py`;
+  `is_position_name` was removed.)

README.md

@@ -1,18 +1,25 @@
 # Arkham City Audio Toolkit
 
-Replace any voice line, music track or ambience in **Batman: Arkham City (PC, Steam)** with your own audio:
+Replace any voice line, music track, ambience or sound effect in **Batman: Arkham City (PC, Steam)** with your own
-by name, from a file or a recording, with one-click revert. No Wwise, no game patch, no external programs.
+audio (by name, from a file or a recording) and edit subtitles and menu text, with one-click revert. No Wwise, no
+game patch, no external programs.
+
+<img src="docs/img/main.png" alt="Unheard City, the editor" width="720">
 
 - **Unheard City** (`app/`, [user guide](docs/user-guide.md)): the desktop editor for Windows and Linux. Browse and
-  search the ~11,500 sounds of the game's audio archives, filter by category and character, listen, replace or
+  search all ~21,000 sounds of one language (its voice lines, music, ambience, and the effects stored inside
-  record, apply and revert. Most short effects (footsteps, foley, weapons, UI) live inside soundbanks and are
+  soundbanks: footsteps, foley, combat, weapons, UI), filter by category and character, listen; replace or
-  not listed yet ([docs/formats/banks.md](docs/formats/banks.md)).
+  record in the Edit tab (an edit list, the subtitle shown large, effects for radios, phones, speakers,
+  announcements...), apply and revert. Edit subtitles (linked to their speech lines) and the menu text in their
+  own sections. One Language setting (default: the game's) applies to sounds, subtitles and menu text.
 - **`ackit`** (`ackit/`): the Python library and command-line tool behind it.
 
 How it works in one paragraph: the game's Wwise 2011.1 decoder only knows the standard Vorbis codebooks, so ackit
 encodes with libvorbis and rewrites the stream into exactly the layout the game expects (codebook IDs, packet
-format, seek table, header hash and memory size reversed from the game), then writes it into the `.wad`
+format, seek table, header hash and memory size reversed from the game), then writes it in place into the
-archives atomically, keeping the original bytes in a store for exact reverts. Details in `docs/`.
+`.wad` archives, the soundbanks and the game's compressed `.upk` packages (ackit reads and patches them itself),
+keeping the original files in a store for exact reverts. Subtitles live in the packages too, menu text in the
+localization files. Details in `docs/`.
 
 ## Install
 - **Editor (users), Windows:** unzip `UnheardCity-<version>-windows-amd64.zip` and double-click **Install.cmd**.
@@ -23,23 +30,29 @@ archives atomically, keeping the original bytes in a store for exact reverts. De
 - **Toolkit (developers):** Python 3.10+:
   ```
   python -m venv .venv && .venv/Scripts/activate         # Linux: source .venv/bin/activate
-  pip install -e .[app,dev]                              # app = editor, dev = tests/packaging
+  pip install -e .[app,dev,verify]                       # app = editor, dev = tests/packaging, verify = in-game test
   python -m app.main                                     # run the editor from source
   ```
 
 ## Quick start (command line)
 ```
 ackit find joker tape                             # search by name, category or character
-ackit extract Speech_TAPE_JOKER_1_4 orig.wav     # listen to the original
+ackit extract Speech_TAPE_JOKER_1_04 orig.wav     # listen to the original (a line has one name in every language)
-ackit replace Speech_TAPE_JOKER_1_4 my_line.wav --match-loudness
+ackit replace Speech_TAPE_JOKER_1_04 my_line.wav --match-loudness
-ackit revert Speech_TAPE_JOKER_1_4               # or: ackit revert --all
+ackit fx-template fx.json                         # effects, all off: set "on" for band-pass EQ, echo, reverb...
+ackit replace Speech_TAPE_JOKER_1_04 my_line.wav --fx fx.json
+ackit revert Speech_TAPE_JOKER_1_04               # or: ackit revert --all (every sound and text)
+ackit text find --lang INT protocol               # subtitles and menu texts (keys SUB:... / MENU:...; any of the
+                                                  # 11 languages, e.g. --lang JPN)
+ackit text set "MENU:INT:FRONTEND.int:Buttonpress:DoubleTap" "Tap twice"
 ```
 More: `ackit status`, `ackit list --character Joker`, `ackit export-mod my.acmod` / `ackit import-mod`,
-`ackit verify` (offline self-check), `ackit game` (detected install, language, store). `ackit -h` lists all
+`ackit verify` (offline self-check), `ackit verify --catalog` (catalog vs game files), `ackit game` (detected
-commands. The game folder is found through Steam (Windows, Linux, Flatpak, Proton); otherwise pass `--game DIR`.
+install, language, store), `ackit restore-backup` (last resort if the originals store is damaged). `ackit -h` lists all commands. The game folder is found through Steam (Windows, Linux, Flatpak, Proton); otherwise pass `--game DIR`.
 
 Rules worth knowing: speech/effects replacements must fit in the original's size (the tool squeezes or refuses
-with the maximum length); music can be any length. Close the game before writing.
+with the maximum length); music can be any length. Subtitles must fit the original text's length (menu text:
+any length). Close the game before writing.
 
 ## Where things are
 ```
@@ -49,9 +62,14 @@ app/            Unheard City (PySide6)
 tests/          pytest suite (synthetic game data; real-game tests run when the game is installed)
 data/catalog/   sound catalogs (CSV), character lists (source/), schema in data/catalog/README.md
 docs/           formats, reverse engineering, decisions, categories, user guide, building
-tools/          catalog builder, packaging (build_app.py, installer/), artwork and license scripts
+tools/          catalog builder, docs figure check, real-game e2e checks (e2e_editor.py, e2e_features.py), packaging
+                (build_app.py, installer/), artwork, license and screenshot scripts
 NOTES.md        project status and working notes; CLAUDE.md: guidance for AI sessions
 ```
 
-Tests: `pytest` (the real-game tests are skipped when the game is not installed).
+Tests: `pytest` (the real-game tests are skipped when the game is not installed). Figures in the docs:
-In-game test (Windows, Steam running, WinDbg installed): `ackit verify --ingame`.
+`python tools/check_docs.py [--scan] [--wems]`.
+Real-game write round trips (no game launch): `python tools/e2e_features.py [--lang all]`,
+`python tools/e2e_editor.py`.
+In-game tests (Windows, Steam running, WinDbg installed): `ackit verify --ingame`, `--ingame --banks`,
+`--ingame --texts`.

REPORT.md

@@ -1,38 +1,127 @@
-# Report — repository cleanup and the editor, now Unheard City (mission 2)
+# Report — full-game audio, subtitles and menu text (mission 3), Edit tab and effects (mission 4)
+
+Briefs: `docs/history/mission3_audio_text_prompt.md`, `docs/history/mission4_dubbing_plan.md` (plan, the effects
+found in the game, the user's decisions). Earlier missions: the WAD/WEM toolkit (mission 1) and the editor Unheard
+City (mission 2).
 
 ## What works (verified)
-- **Repository**: clean layout (`ackit/`, `app/`, `tests/`, `data/catalog/`, `docs/`, `tools/`); old state in git
+- **Every sound of the game is cataloged and editable**: the speech WADs (5 languages x 10,816), the SFX WAD (651)
-  tag `pre-cleanup`. One CLI (`ackit`) and one app (`UnheardCity`).
+  and, new, the **15,355 in-memory sounds** stored inside soundbanks (archive `BANK`: footsteps, foley, combat,
-- **Toolkit**: portable encoder (libvorbis via libsndfile; no Wwise, ffmpeg or vgmstream), WAD/pack injector,
+  weapons, UI, efforts; 8,092 shared by all languages, ~1,450 localized speech lines per voice language, 6
-  originals store with byte-exact revert, conflict detection, mod packages, Steam/Linux/Proton game discovery.
+  shared by several) in 65 loose `.bnk` files and 974 `.upk` packages. ackit reads and patches the
-- **Catalog**: CSVs for ITA/INT/FRA/DEU/ESN/SFX (11,467 sounds per speech language set), categories, verified
+  LZO-compressed UE3 packages itself (`ackit/upk.py`, `lzo.py`; no external tools). A replacement goes into every
-  speakers from the user's lists (3,366 lines in every language), inferred characters for the rest, original
+  copy (93,702 copies). 445 WAD voice recordings (89 lines) also have bank copies (396 prefetch heads, 49
-  sizes and checksums. Schema in `data/catalog/README.md`.
+  whole copies; 2,185 copies), which are rewritten with them.
-- **Editor**: browse/search/filter, listen (A/B, loop, waveforms), replace from file or recording, trim/gain/
+  Names: 13,501 BANK sounds are named after the Wwise event that plays them (`Play_...`, `Speech_...`), the
-  normalize/loudness, apply/revert/revert all, changed-underneath warnings, notes/tags/favorites, mod
+  other 1,854 after their bank.
-  export/import, settings.
+- **Catalog check**: `ackit verify --catalog` compares the catalog with a fresh scan of the game (missing, extra,
-- **Tests**: 75 automated tests (synthetic game, real-game read-only tests, offscreen GUI tests) pass.
+  duplicate, size/checksum, category, character, language, bank copies, subtitle copies/texts) and re-derives
-  `ackit verify` (offline self-check on the installed game) PASS.
+  every relation from the game files (lines, names, subtitle links, prefetch heads, copies): **catalog OK** on
-- **Real game**: `ackit verify --ingame` PASS (ncc 1.0); `tools/e2e_editor.py` — replacement applied through the
+  the full game.
-  editor is heard in the game (ncc 1.0, no crash), revert leaves WAD and pack index byte-identical; a speech line
+- **Subtitles**: found in `RDialogueEvent` objects inside the packages (9,032 speech lines x 11 languages) and
-  applied and reverted byte-identically. Game files are original at the end.
+  in 41 scripted `RSubtitle` objects (radio, encryption cards, Riddler; linked through `RSubtitleLookup`),
-- **Package (Windows x86-64)**: `dist/UnheardCity-2.0.0-windows-amd64.zip` (78 MB) with
+  parsed, cataloged (`SUB_<LANG>.csv`) and linked to their voice recordings through the soundbanks: 103,304 of
-  `THIRD_PARTY_LICENSES.txt`; runs its smoke test and starts the GUI with `PATH` reduced to `C:\Windows`.
+  103,658 strings in 11 languages (INT 9,373 of 9,404), each on the same line in every language; an event that
-  Contains a per-user installer (`Install.cmd`): install, upgrade and uninstall (Settings > Apps) verified.
+  plays two versions of a line links both. Edited in place in every copy (168,688 copies).
+- **One recording per language, from the game data alone** (`docs/formats/banks.md`, *Lines*): every voice
+  recording is tied to its line by the game's Wwise Sound objects (same IDs in every language), and each
+  WAD recording to its media by its own language's BKPK table (10,816 / 10,816 in each language). Result:
+  12,434 lines (WAD + in-memory) with exactly one recording in each of the 5 voice languages, no gap and no
+  conflict; recordings stored twice (5 per language) are written together. Independent checks: every WAD
+  recording is a source in a bank of its own language (54,055 / 54,055), every prefetch head or whole bank copy
+  is the start of its recording (2,185 / 2,185). Each line has one name in every language and carries its identity (`line`):
+  speaker labels (3,366 in every language), subtitles and mod packages (another language: the same line)
+  follow it. Speaker labels vs the game's dialogue speaker field: 2,766 agree, 19 differ (categories.md).
+- **Menu text**: the localization files (`BmGame\Localization\<LANG>\*.int`, ~7,000 strings per language),
+  edited value by value; everything else in the files stays byte-identical.
+- **Editor**: one *Language* setting (default: the game's language) for voices, subtitles and menu text
+  (languages without voices use the INT ones, like the game); tabs *Sounds* / *Edit* / *Subtitles* / *Menu text*.
+  *Sounds* only browses (filters, list, a player with Play original / Play installed / Stop, Edit / Add to edit
+  list). *Edit* is where every kind of sound is replaced: an edit list (Previous / Next) whose sounds keep their
+  replacement audio, trim/level values and effects in the user data folder until applied; the subtitle shown
+  large with the original (and English) line; original / installed / replacement waveforms; record or load;
+  the effects. No scrolling at 1400 x 900 or 1280 x 800. The subtitle list links back to the sound. Search,
+  filters (character, category, file, modified), room counter, apply/revert/revert all, find & replace on the
+  filtered rows, mod export/import with texts, changed-outside banner for sounds and texts.
+- **Effects** (mission 4, `ackit/fx.py`, pedalboard + numpy): the effects found in the game's sounds (metadata
+  survey of every bank + signal analysis of every INT speech recording + the user listening to sample groups):
+  pitch / formant / pitch drift, octave layers, ring modulator and comb, glitch repeats, band-pass EQ,
+  distortion and phone codec, noise, echo, reverb (a generated room response) with reverse swells, and the
+  channel layout of 2-5 channel sounds (WAVE mask order; the PA lines' dry centre and delayed surrounds).
+  Presets per effect and whole presets (Radio, Phone, PA announcement, Mr. Freeze, ...); set by hand in the
+  Edit tab, or `ackit replace --fx` on the command line. Effects the game adds while it plays (the McDSP
+  FutzBox radio sound of many thug and street lines, room reverbs) are not in the files and must not be added
+  again.
+- **Safety**: every changed file (WAD, bank, package, localization file) is rebuilt from a verified original copy
+  in the store + the edits applied now; reverting the last edit restores it byte-for-byte; files changed by
+  another tool or Steam are detected (whole-file SHA-1) and not overwritten without confirmation (also when
+  installing a mod in the editor). The original copy is taken only from a file that matches the catalog
+  (another tool's or another originals store's change is a conflict; checked on all 1,446 files with sounds or
+  subtitles: no false alarm), and the editor does not switch stores while changes are applied. Writes are
+  serialized, so a sound and its subtitle applied at the same time both reach the package. **Revert all**
+  (editor and `ackit revert --all`) reverts sounds and texts; `ackit restore-backup` is the last resort for a
+  damaged store.
+- **Tests**: 177 automated tests (unit, synthetic game with packages/banks/dialogues/localization, a
+  two-language game for the lines (incl. a recording also stored whole in a bank), the shipped catalog's cross-references, real-game read-only (every
+  name of every language's catalog resolves to its own sound), offscreen GUI incl. list order, selection,
+  loading overlays and the Edit tab (edit list, saved replacements, effects panel, no scrolling), every effect
+  measured back on synthetic signals) pass. `ackit verify` PASS (incl.
+  300/300 in-memory WEMs rebuilt byte-identically).
+- **Real game** (2026-10-10), all reverting and checking byte-identity themselves:
+  `tools/e2e_features.py --lang all` 89/89 PASS in all 11 languages (every copy of an in-memory sound, each
+  language's localized line, prefetch heads, a line also stored whole in banks, a multi-copy subtitle, a scripted subtitle, a menu text, sound +
+  subtitle in one package, a mod moved to another voice language and matched to the same line at another
+  position); `ackit verify --ingame` PASS (music heard, ncc 1.0);
+  `--banks` PASS (77 in-memory sounds of the front-end packages replaced in 54 files: the game loads them, 71
+  probes found in its memory, no crash); `--texts` PASS (12/12 menu texts and 23/23 subtitles of Startup.upk
+  found in the game's memory, no crash); `tools/e2e_editor.py` PASS. 2026-10-27 (Linux, the game's Flatpak
+  install): `tools/e2e_features.py --lang INT` 10/10 PASS, incl. a 5-channel line replaced with effects (EQ,
+  echo, PA layout), decoded back from the game file (dry centre, surround echo 0.160 s later) and reverted
+  byte-identically. Game files original at the end (`ackit status`). The catalog builder refuses to run on a modified game, so the catalog always describes the
+  original files.
+- **Package**: `dist/UnheardCity-3.3.0-windows-amd64.zip` (86 MB, catalogs included; built 2026-10-13). With `PATH` reduced to
+  the Windows folders the packaged exe passes its smoke test (in the game's language - Italian on the test PC:
+  21,015 sounds loaded, encode + inject, 9,377 subtitles and 7,020 menu texts loaded, an in-memory sound decoded from a package) and the GUI starts and
+  quits cleanly. The installer is unchanged since mission 2 (install/upgrade/uninstall verified then); the
+  uninstaller now also counts text edits before offering to delete the user data (2026-10-13; that count was
+  tested, the full uninstall cycle was not run again). Version 3.4.0: the Linux package
+  (`dist/UnheardCity-3.4.0-linux-x86_64.tar.gz`, built 2026-10-27, the first Linux build) passes the same smoke
+  test (incl. the effects) with `PATH=/usr/bin:/bin`, and the GUI starts.
 
-## What is not verified / limitations
+## Unresolved / limitations
-- **Coverage**: only sounds in the WAD archives are listed and editable. About 15,800 short effects (footsteps,
+- **Checked by a person in game** (2026-10-10), not automatically (nothing of this plays or shows on the title
-  foley, weapons, UI, grunts) are stored inside soundbanks (loose `.bnk` and banks embedded in `.upk`
+  screen without input, findings 18): replaced voice lines and edited subtitles are heard and shown; replaced
-  packages) and are not covered yet (`docs/formats/banks.md`).
+  footsteps play on every step. That test found the one defect the automatic checks could not: in-memory
-- Speech lines being *heard* in the game needs gameplay; verified indirectly (same decoder/WAD path as the music
+  replacements followed by zero padding in their slot played as silence (fixed in 3.2.2, findings 24).
-  test, byte-level apply/revert, boot test with modified speech WAD).
+- **Size limits**: in-memory sounds and speech must fit their original size; subtitles must fit the original
-- Speech/effects replacements must fit the original's size (index lives in compressed `.upk` packages).
+  text's length ("room"); cp1252 subtitles accept cp1252 characters only. Lifting these limits means moving
-- Channel order for 4-6 channel sounds is unverified (tests used identical channels).
+  objects inside the packages (new export offsets/sizes, added chunks): designed but not done, because it could
-- Inputs: WAV/FLAC/OGG/OPUS/MP3/AIFF built in; M4A/AAC/WMA only with an ffmpeg installed on the system.
+  not be verified automatically in the game (decisions 4c).
-- Character labels outside the user's lists are inferred from names (marked as such), not verified speakers.
+- **Unlinked subtitles**: 24 speech names (24-36 strings per language, INT 31: Riddler trophy texts, Arkham City
+  Stories, radio reports and tapes) have no event in any bank; they are listed and editable but not linked.
+- **Subtitle coverage is uneven in the game itself** (INT voice lines with a subtitle): none for efforts/grunts,
+  few for combat barks (9 %), cinematics and boss fights (23 %; most cutscene lines are not played through speech
+  events and have no dialogue object), 89-97 % for story, predator, character and open-world lines. No other
+  subtitle storage was found in the packages (all classes with subtitle data checked).
+- 637 voice lines are played by no event whose name the game stores; they are named after their English
+  recording's position (the same name in every language). Mod packages from versions before 3.1 carry no line
+  identity: their voice lines are not moved to another language (listed instead of guessed).
+- Speaker labels: 2 names of the curated lists match two recordings each (wemscan name collision), both are
+  labelled; 19 labels differ from the game's dialogue speaker field (to be checked by listening).
+- **Unnamed sounds**: 1,854 BANK sounds are reached by no event whose name the game stores and are named
+  `<Bank>_<n>`.
+- 25 SFX originals (music seek tables and sample counts, looping ambience) are not reproduced byte-for-byte by
+  the WEM rules (`docs/formats/wem.md`); writing is not affected (replaced music plays in game).
+- Two copies of media `1C898731` differ; its checksum is left empty. Channel order of 3-6 channel sounds:
+  the WEMs carry WAVE channel masks and ackit keeps that order (mission 4). M4A/AAC/WMA input needs an ffmpeg
+  on the system.
+- **Effects are approximations** set by ear; the presets are starting points. The Edit tab does not mark the
+  lines the game itself adds an effect to (known from the bank survey, not in the catalog).
+- Another game build is scanned at first start (WADs: seconds; banks, lines and subtitles: a few minutes,
+  cached); it gets the same line identities, but no original checksums.
 
 ## Next steps
-1. Support the in-memory sounds in soundbanks: the loose footstep banks first, then the banks embedded in
+1. Optional: lift the subtitle length limit by relocating dialogue objects (checked in game by a person).
-   `.upk` packages (`docs/formats/banks.md`).
+2. Listen to the 19 lines whose speaker label differs from the game's dialogue speaker field
-2. Extend the curated speaker lists (`data/catalog/source/<lang>/*.txt`) and rebuild with `tools/build_catalog.py`.
+   (`docs/categories.md`) and fix the curated lists if needed; extend them (`data/catalog/source/<lang>/*.txt`)
-3. Optional: rewrite the BKPK indexes inside `.upk` packages (LZO) to lift the size limit for speech.
+   and rebuild the catalog.

ackit/__init__.py

@@ -1,10 +1,11 @@
 """ackit - Batman: Arkham City (PC) audio toolkit.
 
 Library layers (low -> high):
-  formats   wad, pack, wem, ogg, bits, codebooks, wwvorbis, allocsize, builder, convert, known_setups
+  formats   wad, pack, bnk (soundbanks), upk + lzo (UE3 packages), wem, ogg, bits, codebooks, wwvorbis,
-  audio     audio (load/convert/decode), encoder (PCM -> game WEM)
+            allocsize, builder, convert, known_setups
+  audio     audio (load/convert/decode), encoder (PCM -> game WEM), fx (effects for replacements)
   game      paths, game (install discovery), catalog + classify (names/categories/characters),
             inject (WAD writing), originals (store), workspace (apply/revert), modpack (share)
   tools     cli (`ackit` command), verify/ (offline self-check, in-game test on Windows)
 """
-__version__ = "2.0.0"
+__version__ = "3.4.0"

ackit/audio.py

@@ -12,9 +12,6 @@ import subprocess
 import numpy as np
 import soundfile as sf
 
-READABLE = (".wav", ".flac", ".ogg", ".oga", ".opus", ".mp3", ".aif", ".aiff", ".au", ".w64", ".caf")
-FFMPEG_ONLY = (".m4a", ".aac", ".wma", ".mp4", ".webm", ".mkv", ".ac3")
-
 
 class AudioError(Exception):
     pass
@@ -56,21 +53,29 @@ def resample(x, rate, new_rate):
 
 
 def to_channels(x, channels):
-    """Up/down-mix. Mono is copied to every channel; extra channels get the average."""
+    """Up/down-mix. Up: mono is copied to every channel, other sources keep their channels and the new ones get
+    the average. Down: nothing is dropped - every channel goes into the mono sum, or the channels beyond the
+    kept ones (e.g. the centre and surrounds of 5.1) are added to every kept one at -3 dB (the standard
+    fold-down); a result louder than the source's peak is scaled back to it, so it does not clip."""
     x = _as2d(x)
     cur = x.shape[1]
     if cur == channels:
         return x
-    if channels == 1:
-        return x.mean(axis=1, keepdims=True)
     if cur == 1:
         return np.repeat(x, channels, axis=1)
+    if channels > cur:
         out = np.empty((len(x), channels), dtype=np.float32)
-    keep = min(cur, channels)
+        out[:, :cur] = x
-    out[:, :keep] = x[:, :keep]
+        out[:, cur:] = x.mean(axis=1, keepdims=True)
-    if channels > keep:
-        out[:, keep:] = x.mean(axis=1, keepdims=True)
         return out
+    if channels == 1:
+        out = x.sum(axis=1, keepdims=True)
+    else:
+        out = x[:, :channels] + 0.7071 * x[:, channels:].sum(axis=1, keepdims=True)
+    peak_in, peak_out = float(np.max(np.abs(x), initial=0.0)), float(np.max(np.abs(out), initial=0.0))
+    if peak_out > peak_in > 0:
+        out = out * (peak_in / peak_out)
+    return out.astype(np.float32)
 
 
 def trim(x, rate, start=None, end=None):

ackit/banks.py

@@ -0,0 +1,465 @@
+"""In-memory sounds: WEMs held inside soundbanks, i.e. the loose SFX\\*.bnk files and the banks embedded
+in the .upk packages (docs/formats/banks.md, upk.md). Catalog archive `BANK`, target = media ID.
+
+A media ID can be stored many times (the same bank copied into several packages, or several banks
+holding the same media). The location index (data/catalog/BANK_locations.csv.gz) lists every copy:
+  media, file (relative to CookedPCConsole, '/'-separated), offset (in the *uncompressed* file), size, kind
+kind 'm' = an in-memory sound, 'p' = a bank copy of a streamed WAD recording (the in-memory head of a prefetched
+sound, or a whole copy with the same media ID); for 'p' rows `sound` is the WAD sound's key (e.g. ITA:2B14E472:3):
+speech lines have per-language media IDs and heads.
+Replacements are written in place into every copy, no larger than the smallest; a whole in-memory WEM fills its
+slot exactly (wem.fill: the game plays a zero-padded one as silence), a head gets the start of the new
+recording. So bank and package layouts never change.
+"""
+import csv
+import gzip
+import hashlib
+import io
+import os
+import re
+import struct
+from collections import defaultdict, namedtuple
+
+from .bnk import Bank, BankError, find_banks, fnv1, PREFETCH
+from .upk import Package, PackageError, is_package
+
+ARCHIVE = "BANK"
+LOCATIONS = "BANK_locations.csv.gz"
+SOUND, MEMORY, PREFETCH_HEAD = 2, "m", "p"
+EVENT, ACTION = 4, 3
+CONTAINERS = (5, 6, 7, 9)            # random/sequence, switch, actor-mixer, layer
+
+# lang: language of the bank holding this copy ('' = language-neutral bank); known after a scan only
+Location = namedtuple("Location", "media file offset size kind sound lang", defaults=("", ""))
+
+
+# --- files ----------------------------------------------------------------------------------------
+def bank_files(cooked):
+    """Relative paths ('/'-separated) of every loose regular bank and every package."""
+    out = []
+    sfx = os.path.join(cooked, "SFX")
+    if os.path.isdir(sfx):
+        for f in sorted(os.listdir(sfx)):
+            if f.lower().endswith(".bnk"):
+                with open(os.path.join(sfx, f), "rb") as fh:
+                    if fh.read(4) == b"BKHD":
+                        out.append("SFX/" + f)
+    for f in sorted(os.listdir(cooked)):
+        if f.lower().endswith(".upk") and is_package(os.path.join(cooked, f)):
+            out.append(f)
+    return out
+
+
+class _Plain:
+    """Uniform read access for a loose bank (offsets = file offsets)."""
+
+    def __init__(self, path):
+        self.path = path
+
+    def read(self, off, n):
+        with open(self.path, "rb") as f:
+            f.seek(off)
+            return f.read(n)
+
+    def read_all(self):
+        return 0, open(self.path, "rb").read()
+
+    def fingerprint(self):
+        return file_fingerprint(self.path)
+
+    def patched(self, edits):
+        out = bytearray(open(self.path, "rb").read())
+        for off, data in edits:
+            out[off:off + len(data)] = data
+        return bytes(out)
+
+
+def file_fingerprint(path):
+    """Identity of a loose bank (small files: content hash)."""
+    return hashlib.sha1(open(path, "rb").read()).hexdigest()[:16]
+
+
+_open_cache = {}
+
+
+def open_file(path):
+    """A reader for a bank file or package (cached while the file is unchanged)."""
+    st = os.stat(path)
+    key = (os.path.normcase(os.path.abspath(path)), st.st_size, st.st_mtime_ns)
+    r = _open_cache.get(key)
+    if r is None:
+        r = Package(path) if path.lower().endswith(".upk") else _Plain(path)
+        if len(_open_cache) > 64:
+            _open_cache.clear()
+        _open_cache[key] = r
+    return r
+
+
+def file_path(cooked, rel):
+    return os.path.join(cooked, *rel.split("/"))
+
+
+def riff_trim(b):
+    """A WEM slot without the zero padding after a smaller replacement (RIFF length)."""
+    if len(b) >= 8 and b[:4] == b"RIFF":
+        n = 8 + struct.unpack_from("<I", b, 4)[0]
+        if n <= len(b):
+            return b[:n]
+    return b
+
+
+# --- HIRC graph (for names) ---------------------------------------------------------------------
+def _hirc_objects(buf, bank):
+    """(type, id, payload memoryview) of every HIRC object of a bank."""
+    if "HIRC" not in bank.chunks:
+        return
+    o, n = bank.chunks["HIRC"]
+    p = bank.at + o
+    end = p + n
+    count = struct.unpack_from("<I", buf, p)[0]
+    p += 4
+    for _ in range(count):
+        if p + 9 > end:
+            return
+        t = buf[p]
+        ln = struct.unpack_from("<I", buf, p + 1)[0]
+        yield t, struct.unpack_from("<I", buf, p + 5)[0], bytes(buf[p + 5:p + 5 + ln])
+        p += 5 + ln
+
+
+def _children(payload, known):
+    """A container's child list: the first `u32 count, count x u32 id` whose ids are all known objects."""
+    n = len(payload)
+    for p in range(4, n - 7):
+        c = struct.unpack_from("<I", payload, p)[0]
+        if 0 < c <= 4096 and p + 4 + 4 * c <= n:
+            ids = struct.unpack_from("<%dI" % c, payload, p + 4)
+            if all(i in known for i in ids):
+                return ids
+    return ()
+
+
+# --- scan ---------------------------------------------------------------------------------------
+def _by_language():
+    return defaultdict(dict)
+
+
+class Scan:
+    """Everything found in the game's banks (built by `scan`)."""
+
+    def __init__(self):
+        self.locations = []                  # [Location]
+        self.files = {}                      # rel -> fingerprint
+        self.media = {}                      # id -> {"size", "sha1": set, "head": bytes, "banks": set}
+        self.bank_names = {}                 # bank id -> name
+        self.names = set()                   # identifier-like names seen in packages (for event names)
+        self.objects = {}                    # HIRC id -> type
+        self.containers = {}                 # id -> payload
+        self.events = {}                     # id -> [action ids]
+        self.actions = {}                    # id -> target id
+        self.sound_media = defaultdict(set)  # sound object id -> media ids
+        self.prefetch = set()                # media ids whose bank copies belong to a WAD recording (prefetch
+                                             # heads; whole recordings: catalog.whole_recordings)
+        self.lang_media = defaultdict(set)   # language -> media ids named by Sound objects of that language's banks
+        # BKPK tables: WAD entry id -> {language: {media id: (offset in entry, size)}}. A table belongs to the
+        # language its localized AkBank export declares ('' = outside such an export). The same media ID can be
+        # used by several languages (at a different position in each language's WAD), so tables never merge.
+        self.packs = defaultdict(_by_language)
+        self.subtitle_objects = {}           # counts of dialogue / RSubtitle objects (catalog.scan_text)
+        self.issues = []                     # (kind, detail)
+
+
+_LANG = re.compile(rb"([\x01-\x10])\x00\x00\x00\x04\x00\x00\x00[A-Z]{3}\x00")
+
+
+def localized_exports(data, start, imports, exports):
+    """[(begin, end, [languages])] of the localized AkBank exports of a package (offsets in data). Such an
+    export starts with its list of languages (FStrings, e.g. INT FRA ITA DEU ESN) and then holds one block per
+    language, in that order: a bank and/or the BKPK table of the speech WAD entry (docs/formats/banks.md)."""
+    out = []
+    akbank = {-(i + 1) for i, (c, o, _) in enumerate(imports) if c == "Class" and o == "AkBank"}
+    for x in exports:
+        if x.cls not in akbank or x.size <= 0:
+            continue
+        o0 = x.offset - start
+        m = _LANG.search(data, o0, min(o0 + 256, o0 + x.size))
+        if not m:
+            continue
+        n = m.group(1)[0]
+        langs, o = [], m.start() + 4
+        for _ in range(n):
+            ln = struct.unpack_from("<i", data, o)[0]
+            langs.append(bytes(data[o + 4:o + 3 + ln]).decode("latin-1"))
+            o += 4 + ln
+        out.append((o0, o0 + x.size, langs, x.name))
+    return out
+
+
+def _assign(positions, regions, issues, rel, what):
+    """{position: language} for the blocks (banks or BKPK tables) inside localized exports: the k-th block of
+    an export belongs to its k-th language. An export whose block count differs is reported, not guessed."""
+    out = {}
+    for o0, o1, langs, name in regions:
+        inside = [a for a in positions if o0 <= a < o1]
+        if not inside:
+            continue
+        if len(inside) != len(langs):
+            if issues is not None:
+                issues.append(("languages", "%s: %s has %d %s for %d languages" % (rel, name, len(inside), what, len(langs))))
+            continue
+        out.update(zip(inside, langs))
+    return out
+
+
+def bkpk_positions(data):
+    """Offsets of the BKPK tables (docs/formats/wad.md) embedded in a package."""
+    out = []
+    for m in re.finditer(b"BKPK", data):
+        p = m.start()
+        if p + 0x48 > len(data) or struct.unpack_from("<I", data, p + 0x1C)[0] != 0x48:
+            continue
+        n = struct.unpack_from("<I", data, p + 0x10)[0]
+        if n <= 100000 and p + 0x48 + 12 * n <= len(data):
+            out.append(p)
+    return out
+
+
+def read_bkpk(data, p):
+    """(WAD entry id, {media id: (offset in entry, size)}) of the BKPK table at p."""
+    entry, n = struct.unpack_from("<I", data, p + 8)[0], struct.unpack_from("<I", data, p + 0x10)[0]
+    table = {}
+    for i in range(n):
+        mid, size, off = struct.unpack_from("<III", data, p + 0x48 + 12 * i)
+        table[mid] = (off, size)
+    return entry, table
+
+
+def _bkpk_tables(data, packs, regions, issues=None, rel=""):
+    """Add a package's BKPK tables to `packs` under the language their export declares. Copies of a table
+    in several packages must agree (reported otherwise)."""
+    pos = bkpk_positions(data)
+    lang_at = _assign(pos, regions, issues, rel, "BKPK tables")
+    for p in pos:
+        entry, table = read_bkpk(data, p)
+        lang = lang_at.get(p, "")
+        have = packs[entry].get(lang)
+        if have and have != table and issues is not None:
+            issues.append(("bkpk", "%s: table of %08X (%s) differs from another copy" % (rel, entry, lang or "-")))
+        packs[entry][lang] = table
+
+
+def scan(cooked, progress=None, cooked_banks=None, files=None, visit=None):
+    """Scan every loose bank and package of the game (minutes), or only `files`. Returns a `Scan`.
+    `visit(rel, package, start, data)` is called for every package (other scans share the decompression)."""
+    sc = Scan()
+    files = bank_files(cooked) if files is None else files
+    if cooked_banks is None:
+        p = os.path.join(cooked, "CookedBanks.txt")
+        cooked_banks = [l.strip() for l in open(p, encoding="latin-1")] if os.path.exists(p) else []
+    sc.bank_names.update({fnv1(n): n for n in cooked_banks if n})
+    for k, rel in enumerate(files):
+        if progress and k % 50 == 0:
+            progress(k, len(files))
+        path = file_path(cooked, rel)
+        try:
+            r = open_file(path)
+            start, data = r.read_all()
+        except (PackageError, OSError) as e:
+            sc.issues.append(("unreadable", "%s: %s" % (rel, e)))
+            continue
+        lang_of = {}
+        if isinstance(r, Package):
+            names, imports, exports = r.tables(data, start)
+            sc.names.update(names)                      # base names ...
+            sc.names.update(x.name for x in exports)    # ... and numbered object names (Speech_X_04)
+            regions = localized_exports(data, start, imports, exports)
+            _bkpk_tables(data, sc.packs, regions, sc.issues, rel)
+            if visit:
+                visit(rel, r, start, data)
+            banks = list(find_banks(data))
+            lang_of = _assign(sorted(bk.at for bk in banks), regions, sc.issues, rel, "banks")
+        else:
+            try:
+                banks = [Bank(data, 0)]
+            except BankError:
+                continue
+        if banks:
+            sc.files[rel] = r.fingerprint()
+        for bk in banks:
+            sc.bank_names.update(bk.names())
+            media = {m.id: m for m in bk.media()}
+            prefetched = set()
+            for s in bk.sources():
+                if s.stream == PREFETCH and s.media in media:
+                    prefetched.add(s.media)
+            for m in media.values():
+                w = bytes(data[bk.at + m.off:bk.at + m.off + m.size])
+                kind = PREFETCH_HEAD if m.id in prefetched else MEMORY
+                if kind == PREFETCH_HEAD:
+                    sc.prefetch.add(m.id)
+                e = sc.media.setdefault(m.id, {"size": m.size, "sha1": set(), "head": w[:0x60], "banks": set(),
+                                               "kinds": set(), "langs": set()})
+                e["langs"].add(lang_of.get(bk.at, ""))
+                e["sha1"].add(hashlib.sha1(w).hexdigest()[:16])
+                e["banks"].add(bk.id)
+                e["kinds"].add(kind)
+                sc.locations.append(Location(m.id, rel, start + bk.at + m.off, m.size, kind, "", lang_of.get(bk.at, "")))
+            for t, oid, pl in _hirc_objects(data, bk):
+                sc.objects[oid] = t
+                if t == EVENT and len(pl) >= 8:
+                    n = struct.unpack_from("<I", pl, 4)[0]
+                    if len(pl) >= 8 + 4 * n:
+                        sc.events[oid] = list(struct.unpack_from("<%dI" % n, pl, 8))
+                elif t == ACTION and len(pl) >= 12:
+                    sc.actions[oid] = struct.unpack_from("<I", pl, 8)[0]
+                elif t in CONTAINERS:
+                    sc.containers[oid] = pl
+            for s in bk.sources():          # every source, prefetched ones too (streamed lines whose start
+                sc.sound_media[s.object].add(s.media)          # is kept in the bank: the game-over taunts)
+                if lang_of.get(bk.at):
+                    sc.lang_media[lang_of[bk.at]].add(s.media)
+    if progress:
+        progress(len(files), len(files))
+    return sc
+
+
+# --- names --------------------------------------------------------------------------------------
+def event_objects(sc):
+    """{event name: [sound object ids in traversal order]} for every event whose name is known. Events,
+    actions, containers and sound objects have the same IDs in every language (only the media differ), so
+    this order is the same in every language."""
+    known = set(sc.objects)
+    kids = {cid: _children(pl, known) for cid, pl in sc.containers.items()}
+    by_hash = {}
+    for n in sc.names:
+        if n and len(n) < 128:
+            by_hash.setdefault(fnv1(n), n)
+    out = {}
+    for eid, acts in sc.events.items():
+        name = by_hash.get(eid)
+        if not name:
+            continue
+        seen, order, stack = set(), [], [sc.actions.get(a) for a in acts][::-1]
+        while stack:
+            o = stack.pop()
+            if o is None or o in seen:
+                continue
+            seen.add(o)
+            if o in sc.sound_media:
+                order.append(o)
+            stack.extend(list(kids.get(o, ()))[::-1])
+        if order:
+            out[name] = order
+    return out
+
+
+def event_media(sc):
+    """{event name: [media ids in traversal order]} for every event whose name is known."""
+    out = {}
+    for name, objs in event_objects(sc).items():
+        order = []
+        for o in objs:
+            for m in sorted(sc.sound_media[o]):
+                if m not in order:
+                    order.append(m)
+        out[name] = order
+    return out
+
+
+def media_language(sc, m):
+    """Language of a localized in-memory media ('' = language-neutral)."""
+    langs = sc.media[m].get("langs", set()) - {""}
+    return next(iter(langs)) if len(langs) == 1 else ""
+
+
+def media_names(sc, reserved=()):
+    """{media id: (display name, primary bank name)}: named after the most specific 'Play' event that
+    reaches the media, otherwise after its bank (<Bank>_<n>). Localized speech has one media per language
+    for the same line: they share the name, variations are numbered within a language. Names are unique
+    among the language-neutral media plus any one language (what a catalog shows at a time) and avoid the
+    lower-case names in `reserved` (other archives' sounds)."""
+    ev = event_media(sc)
+    best = {}
+    for name, ms in ev.items():
+        groups = defaultdict(list)
+        for m in ms:
+            if m in sc.media and PREFETCH_HEAD not in sc.media[m]["kinds"]:     # heads belong to WAD lines
+                groups[media_language(sc, m)].append(m)
+        for g in groups.values():
+            rank = (0 if name.lower().startswith("play") else 1, len(g), name.lower())
+            for i, m in enumerate(g):
+                if m not in best or rank < best[m][0]:
+                    best[m] = (rank, name, i, len(g))
+    out = {}
+    taken = {("", n) for n in reserved}  # (language, lower-case name)
+
+    def free(lang, nm):
+        k = nm.lower()
+        if lang:
+            return (lang, k) not in taken and ("", k) not in taken
+        return not any(n == k for _, n in taken)
+
+    def take(lang, nm, m, base):
+        if not free(lang, nm):
+            nm = "%s_%08X" % (base, m)
+        taken.add((lang, nm.lower()))
+        return nm
+
+    for m, (_, name, i, n) in sorted(best.items(), key=lambda x: (x[1][1].lower(), x[1][2], x[0])):
+        out[m] = take(media_language(sc, m), name if n == 1 else "%s_%d" % (name, i + 1), m, name)
+    # the rest: by bank (and language), numbered in media id order
+    groups = defaultdict(list)
+    for m, e in sc.media.items():
+        bank = sorted(sc.bank_names.get(b, "%08X" % b) for b in e["banks"])[0]
+        if m in out:
+            out[m] = (out[m], bank)
+        else:
+            groups[(bank, media_language(sc, m))].append(m)
+    for (bank, lang), ms in sorted(groups.items()):
+        for i, m in enumerate(sorted(ms)):
+            out[m] = (take(lang, "%s_%d" % (bank, i + 1), m, bank), bank)
+    return out
+
+
+# --- location index -------------------------------------------------------------------------------
+def write_locations(path, locs):
+    raw = io.StringIO()
+    w = csv.writer(raw, lineterminator="\n")
+    w.writerow(["media", "file", "offset", "size", "kind", "sound"])
+    for l in sorted(locs, key=lambda l: (l.media, l.file, l.offset)):
+        w.writerow(["%08X" % l.media, l.file, "%X" % l.offset, l.size, l.kind, l.sound])
+    data = raw.getvalue().encode("utf-8")
+    with open(path + ".tmp", "wb") as f:
+        # mtime=0: identical content gives identical bytes (no spurious git changes)
+        with gzip.GzipFile(fileobj=f, mode="wb", mtime=0) as gz:
+            gz.write(data)
+    os.replace(path + ".tmp", path)
+
+
+def read_locations(path):
+    """{media id: [Location]}"""
+    out = defaultdict(list)
+    with gzip.open(path, "rt", encoding="utf-8", newline="") as f:
+        for r in csv.DictReader(f):
+            m = int(r["media"], 16)
+            out[m].append(Location(m, r["file"], int(r["offset"], 16), int(r["size"]), r["kind"], r["sound"]))
+    return dict(out)
+
+
+def game_fingerprint(cooked):
+    """Cheap identity of the game's bank set: CookedBanks.txt + names/sizes of the loose banks.
+    (Each file is checked again before it is written: `file_fingerprint` / Package.fingerprint.)"""
+    h = hashlib.sha1()
+    p = os.path.join(cooked, "CookedBanks.txt")
+    if os.path.exists(p):
+        h.update(open(p, "rb").read())
+    sfx = os.path.join(cooked, "SFX")
+    if os.path.isdir(sfx):
+        for f in sorted(os.listdir(sfx)):
+            if f.lower().endswith(".bnk"):
+                h.update(("%s:%d;" % (f.upper(), os.path.getsize(os.path.join(sfx, f)))).encode())
+    return h.hexdigest()[:16]
+
+
+def has_banks(cooked):
+    return os.path.exists(os.path.join(cooked, "CookedBanks.txt"))

ackit/bits.py

@@ -14,9 +14,6 @@ class BitReader:
             self.pos += 1
         return v
 
-    def bits_left(self) -> int:
-        return len(self.data) * 8 - self.pos
-
 
 class BitWriter:
     def __init__(self):
@@ -31,13 +28,6 @@ class BitWriter:
                 self.buf[self.pos >> 3] |= 1 << (self.pos & 7)
             self.pos += 1
 
-    def put_bits_from(self, r: BitReader, n: int):
-        """Copy n bits from a reader (fast path in 32-bit chunks)."""
-        while n > 0:
-            k = 32 if n >= 32 else n
-            self.put(k, r.get(k))
-            n -= k
-
     def bytes(self) -> bytes:
         return bytes(self.buf)
 

ackit/bnk.py

@@ -0,0 +1,108 @@
+"""Wwise soundbanks (bank version 56): chunks, in-memory media (DIDX/DATA) and HIRC sound sources.
+
+Works on a buffer that may hold a bank anywhere (a loose .bnk file, a BKPK pack's embedded bank, or
+an uncompressed .upk package with many banks). Offsets are relative to the start of the bank.
+Format: docs/formats/banks.md.
+"""
+import re
+import struct
+from collections import namedtuple
+
+CHUNKS = (b"BKHD", b"DIDX", b"DATA", b"HIRC", b"STID", b"STMG", b"ENVS", b"FXPR")
+SOUND = 2                         # HIRC object type of a Sound
+IN_MEMORY, STREAMED, PREFETCH = 0, 1, 2
+
+Media = namedtuple("Media", "id off size")                     # off: from bank start
+Source = namedtuple("Source", "object stream media bank off size pos")  # pos: of `off` field, from bank start
+
+
+class BankError(ValueError):
+    pass
+
+
+class Bank:
+    def __init__(self, buf, at=0):
+        self.buf, self.at = buf, at
+        self.chunks = {}
+        p = at
+        while p + 8 <= len(buf) and bytes(buf[p:p + 4]) in CHUNKS:
+            tag = bytes(buf[p:p + 4]).decode()
+            n = struct.unpack_from("<I", buf, p + 4)[0]
+            if p + 8 + n > len(buf) or tag in self.chunks:
+                break
+            self.chunks[tag] = (p + 8 - at, n)
+            p += 8 + n
+        if "BKHD" not in self.chunks or self.chunks["BKHD"][1] < 8:
+            raise BankError("no bank at 0x%X" % at)
+        self.size = p - at
+        o = at + self.chunks["BKHD"][0]
+        self.version, self.id = struct.unpack_from("<II", buf, o)
+
+    def _abs(self, tag):
+        o, n = self.chunks[tag]
+        return self.at + o, n
+
+    def media(self):
+        """In-memory WEMs: [Media(id, offset from bank start, size)]."""
+        if "DIDX" not in self.chunks or "DATA" not in self.chunks:
+            return []
+        o, n = self._abs("DIDX")
+        data = self.chunks["DATA"][0]
+        out = []
+        for i in range(n // 12):
+            mid, off, size = struct.unpack_from("<III", self.buf, o + 12 * i)
+            out.append(Media(mid, data + off, size))
+        return out
+
+    def sources(self):
+        """Sources of the HIRC Sound objects: [Source]. In-memory sources carry offset/size."""
+        if "HIRC" not in self.chunks:
+            return []
+        o, n = self._abs("HIRC")
+        end = o + n
+        count = struct.unpack_from("<I", self.buf, o)[0]
+        p = o + 4
+        out = []
+        for _ in range(count):
+            if p + 5 > end:
+                raise BankError("HIRC overruns its chunk in bank %08X" % self.id)
+            t = self.buf[p]
+            ln = struct.unpack_from("<I", self.buf, p + 1)[0]
+            if t == SOUND and ln >= 28:
+                obj, _plugin, stream, mid, bank = struct.unpack_from("<5I", self.buf, p + 5)
+                off, size = struct.unpack_from("<II", self.buf, p + 25)
+                out.append(Source(obj, stream, mid, bank, off, size, p + 25 - self.at))
+            p += 5 + ln
+        return out
+
+    def names(self):
+        """STID: {bank id: name}."""
+        if "STID" not in self.chunks:
+            return {}
+        o, n = self._abs("STID")
+        count = struct.unpack_from("<I", self.buf, o + 4)[0]
+        p = o + 8
+        out = {}
+        for _ in range(count):
+            bid, ln = struct.unpack_from("<IB", self.buf, p)
+            out[bid] = bytes(self.buf[p + 5:p + 5 + ln]).decode("latin-1")
+            p += 5 + ln
+        return out
+
+
+def find_banks(buf, start=0):
+    """Every bank in `buf` (e.g. an uncompressed package): yields Bank."""
+    for m in re.finditer(b"BKHD", buf[start:] if start else buf):
+        try:
+            yield Bank(buf, m.start() + start)
+        except (BankError, struct.error):
+            continue
+
+
+def fnv1(name):
+    """Wwise ID of a name (bank, event, ...): FNV-1 32 of the lower-cased name."""
+    h = 2166136261
+    for c in name.lower().encode("utf-8"):
+        h = (h * 16777619) & 0xFFFFFFFF
+        h ^= c
+    return h

ackit/catalog.py

@@ -1,14 +1,16 @@
-"""Index of every sound in the game's WAD archives, with names, categories and characters
+"""Index of every sound of the game, with names, categories and characters: the WAD archives (one per voice
-(sounds stored inside soundbanks are not indexed yet: docs/formats/banks.md).
+language, SFX) and the in-memory sounds stored inside soundbanks (archive BANK, docs/formats/banks.md).
 
 Sources, in order:
-  1. data/catalog/<ARCHIVE>.csv  -- shipped catalog (instant) when the WAD fingerprint matches;
+  1. data/catalog/<ARCHIVE>.csv  -- shipped catalog (instant) when the game's fingerprint matches;
-  2. otherwise a header scan of the WAD (a few seconds, cached in the user cache), classified with
+  2. otherwise a scan of the game (WAD headers: seconds; banks: minutes; cached in the user cache), classified
-     `classify` and labelled with data/catalog/characters.csv (speaker per sound name).
+     with `classify` and labelled with data/catalog/characters.csv (speaker per sound name).
-Names follow the community hash list (ackit/data/hashlist.txt): an entry with one WEM is NAME
+Names come from the game: a WAD entry ID is the Wwise ID (FNV-1) of a soundbank listed in the game's
+CookedPCConsole/CookedBanks.txt (`bank_names`); an entry with one WEM is NAME
 (alias NAME_1), entries with n variations are NAME_1..NAME_n (alias NAME#i); stream-pack media are
 named by their ID or <Pack>_<ID>.
 """
+import collections
 import csv
 import difflib
 import json
@@ -17,16 +19,19 @@ import re
 import struct
 from dataclasses import dataclass, asdict
 
-from . import classify
+from . import banks, classify, subtitles
+from . import lines as lines_mod
 from .game import wad_fingerprint
+from .bnk import fnv1
+from .lines import bank_langs
 from .inject import pack_bnk_path
 from .pack import Pack
-from .paths import catalog_dir, package_data, user_dir
+from .paths import catalog_dir, user_dir
 from .wad import Wad
 
-FIELDS = ["archive", "target", "name", "aliases", "group", "category", "character",
+FIELDS = ["archive", "target", "name", "aliases", "group", "category", "bank", "language", "line", "character",
           "characters_inferred", "seconds", "rate", "channels", "size", "sha1", "notes"]
-SCAN_VERSION = 1
+SCAN_VERSION = 3
 
 
 @dataclass
@@ -37,6 +42,9 @@ class Sound:
     aliases: tuple = ()
     group: str = "Unknown"
     category: str = "Unknown"
+    bank: str = ""                  # BANK sounds: the soundbank holding it (name); WAD sounds: ''
+    language: str = ""              # BANK sounds: language(s) of a localized speech line, ';'-separated ('' = every language)
+    line: str = ""                  # speech: the game's identity of the line in every language (ackit/lines.py)
     character: str = ""             # verified speaker(s), '; '-separated (curated catalog)
     characters_inferred: str = ""   # characters/roles mentioned by the name (hint)
     seconds: float = 0.0
@@ -62,41 +70,67 @@ class Sound:
     def names(self):
         return (self.name,) + tuple(self.aliases)
 
+    @property
+    def languages(self):
+        """Voice languages of a localized in-memory sound (() = every language)."""
+        return tuple(self.language.split(";")) if self.language else ()
+
+    def in_language(self, lang):
+        return not self.language or lang in self.languages
+
 
 # --- static data ---------------------------------------------------------------------------
-_names = None
+_names = {}                     # cooked folder -> {Wwise ID: bank name}
 
 
-def hash_names():
+def bank_names(cooked):
-    global _names
+    """{Wwise ID: name} of the game's soundbanks, from its own CookedBanks.txt (ID = FNV-1 of the name). They
-    if _names is None:
+    name the WAD entries (each entry holds the streamed media of one bank) and the stream packs."""
-        _names = {}
+    key = os.path.normcase(os.path.abspath(cooked))
-        for ln in open(package_data("hashlist.txt"), encoding="latin-1"):
+    if key not in _names:
-            p = ln.rstrip("\r\n").split("\t")
+        out = {}
-            if len(p) >= 2 and p[1]:
+        p = os.path.join(cooked, "CookedBanks.txt")
-                try:
+        if os.path.exists(p):
-                    _names[int(p[0], 16)] = p[1]
+            for ln in open(p, encoding="latin-1"):
-                except ValueError:
+                n = ln.strip()
-                    pass
+                if n:
-    return _names
+                    out.setdefault(fnv1(n), n)
+        _names[key] = out
+    return _names[key]
 
 
-def load_characters(path=None):
+def cooked_of(wad_path):
-    """{sound name (lower): 'Speaker' or 'A; B'} from data/catalog/characters.csv."""
+    """CookedPCConsole folder of a WAD (WADs are in CookedPCConsole/<language or SFX>/)."""
+    return os.path.dirname(os.path.dirname(os.path.abspath(wad_path)))
+
+
+def load_characters(path=None, lang=None):
+    """{sound name (lower): 'Speaker' or 'A; B'} from data/catalog/characters.csv: the rows valid in every
+    language (keyed by line names, the same line everywhere) plus the rows of `lang` (a sound of that
+    language only; none at present)."""
     path = path or os.path.join(catalog_dir(), "characters.csv")
     out = {}
     if os.path.exists(path):
         for row in csv.DictReader(open(path, encoding="utf-8")):
+            if not row.get("language") or row["language"] == lang:
                 out[row["name"].lower()] = row["character"]
     return out
 
 
+def character_of(s, characters):
+    """Verified speaker of a sound from a load_characters() map, by its name only: names are unique in a
+    language and a line has the same name in every language, while aliases (positions) are local."""
+    return characters.get(s.name.lower(), "")
+
+
 def read_csv(path):
     out = []
     for r in csv.DictReader(open(path, encoding="utf-8", newline="")):
         out.append(Sound(archive=r["archive"], target=r["target"], name=r["name"],
                          aliases=tuple(a for a in r["aliases"].split("|") if a), group=r["group"],
-                         category=r["category"], character=r["character"],
+                         category=r["category"], bank=r.get("bank") or "", language=r.get("language") or "",
+                         line=r.get("line") or "",
+                         character=r["character"],
                          characters_inferred=r["characters_inferred"], seconds=float(r["seconds"] or 0),
                          rate=int(r["rate"] or 0), channels=int(r["channels"] or 0), size=int(r["size"] or 0),
                          sha1=r["sha1"], notes=r["notes"]))
@@ -155,8 +189,8 @@ def _walk_entry(f, offset, size):
 
 def scan_archive(archive, wad_path, characters=None, progress=None):
     """Header scan of one WAD -> [Sound]."""
-    names = hash_names()
+    names = bank_names(cooked_of(wad_path))
-    characters = characters if characters is not None else load_characters()
+    characters = characters if characters is not None else load_characters(lang=archive)
     wad = Wad(wad_path)
     out = []
     with open(wad_path, "rb") as f:
@@ -185,12 +219,255 @@ def scan_archive(archive, wad_path, characters=None, progress=None):
                 s.seconds, s.rate, s.channels, s.size = secs, rate, ch, ln
                 s.group, s.category = classify.classify(s.name)
                 out.append(s)
+    # a variation named NAME_i can collide with an entry whose own name is NAME_i: the variation is
+    # shown as NAME#i and keeps NAME_i as alias (both stay findable, display names stay unique)
+    taken = collections.Counter(s.name.lower() for s in out)
+    for s in out:
+        if taken[s.name.lower()] > 1 and ":" in s.target:
+            s.name, s.aliases = s.aliases[0], (s.name,)
     for s in out:
         s.characters_inferred = "; ".join(classify.infer_characters(s.name))
-        s.character = next((characters[a.lower()] for a in s.names() if a.lower() in characters), "")
+        s.character = character_of(s, characters)
     return out
 
 
+_CAMEL = re.compile(r"(?<=[a-z])(?=[A-Z])")
+
+
+def bank_sounds(sc, characters=None, reserved=()):
+    """[Sound] of the in-memory sounds found by `banks.scan` (prefetch heads belong to WAD sounds). `reserved`:
+    lower-case names taken by other archives (sfx_names)."""
+    characters = characters if characters is not None else load_characters()
+    names = banks.media_names(sc, reserved)
+    every = set().union(*(e.get("langs", set()) for e in sc.media.values())) - {""}
+    out = []
+    for m, e in sc.media.items():
+        if banks.PREFETCH_HEAD in e["kinds"]:
+            continue
+        name, bank = names[m]
+        rate, ch, secs = _hdr(e["head"])
+        ls = bank_langs(sc, m)                  # localized: the languages whose banks hold it
+        s = Sound(banks.ARCHIVE, "%08X" % m, name, ("%s/%08X" % (bank, m),), bank=bank,
+                  language="" if set(ls) == every else ";".join(ls), seconds=secs, rate=rate, channels=ch,
+                  size=min(l.size for l in sc.locations if l.media == m) if len(e["sha1"]) > 1 else e["size"],
+                  sha1=next(iter(e["sha1"])) if len(e["sha1"]) == 1 else "")
+        s.group, s.category = classify.classify(name, bank=bank)
+        s.characters_inferred = "; ".join(classify.infer_characters(_CAMEL.sub("_", name + "_" + bank)))
+        s.character = character_of(s, characters)
+        out.append(s)
+    out.sort(key=lambda s: (s.bank.lower(), s.name.lower()))
+    return out
+
+
+def build_lines(game, sc):
+    """The game's speech lines across languages (ackit/lines.py)."""
+    return lines_mod.build(game, sc, _walk_entry)
+
+
+def scan_banks(game, progress=None, characters=None, visit=None):
+    """Full scan of the game's banks -> ([Sound], [Location] with WAD keys on prefetch heads, Scan, Lines).
+    Localized in-memory speech lines carry their line identity and the name shared by every language."""
+    sc = banks.scan(game.cooked, progress, visit=visit)
+    snd, locs, lines = derive_banks(game, sc, characters)
+    return snd, locs, sc, lines
+
+
+def derive_banks(game, sc, characters=None):
+    """(bank sounds, locations, Lines) from a Scan: the lines, the prefetch heads' lines, the in-memory sounds
+    with their line identities and names."""
+    lines = build_lines(game, sc)
+    mark_heads(sc, whole_recordings(sc, lines))
+    locs = [l._replace(sound=lines.head_key(l)) if l.kind == banks.PREFETCH_HEAD else l for l in sc.locations]
+    snd = bank_sounds(sc, characters, sfx_names(game))
+    lines.names = line_names(game, sc, lines, {s.key: s.name for s in snd})
+    apply_lines(snd, lines, lines.names)
+    return snd, locs, lines
+
+
+def whole_recordings(sc, lines):
+    """Media stored in memory that are WAD recordings of their bank's language (the same media ID, identical
+    bytes: 49 in the game, e.g. game-over taunts). Like prefetch heads, these copies belong to the WAD line
+    and are written with it; they are not sounds of their own."""
+    keys = collections.defaultdict(list)
+    for l in sc.locations:
+        if l.kind == banks.MEMORY:
+            k = lines.head_key(l)
+            keys[l.media].append(bool(k) and not k.startswith(banks.ARCHIVE + ":"))
+    return {m for m, wad in keys.items() if all(wad)}
+
+
+def mark_heads(sc, media):
+    """Treat the bank copies of `media` as heads of WAD recordings (see whole_recordings)."""
+    if not media:
+        return
+    for m in media:
+        if m in sc.media:
+            sc.media[m]["kinds"] = {banks.PREFETCH_HEAD}
+            sc.prefetch.add(m)
+    sc.locations = [l._replace(kind=banks.PREFETCH_HEAD) if l.media in media else l for l in sc.locations]
+
+
+def sfx_names(game):
+    """Lower-case names and aliases of the SFX WAD's sounds: in-memory sounds of the same bank must not take
+    them (`<Bank>_<n>` is the position name of the bank's streamed sounds too)."""
+    if "SFX" not in game.archives:
+        return set()
+    return {n.lower() for s in scan_archive("SFX", game.wad("SFX"), characters={}) for n in s.names()}
+
+
+def line_names(game, sc, lines, bank_names=None):
+    """{line id: name} (lines.line_names). Lines no event names keep their English recording's name: the
+    position-based name in the INT WAD (`NAME_i`, or `NAME#i` where that is taken) or the bank-numbered name."""
+    pos = {}
+    for L in {k.split(":")[0] for per in lines.members.values() for k in per.values()} - {banks.ARCHIVE}:
+        for s in scan_archive(L, game.wad(L), characters={}):
+            pos[s.key] = s
+    bank_names = bank_names if bank_names is not None else {}
+
+    def fallback(k):
+        return pos[k].name if k in pos else bank_names.get(k, k)
+
+    out = lines_mod.line_names(sc, lines, fallback)
+    taken = collections.Counter(n.lower() for n in out.values())
+    for lid, n in out.items():           # a fallback name equal to an event-based one -> NAME#i of the INT WEM
+        if taken[n.lower()] > 1:
+            k = lines.members[lid].get("INT") or next(iter(lines.members[lid].values()))
+            alt = [a for a in (pos[k].names() if k in pos else ()) if "#" in a]
+            if alt and alt[0] != n:
+                out[lid] = alt[0]
+    return out
+
+
+def apply_lines(sounds, lines, names):
+    """Give every speech sound of one archive (a WAD language or BANK) its line identity and the line's name,
+    the same in every language. A WAD sound keeps its position-based names as aliases (NAME#i: position in
+    this language's WAD; single-WEM entries: the entry name). Names stay unique in the archive (and, for
+    BANK, per language together with the language-neutral sounds): a clash is resolved by the sound that
+    is not the line's own name falling back to its position name / media-ID name."""
+    for s in sounds:
+        lid = lines.line_of.get(s.key)
+        if not lid:
+            continue
+        s.line = lid
+        if s.key in lines.copy_of:     # a second place of a recording: keeps its position name, written with it
+            continue
+        name = names[lid]
+        if s.archive != banks.ARCHIVE:
+            if ":" in s.target:
+                s.aliases = tuple(a for a in s.names() if "#" in a)
+            else:
+                s.aliases = (s.name,) + tuple(a for a in s.aliases if a != s.name)
+            if name in s.aliases:
+                s.aliases = tuple(a for a in s.aliases if a != name)
+        s.name = name
+    neutral, by_lang = set(), collections.defaultdict(set)     # lower-case names taken
+
+    def clash(s, n):
+        if s.archive != banks.ARCHIVE or not s.languages:
+            return n in neutral or any(n in v for v in by_lang.values())
+        return n in neutral or any(n in by_lang[L] for L in s.languages)
+
+    # a line's own recordings first: they keep the line's name in a clash (copies and others give way)
+    for s in sorted(sounds, key=lambda s: (not s.line or s.key in lines.copy_of, s.key)):
+        if clash(s, s.name.lower()):
+            if s.archive == banks.ARCHIVE:
+                s.name = "%s_%s" % (s.name, s.target)
+            else:
+                alt = [a for a in s.aliases if "#" in a and a != s.name]
+                s.aliases, s.name = (s.name,), alt[0] if alt else "%s@%s" % (s.name, s.target)
+        if s.archive == banks.ARCHIVE and s.languages:
+            for L in s.languages:
+                by_lang[L].add(s.name.lower())
+        else:
+            neutral.add(s.name.lower())
+    for s in sounds:
+        if s.archive != banks.ARCHIVE:
+            s.group, s.category = classify.classify(s.name)
+            s.characters_inferred = "; ".join(classify.infer_characters(s.name))
+    return sounds
+
+
+def subtitle_links(sc, lines, speech_names):
+    """{speech event name: {language: sound keys}}: event -> actions -> containers -> Sound objects (the same
+    in every language) -> that language's recording of each line (a WAD line or a localized in-memory one).
+    An event that plays several lines (two versions) links its subtitle to all of them: 'k1;k2', in order."""
+    ev = banks.event_objects(sc)
+    out = {}
+    for n in speech_names:
+        keys = collections.defaultdict(list)
+        for o in ev.get(n, ()):
+            for L, k in lines.obj_keys.get(o, {}).items():
+                if k not in keys[L]:
+                    keys[L].append(k)
+        if keys:
+            out[n] = {L: ";".join(ks) for L, ks in keys.items()}
+    return out
+
+
+def scan_text(game, progress=None, characters=None):
+    """One pass over every package: (bank sounds, bank locations, Scan, {lang: [SubLine]}, [SubLoc], issues,
+    Lines)."""
+    col = subtitles.Collector()
+    sc = banks.scan(game.cooked, progress, visit=col)
+    names = {n for n in sc.names if n.lower().startswith("speech_")}
+    names |= {n for n, _ in banks.media_names(sc).values() if n.lower().startswith("speech_")}
+    lookup = [n for n in col.dialogues if n.startswith("#lookup:")]
+    sc.subtitle_objects = {"dialogue_objects": sum(len(v) for n, v in col.dialogues.items() if n not in lookup),
+                           "dialogue_lines": len(col.dialogues) - len(lookup),
+                           "scripted_objects": sum(len(col.dialogues[n]) for n in lookup)}
+    col.resolve(names)
+    # the subtitle objects name their speech events too: an event ID is the FNV-1 hash of its name, so these
+    # names find events no package name table lists (e.g. Vicki Vale's radio reports)
+    sc.names.update(col.dialogues)
+    snd, locs, lines = derive_banks(game, sc, characters)
+    links = subtitle_links(sc, lines, list(col.dialogues))
+    texts, sublocs = subtitles.rows(col, links)
+    return snd, locs, sc, texts, sublocs, col.issues, lines
+
+
+def load_subtitles(game, lang, progress=None):
+    """([SubLine], {SUB key: [SubLoc]}) of one language: shipped catalog when the game matches it, else a
+    cached or fresh scan of the packages (several minutes)."""
+    fp = banks.game_fingerprint(game.cooked)
+    shipped = _shipped_index().get("SUB", {})
+    t = os.path.join(catalog_dir(), "SUB_%s.csv" % lang)
+    loc = os.path.join(catalog_dir(), subtitles.LOCATIONS)
+    if shipped.get("fingerprint") == fp and os.path.exists(t) and os.path.exists(loc):
+        return subtitles.read_texts(t, lang), subtitles.read_locations(loc, lang)
+    cache = os.path.join(user_dir("cache"), "scan_SUB_%s_v%d" % (fp, SCAN_VERSION))
+    if not os.path.exists(cache + ".locations.csv.gz"):
+        _, _, _, texts, sublocs, _, _ = scan_text(game, progress)
+        for lg, rows in texts.items():
+            subtitles.write_texts(cache + "_%s.csv" % lg, rows)
+        subtitles.write_locations(cache + ".locations.csv.gz", sublocs)
+    t = cache + "_%s.csv" % lang
+    return (subtitles.read_texts(t, lang) if os.path.exists(t) else []), \
+        subtitles.read_locations(cache + ".locations.csv.gz", lang)
+
+
+def load_banks(game, progress=None):
+    """([Sound], {media: [Location]}) of the in-memory sounds: shipped catalog when the game's bank set
+    matches it, else a cached or fresh scan (several minutes)."""
+    fp = banks.game_fingerprint(game.cooked)
+    shipped = _shipped_index().get(banks.ARCHIVE, {})
+    csv_path = os.path.join(catalog_dir(), "%s.csv" % banks.ARCHIVE)
+    loc_path = os.path.join(catalog_dir(), banks.LOCATIONS)
+    if shipped.get("fingerprint") == fp and os.path.exists(csv_path) and os.path.exists(loc_path):
+        return read_csv(csv_path), banks.read_locations(loc_path)
+    cache = os.path.join(user_dir("cache"), "scan_%s_%s_v%d" % (banks.ARCHIVE, fp, SCAN_VERSION))
+    if not (os.path.exists(cache + ".csv") and os.path.exists(cache + ".locations.csv.gz")):
+        sounds, locs, _, lines = scan_banks(game, progress)
+        for s in sounds:
+            s.sha1 = ""              # unknown build: no trusted original checksums
+        # the speech WADs of this build with the lines found now (load_archive reads these caches)
+        for a in game.languages:
+            snd = apply_lines(scan_archive(a, game.wad(a), progress=progress), lines, lines.names)
+            write_csv(_wad_cache(game, a), snd)
+        banks.write_locations(cache + ".locations.csv.gz", locs)
+        write_csv(cache + ".csv", sounds)
+    return read_csv(cache + ".csv"), banks.read_locations(cache + ".locations.csv.gz")
+
+
 # --- catalog -------------------------------------------------------------------------------
 def _shipped_index():
     p = os.path.join(catalog_dir(), "index.json")
@@ -200,6 +477,18 @@ def _shipped_index():
         return {}
 
 
+def is_shipped(game, archive):
+    """True if the game's archive matches the shipped catalog (not a scan of another build)."""
+    shipped = _shipped_index().get(archive, {})
+    if archive == banks.ARCHIVE:
+        return shipped.get("fingerprint") == banks.game_fingerprint(game.cooked)
+    return shipped.get("fingerprint") == wad_fingerprint(game.wad(archive))
+
+
+def _wad_cache(game, archive):
+    return os.path.join(user_dir("cache"), "scan_%s_%s_v%d.csv" % (archive, wad_fingerprint(game.wad(archive)), SCAN_VERSION))
+
+
 def load_archive(game, archive, progress=None):
     """[Sound] for one archive of a game (shipped CSV, user-cache scan, or fresh scan)."""
     path = game.wad(archive)
@@ -208,7 +497,7 @@ def load_archive(game, archive, progress=None):
     csv_path = os.path.join(catalog_dir(), "%s.csv" % archive)
     if shipped.get("fingerprint") == fp and os.path.exists(csv_path):
         return read_csv(csv_path)
-    cache = os.path.join(user_dir("cache"), "scan_%s_%s_v%d.csv" % (archive, fp, SCAN_VERSION))
+    cache = _wad_cache(game, archive)
     if os.path.exists(cache):
         return read_csv(cache)
     sounds = scan_archive(archive, path, progress=progress)
@@ -217,16 +506,43 @@ def load_archive(game, archive, progress=None):
     return sounds
 
 
+def available_archives(game):
+    """WAD archives of the game plus BANK when it has soundbanks."""
+    return list(game.archives) + ([banks.ARCHIVE] if banks.has_banks(game.cooked) else [])
+
+
 class Catalog:
-    """All sounds of the selected archives (default: the game's speech language + SFX)."""
+    """All sounds of the selected archives (default: the game's speech language, SFX and BANK)."""
 
     def __init__(self, game, archives=None, progress=None):
         self.game = game
-        self.archives = archives or [a for a in (game.language(), "SFX") if a in game.archives]
+        avail = available_archives(game)
+        self.archives = archives or [a for a in (game.voice_archive(game.language()), "SFX", banks.ARCHIVE) if a in avail]
         self.sounds = []
+        self.locations = {}          # BANK media id -> [Location]
+        self.heads = {}              # WAD sound key -> [Location] of its prefetch heads
+        self.bank_sha1 = {}          # BANK media id -> original checksum ('' if unknown), every language
+        bank_sounds_ = []
+        if banks.ARCHIVE in avail:
+            # always loaded: speech lines may have prefetch heads that must be written with them
+            bank_sounds_, self.locations = load_banks(game, progress)
+            self.bank_sha1 = {int(s.target, 16): s.sha1 for s in bank_sounds_}
+            for ls in self.locations.values():
+                for l in ls:
+                    if l.kind == banks.PREFETCH_HEAD and l.sound:
+                        self.heads.setdefault(l.sound, []).append(l)
+        # localized in-memory speech: only the languages loaded here (all of them if none is)
+        langs = [a for a in self.archives if a in game.languages] or None
+        bank_sounds_ = [s for s in bank_sounds_ if not langs or any(s.in_language(L) for L in langs)]
         for a in self.archives:
-            self.sounds += load_archive(game, a, progress)
+            self.sounds += bank_sounds_ if a == banks.ARCHIVE else load_archive(game, a, progress)
         self.by_key = {s.key: s for s in self.sounds}
+        # copies: one recording stored in two places of a WAD (same line, same archive); written together
+        same = collections.defaultdict(list)
+        for s in self.sounds:
+            if s.line and s.archive != banks.ARCHIVE:
+                same[(s.archive, s.line)].append(s)
+        self.copies = {s.key: [c for c in ss if c is not s] for ss in same.values() if len(ss) > 1 for s in ss}
         self._by_name = {}
         for s in self.sounds:
             for n in s.names():
@@ -242,6 +558,11 @@ class Catalog:
         key = os.path.splitext(os.path.basename(name))[0].lower() if re.search(r"\.\w{2,4}$", name) else name.lower()
         hits = self._by_name.get(key, [])
         uniq = list({id(s): s for s in hits}.values())
+        if len(uniq) > 1:
+            # a name wins over another sound's alias: a line named after its INT position (NAME#i) can equal
+            # the position alias of another recording in another language's WAD
+            own = [s for s in uniq if s.name.lower() == key]
+            uniq = own if len(own) == 1 else uniq
         if len(uniq) == 1:
             return uniq[0]
         if len(uniq) > 1:

ackit/classify.py

@@ -28,17 +28,26 @@ _RULES = [
     (r"^SPEECH_CHAR_", "Speech", "Character lines"),
     (r"^SPEECH_(CH|SEQUENCE|STEEL|MUSEUM|CHURCH|COURT|UNDER|GCPD|ZP1)_", "Speech", "Story missions"),
     (r"^SPEECH_(GAME|MAP|AUGMENTED|FORENSICS|TEMP)_", "Speech", "Gameplay & hints"),
-    (r"^NPC_", "Ambience", "Crowds"),
+    (r"^NPC_.*CROWD", "Ambience", "Crowds"),
     (r"^SPEECH_", "Speech", "Other speech"),
     (r"^MUSIC_PDLC", "Music", "Music (DLC)"),
-    (r"^MUSIC", "Music", "Music"),
+    (r"^(MUSIC|MUS_)", "Music", "Music"),
     (r"^(AMB|PSA)_", "Ambience", "Ambience"),
+    (r"^PSO_", "Ambience", "Ambient emitters"),
     (r"^WND_", "Ambience", "Wind"),
+    (r"^WTR_", "Ambience", "Weather"),
     (r"^P?VOL_", "Ambience", "Room tone"),
-    (r"^SEQ_", "SFX", "Scripted sequences"),
+    (r"FOOTSTEP", "SFX", "Footsteps"),
+    (r"^(CHR|NPC)_.*(COMBAT|BONEBREAK|BODYFALL)", "SFX", "Combat"),
+    (r"^(SEQ|MTN)_", "SFX", "Scripted sequences"),
     (r"^OBJ_", "SFX", "Objects"),
+    (r"^(PX|PHY)_", "SFX", "Physics"),
     (r"^WPN_", "SFX", "Weapons"),
-    (r"^CHR_", "SFX", "Character foley"),
+    (r"^DVC_", "SFX", "Gadgets"),
+    (r"^(GUI|SBJ)_", "SFX", "Interface"),
+    (r"^VFX_", "SFX", "Visual effects"),
+    (r"^NPC_", "SFX", "Enemies & NPCs"),
+    (r"^(CHR|GEN)_", "SFX", "Character foley"),
     (r"^GLOBAL", "SFX", "Global"),
 ]
 _RULES = [(re.compile(p), g, c) for p, g, c in _RULES]
@@ -46,9 +55,10 @@ _RULES = [(re.compile(p), g, c) for p, g, c in _RULES]
 GROUPS = ("Speech", "Music", "Ambience", "SFX", "Unknown")
 
 
-def classify(name: str, pack: str = None):
+def classify(name: str, pack: str = None, bank: str = None):
-    """-> (group, category). `pack` = name of the stream pack a media belongs to, if any."""
+    """-> (group, category). `pack` = name of the stream pack a media belongs to, if any; `bank` = name of
-    n = name.upper()
+    the soundbank holding an in-memory sound (its prefix is more reliable than event names)."""
+    n = (bank or name).upper()
     if pack:
         p = pack.upper()
         if p.startswith("MUSIC"):

ackit/cli.py

@@ -2,16 +2,19 @@
 
 Everyday use, by sound name (see `ackit find`):
   ackit find joker tape                           search names, categories, characters
-  ackit info Speech_TAPE_JOKER_1_4               details, size limit, modified state
+  ackit info Speech_TAPE_JOKER_1_04               details, size limit, modified state
-  ackit extract Speech_TAPE_JOKER_1_4 orig.wav   save the original (or --current)
+  ackit extract Speech_TAPE_JOKER_1_04 orig.wav   save the original (or --current)
-  ackit replace Speech_TAPE_JOKER_1_4 line.wav   encode + install a replacement
+  ackit replace Speech_TAPE_JOKER_1_04 line.wav   encode + install a replacement
-  ackit revert Speech_TAPE_JOKER_1_4             put the original back (--all for everything)
+  ackit revert Speech_TAPE_JOKER_1_04             put the original back (--all: every sound and text)
   ackit status                                    what you have changed
+  ackit text find --lang JPN batman               subtitles and menu text (any of the 11 languages)
   ackit export-mod my.acmod / import-mod my.acmod  share your changes
 
-The game folder is detected from Steam (Windows, Linux, Proton); override with --game DIR.
+The game folder is detected from Steam (Windows, Linux, Proton); override with --game DIR. --lang picks the
+language (default: the game's); languages without their own voices use the INT voice lines, like the game.
 """
 import argparse
+import json
 import os
 import sys
 
@@ -19,8 +22,7 @@ from . import audio, settings
 from .game import Game, GameError, find_installs, default_game_dir
 
 
-def _workspace(a, archives=None):
+def _game(a):
-    from .workspace import Workspace
     st = settings.load()
     gd = a.game or default_game_dir(st)
     if not gd:
@@ -29,9 +31,15 @@ def _workspace(a, archives=None):
     if a.game:
         st["game_dir"] = os.path.abspath(a.game)
         settings.save(st)
-    game = Game(gd)
+    return Game(gd)
+
+
+def _workspace(a, archives=None):
+    from .workspace import Workspace
+    st = settings.load()
+    game = _game(a)
     if archives is None and getattr(a, "lang", None):
-        archives = [a.lang, "SFX"]
+        archives = [game.voice_archive(a.lang), "SFX", "BANK"]
     return Workspace(game, a.store or st.get("store_root"), archives,
                      full_backup=not getattr(a, "no_full_backup", False) and st.get("full_backup", True),
                      progress=lambda n, t: None)
@@ -68,7 +76,8 @@ def cmd_list(a):
         c = a.character.lower()
         rows = [s for s in rows if c in s.character.lower() or (a.inferred and c in s.characters_inferred.lower())]
     if a.modified:
-        rows = [s for s in rows if s.key in ws.modified_keys()]
+        mod = set(ws.modified_keys())
+        rows = [s for s in rows if s.key in mod]
     if a.csv:
         from .catalog import write_csv
         write_csv(a.csv, rows)
@@ -116,6 +125,9 @@ def _process(a, x, sr, ws, s):
         x = audio.trim(x, sr, float(start or 0), float(end) if end else None)
     if a.trim_silence:
         x = audio.trim_silence(x, rate=sr)
+    if a.fx:
+        from . import fx
+        x = fx.render(x, sr, json.load(open(a.fx, encoding="utf-8")), s.channels)
     if a.match_loudness:
         ref, _ = ws.original_audio(s)
         x = audio.match_loudness(x, ref)
@@ -126,6 +138,14 @@ def _process(a, x, sr, ws, s):
     return x
 
 
+def cmd_fx_template(a):
+    from . import fx
+    with open(a.out, "w", encoding="utf-8") as f:
+        json.dump(fx.defaults(), f, indent=1)
+    print("wrote %s: set \"on\": true on the effects you want (blocks: %s)"
+          % (a.out, ", ".join(b.key for b in fx.BLOCKS)))
+
+
 def cmd_replace(a):
     ws = _workspace(a)
     s = ws.sound(a.name)
@@ -166,37 +186,99 @@ def cmd_revert(a):
     ws = _workspace(a)
     if a.all:
         done = ws.revert_all(progress=print)
-    else:
+        texts = ws.texts.revert_all(progress=print)
+        print("reverted %d sound(s) and %d text(s)" % (len(done), len(texts)))
+        return
     done = ws.revert([ws.sound(n) for n in a.names], progress=print)
     print("reverted %d sound(s)" % len(done))
 
 
+def cmd_restore_backup(a):
+    ws = _workspace(a)
+    full = sorted(ws.store.data.get("full", {}))
+    if not full:
+        print("no full copies in this store (%s)" % ws.store.root)
+        return
+    if not a.yes:
+        print("would copy back (or rebuild) %d file(s) from the full copies in %s:" % (len(full), ws.store.root))
+        for rel in full:
+            print("  " + rel)
+        print("Sounds and texts in these files are forgotten; sounds of WADs without a full copy stay recorded.\n"
+              "Use this only if the originals store is damaged (normally: ackit revert --all). Run again with --yes.")
+        return
+    done = ws.restore_from_full_backup(progress=print)
+    print("restored %d file(s); %d sound(s) still recorded" % (len(done), len(ws.store.entries)))
+
+
 def cmd_status(a):
     ws = _workspace(a)
     keys = ws.modified_keys()
-    if not keys:
+    texts = ws.texts.modified()
-        print("no sounds modified; game audio is original")
+    if not keys and not texts:
+        print("no sounds or texts modified; game files are original")
         return
     for k in keys:
         s = ws.catalog.by_key.get(k)
         st = ws.state(s) if s else "(other language)"
         print("%-50s %s" % (s.name if s else k, st))
+    for k in texts:
+        print("%-50s %s" % (k, ws.texts.state(k)))
 
 
 def cmd_export_mod(a):
     from .modpack import export_mod
     ws = _workspace(a)
     keys = [ws.sound(n).key for n in a.names] if a.names else ws.modified_keys()
+    texts = [] if a.names else ws.texts.modified()
     n = export_mod(ws, a.out, keys, a.title or os.path.splitext(os.path.basename(a.out))[0], a.author or "",
-                   a.description or "")
+                   a.description or "", texts=texts)
-    print("wrote %s (%d sounds)" % (a.out, n))
+    print("wrote %s (%d sounds, %d texts)" % (a.out, len(keys), n - len(keys)))
 
 
 def cmd_import_mod(a):
     from .modpack import import_mod
     ws = _workspace(a)
-    for im in import_mod(ws, a.file, force=a.force, progress=print):
+    items, texts = import_mod(ws, a.file, force=a.force, progress=print)
+    for im in items:
         print("%-8s %-46s %s" % (im.how, im.item.get("name"), im.note))
+    for k, _, problem in texts:
+        print("%-8s %-46s %s" % ("text", k, problem or "applied"))
+
+
+def cmd_text(a):
+    """Subtitles and menu text: find, show, set, revert."""
+    ws = _workspace(a)
+    tx = ws.texts
+    lang = (a.lang or ws.game.language()).upper()
+    if a.action == "find":
+        words = [w.lower() for w in a.args]
+        rows = []
+        if a.kind in ("all", "subs"):
+            rows += [(subtitles_key(s), s.text) for s in tx.subtitles(lang)]
+        if a.kind in ("all", "menu"):
+            rows += [("MENU:" + e.key, e.value) for e in tx.menu(lang)]
+        hits = [(k, v) for k, v in rows if all(w in (k + " " + v).lower() for w in words)]
+        for k, v in hits[:a.limit]:
+            print("%s\n    %s" % (k, v))
+        print("%d match(es)%s" % (len(hits), "" if len(hits) <= a.limit else " (first %d shown)" % a.limit))
+    elif a.action == "show":
+        for k in a.args:
+            print("%s\n    now: %s\n    original: %s\n    state: %s%s" % (
+                k, tx.current(k), tx.original(k), tx.state(k),
+                "" if tx.limit(k) is None else "\n    room: %d characters" % tx.limit(k)))
+    elif a.action == "set":
+        if len(a.args) != 2:
+            raise GameError("usage: ackit text set KEY \"new text\"")
+        tx.apply({a.args[0]: a.args[1]}, force=a.force, progress=print)
+        print("set %s" % a.args[0])
+    elif a.action == "revert":
+        done = tx.revert_all(progress=print) if a.all else tx.revert(a.args, progress=print)
+        print("reverted %d text(s)" % len(done))
+
+
+def subtitles_key(s):
+    from .subtitles import key
+    return key(s.lang, s.speech, s.line)
 
 
 # --- advanced ---------------------------------------------------------------------------------
@@ -236,19 +318,39 @@ def cmd_game(a):
         from .workspace import default_store_root
         print("game folder:     ", g.root)
         print("archives:        ", ", ".join("%s=%s" % (k, g.rel(v)) for k, v in g.archives.items()))
-        print("speech language: ", g.language())
+        print("language:        ", "%s (voices: %s)" % (g.language(), g.voice_archive(g.language())))
         print("originals store: ", a.store or st.get("store_root") or default_store_root(g))
         print("settings file:   ", settings.path())
 
 
 def cmd_verify(a):
+    if a.catalog:
+        from .verify import catalogcheck
+        archives = [x.upper() for x in a.archive] or ([a.lang] if a.lang else None)
+        rep = catalogcheck.run(_game(a), archives, quick=a.quick)
+        for (arch, kind), n in sorted(rep.counts().items()):
+            print("%-14s %-10s %d" % (arch, kind, n))
+        for arch, kind, key, detail in rep.issues[:a.show]:
+            print("  %s %s %s %s" % (arch, kind, key, detail))
+        print("checked: %s" % ", ".join("%s %d" % kv for kv in sorted(rep.checked.items())))
+        for f in rep.facts:
+            print("fact: " + f)
+        print("catalog OK" if rep.ok else "%d issue(s)" % len(rep.issues))
+        sys.exit(0 if rep.ok else 1)
     from .verify import selfcheck
     ws = _workspace(a)
     if not a.ingame:
         ok = selfcheck.run(ws, log=print)
         sys.exit(0 if ok else 1)
     from .verify import ingame
-    if a.baseline:
+    if a.lang and a.lang != ws.game.language():            # voices and texts follow the game's setting
+        raise GameError("the game plays its own language (%s); set it in Steam (game properties > Language) to "
+                        "test %s in the game" % (ws.game.language(), a.lang))
+    if a.banks:
+        rep = ingame.run_banks(ws, a.sound or None)
+    elif a.texts:
+        rep = ingame.run_texts(ws)
+    elif a.baseline:
         rep = ingame.run(ws, a.target, mode="baseline", seconds=a.seconds)
     elif a.installed:
         rep = ingame.run(ws, a.target, mode="installed", seconds=a.seconds)
@@ -274,6 +376,7 @@ def build_parser():
     proc = argparse.ArgumentParser(add_help=False)
     proc.add_argument("--trim", metavar="START:END", help="keep only this part (seconds), e.g. 0.5:4.2")
     proc.add_argument("--trim-silence", action="store_true", help="cut leading/trailing silence")
+    proc.add_argument("--fx", metavar="FILE", help="effects settings (JSON, see `ackit fx-template`)")
     proc.add_argument("--gain", type=float, metavar="DB", help="volume change in dB")
     proc.add_argument("--normalize", action="store_true", help="peak-normalize to -1 dBFS")
     proc.add_argument("--match-loudness", action="store_true", help="match the original's loudness")
@@ -292,6 +395,9 @@ def build_parser():
     s.add_argument("--group"); s.add_argument("--category"); s.add_argument("--character")
     s.add_argument("--inferred", action="store_true", help="also match characters inferred from names")
     s.add_argument("--modified", action="store_true"); s.add_argument("--csv", metavar="FILE")
+    s = add("fx-template", cmd_fx_template, "write effects settings with every effect off (edit, then "
+            "use with replace --fx)")
+    s.add_argument("out")
     s = add("info", cmd_info, "show details of a sound (or of a .wem file)")
     s.add_argument("name")
     s = add("extract", cmd_extract, "save a sound as .wav (decoded) or .wem")
@@ -301,19 +407,37 @@ def build_parser():
     s.add_argument("name"); s.add_argument("audio")
     s = add("apply", cmd_apply, "replace every sound named by the files in a folder", (enc,))
     s.add_argument("folder")
-    s = add("revert", cmd_revert, "put original sounds back", aliases=["restore"])
+    s = add("revert", cmd_revert, "put original sounds back (--all: every sound and text)", aliases=["restore"])
     s.add_argument("names", nargs="*"); s.add_argument("--all", action="store_true")
-    add("status", cmd_status, "list the sounds you modified")
+    add("status", cmd_status, "list the sounds and texts you modified")
+    s = add("restore-backup", cmd_restore_backup, "last resort if the originals store is damaged: put back the "
+            "full file copies it keeps (lists them; --yes to do it)")
+    s.add_argument("--yes", action="store_true")
+    s = add("text", cmd_text, "subtitles and menu text: find WORDS | show KEY | set KEY TEXT | revert KEY/--all")
+    s.add_argument("action", choices=["find", "show", "set", "revert"])
+    s.add_argument("args", nargs="*")
+    s.add_argument("--kind", choices=["all", "subs", "menu"], default="all", help="with find: what to search")
+    s.add_argument("--limit", type=int, default=40); s.add_argument("--all", action="store_true")
+    s.add_argument("--force", action="store_true", help="overwrite texts changed outside this tool")
     s = add("export-mod", cmd_export_mod, "save your changes as a shareable .acmod package")
     s.add_argument("out"); s.add_argument("names", nargs="*", help="default: everything you modified")
     s.add_argument("--title"); s.add_argument("--author"); s.add_argument("--description")
     s = add("import-mod", cmd_import_mod, "install a .acmod package", (enc,))
     s.add_argument("file")
     add("game", cmd_game, "show the detected game folder, archives, language and store")
-    s = add("verify", cmd_verify, "self-check (offline), or test in the real game with --ingame (Windows)")
+    s = add("verify", cmd_verify, "self-check (offline), catalog cross-check (--catalog), or test in the real "
+            "game with --ingame (Windows)")
+    s.add_argument("--catalog", action="store_true", help="cross-check the catalog against the game files")
+    s.add_argument("--archive", action="append", default=[], help="with --catalog: only this archive (repeatable; --lang X: only X)")
+    s.add_argument("--quick", action="store_true", help="with --catalog: loose banks + a sample of packages")
+    s.add_argument("--show", type=int, default=30, help="with --catalog: issues to list (default 30)")
     s.add_argument("--ingame", action="store_true"); s.add_argument("--target", default="SFX:EDF036D6/393BC53A")
     s.add_argument("--audio"); s.add_argument("--baseline", action="store_true")
     s.add_argument("--installed", action="store_true"); s.add_argument("--seconds", type=float, default=40)
+    s.add_argument("--banks", action="store_true",
+                   help="with --ingame: test in-memory sounds (default: the front-end banks, or --sound NAME ...)")
+    s.add_argument("--sound", action="append", default=[], help="with --banks: a sound to test (repeatable)")
+    s.add_argument("--texts", action="store_true", help="with --ingame: test menu text and subtitle edits")
     # advanced
     s = add("encode", cmd_encode, "[advanced] audio -> .wem file", (enc,))
     s.add_argument("audio"); s.add_argument("out"); s.add_argument("--like", metavar="NAME")

ackit/codebooks.py

@@ -20,17 +20,6 @@ def load_library(path: str = DEFAULT_LIB) -> list:
     return [b[offs[i]:offs[i + 1]] for i in range(n - 1)]
 
 
-def save_library(path: str, books: list):
-    out = bytearray()
-    offs = []
-    for bk in books:
-        offs.append(len(out))
-        out += bk
-    offs.append(len(out))  # table start == end of last book
-    out += struct.pack("<%dI" % len(offs), *offs)
-    open(path, "wb").write(bytes(out))
-
-
 def wwise_book_to_vorbis(blob: bytes, w: BitWriter):
     """Expand a compact Wwise codebook into a standard Vorbis codebook (ww2ogg rebuild)."""
     r = BitReader(blob)

ackit/fx.py

@@ -0,0 +1,508 @@
+"""Audio effects for replacements (any sound: voices, effects, music), set by hand. The blocks cover the effects
+found in the game's sounds (docs/history/mission4_dubbing_plan.md sections 9-10).
+
+`BLOCKS` is the single description of every effect block and its parameters (the editor builds its controls from
+it). Settings are plain dicts {block key: {"on": bool, param key: value}} (JSON-safe). `render` applies the blocks
+that are on, in the order of `BLOCKS`; random parts (glitches, reverse reverb, noise) use a fixed seed, so the same
+settings always give the same audio. Audio is float32 (frames, channels) like `ackit.audio`; effects can make it
+longer (glitch repeats, echo and reverb tails).
+
+Built on pedalboard (GPL-3.0): filters, distortion, delay, reverb, codec and Rubber Band pitch/formant shifting;
+the rest (ring modulator, glitches, noise, reverse reverb, channel layout) is numpy.
+"""
+from collections import namedtuple
+
+import numpy as np
+
+Param = namedtuple("Param", "key label kind lo hi default step unit tip", defaults=("", ""))
+Block = namedtuple("Block", "key label tip params", defaults=((),))
+
+BLOCKS = (
+    Block("pitch", "Pitch", "Pitch shifter (moves the sound and its formants together, like the game's Clayface), "
+          "optional formant shift and a pitch that drifts over time (Mr. Freeze).", (
+              Param("shift", "Shift", "float", -24, 24, 0.0, 0.5, "st", "Pitch shift in semitones"),
+              Param("formant", "Formant", "float", -12, 12, 0.0, 0.5, "st",
+                    "Moves only the formants (voice size/character), the pitch stays"),
+              Param("drift", "Drift", "float", 0, 12, 0.0, 0.1, "st", "Pitch over time: depth of the slow swing"),
+              Param("drift_rate", "Drift rate", "float", 0.05, 5, 0.3, 0.05, "Hz", "Speed of the swing"))),
+    Block("octaves", "Octave layers", "Pitch-shifted copies of the sound mixed under it (Ra's al Ghul, Poison Ivy).", (
+        Param("interval1", "Layer 1", "float", -24, 12, -12.0, 1, "st", "Interval of the first layer"),
+        Param("level1", "Level 1", "float", -40, 6, -6.0, 0.5, "dB", "Level of the first layer"),
+        Param("interval2", "Layer 2", "float", -24, 12, -24.0, 1, "st", "Interval of the second layer"),
+        Param("level2", "Level 2", "float", -60, 6, -60.0, 0.5, "dB", "Level of the second layer (-60 = off)"))),
+    Block("robot", "Robot", "Ring modulator and metallic comb (Mr. Freeze's suit).", (
+        Param("ring_freq", "Ring freq", "float", 10, 1000, 80.0, 1, "Hz", "Ring modulator carrier frequency"),
+        Param("ring_mix", "Ring mix", "float", 0, 1, 0.5, 0.05, "", "0 = dry, 1 = fully modulated"),
+        Param("comb_ms", "Comb", "float", 0, 20, 0.0, 0.1, "ms", "Metallic comb delay (0 = off)"),
+        Param("comb_feedback", "Comb feedback", "float", 0, 0.95, 0.6, 0.05, "", "Ringing of the comb"))),
+    Block("glitch", "Glitch", "Short pieces of the sound repeated right after themselves, optionally at another "
+          "pitch (the Riddler's hacked broadcasts). Makes the line longer.", (
+              Param("rate", "Events", "float", 0, 20, 3.0, 0.5, "/10 s", "How many glitches per 10 seconds of audio"),
+              Param("length", "Length", "float", 20, 400, 120.0, 5, "ms", "Length of the repeated piece"),
+              Param("repeats", "Repeats", "int", 1, 6, 2, 1, "", "How many times a piece repeats"),
+              Param("pitch", "Pitch", "float", -12, 12, 5.0, 0.5, "st", "Pitch of the repeats (0 = same)"),
+              Param("seed", "Variation", "int", 0, 999, 1, 1, "", "Another number = glitches at other places"))),
+    Block("eq", "Band-pass EQ", "Low and high cut with a mid boost: radio, phone, phone microphone, TV speaker, "
+          "small speakers.", (
+        Param("low_cut", "Low cut", "float", 20, 2000, 300.0, 10, "Hz", "Frequencies below are removed"),
+        Param("high_cut", "High cut", "float", 500, 20000, 3400.0, 50, "Hz", "Frequencies above are removed"),
+        Param("slope", "Slope", "int", 6, 48, 24, 6, "dB/oct", "How steeply the cuts fall"),
+        Param("mid_freq", "Mid freq", "float", 200, 5000, 1500.0, 10, "Hz", "Centre of the boost"),
+        Param("mid_gain", "Mid gain", "float", -12, 18, 6.0, 0.5, "dB", "Boost (or cut) around the centre"),
+        Param("mid_q", "Mid Q", "float", 0.3, 5, 1.0, 0.1, "", "Width of the boost (higher = narrower)"))),
+    Block("distortion", "Distortion", "Speaker/radio distortion, clipping, low bit depth, phone codec.", (
+        Param("drive", "Drive", "float", 0, 40, 10.0, 0.5, "dB", "Soft saturation (0 = off)"),
+        Param("clip", "Clip", "float", -30, 0, 0.0, 0.5, "dB", "Hard clipping threshold (0 = off)"),
+        Param("bits", "Bits", "int", 4, 16, 16, 1, "", "Bit depth (16 = off)"),
+        Param("gsm", "Phone codec", "bool", 0, 1, False, 1, "", "GSM mobile phone codec"))),
+    Block("noise", "Noise", "Radio static or hiss under the sound.", (
+        Param("level", "Level", "float", -60, 0, -30.0, 0.5, "dB", "Level relative to the sound"),
+        Param("colour", "Colour", "float", 200, 8000, 2000.0, 50, "Hz", "Centre of the noise band"),
+        Param("crackle", "Crackle", "float", 0, 1, 0.0, 0.05, "", "Random clicks and pops"))),
+    Block("echo", "Echo", "Delay (PA announcements: about 120-170 ms; very short times widen the sound).", (
+        Param("time", "Time", "float", 1, 1000, 160.0, 1, "ms", "Delay time"),
+        Param("feedback", "Feedback", "float", 0, 0.9, 0.3, 0.05, "", "How many repeats"),
+        Param("mix", "Mix", "float", 0, 1, 0.35, 0.05, "", "Level of the echo"))),
+    Block("reverb", "Reverb", "Room reverb (a generated room response: early reflections and a tail that gets "
+          "darker as it fades), and an occasional reverse swell before phrases (Riddler, Ivy).", (
+              Param("decay", "Decay", "float", 0.1, 6, 0.6, 0.05, "s", "Reverb time (RT60): small room 0.3, "
+                    "hall 2, church 4"),
+              Param("mix", "Mix", "float", 0, 1, 0.25, 0.05, "", "Dry sound against reverb: 0 = dry only, "
+                    "1 = reverb only"),
+              Param("predelay", "Pre-delay", "float", 0, 150, 10.0, 1, "ms", "Gap before the reverb starts "
+                    "(bigger spaces: longer)"),
+              Param("early", "Reflections", "float", 0, 1, 0.5, 0.05, "", "Early echoes from nearby walls"),
+              Param("damping", "Damping", "float", 0, 1, 0.5, 0.05, "", "How much darker the tail gets "
+                    "(soft rooms high, tiled rooms low)"),
+              Param("width", "Width", "float", 0, 1, 1.0, 0.05, "", "Different reverb per channel (2+ channels)"),
+        Param("reverse", "Reverse", "float", 0, 1, 0.0, 0.05, "", "Level of the reverse swells (0 = off)"),
+        Param("reverse_every", "Reverse often", "float", 0, 1, 0.3, 0.05, "",
+              "Share of the phrases that get a reverse swell"),
+        Param("seed", "Variation", "int", 0, 999, 1, 1, "", "Another number = swells on other phrases"))),
+    Block("channels", "Channels", "Only for sounds with 2-5 channels (L R C SL SR): stereo width, pan over time, "
+          "the PA layout (dry sound in the centre, echo in the surrounds).", (
+              Param("width", "Width", "float", 0, 30, 0.0, 0.5, "ms", "Delay of the right side (0 = off)"),
+              Param("pan_rate", "Pan rate", "float", 0, 2, 0.0, 0.05, "Hz", "Pan over time (0 = off)"),
+              Param("pan_depth", "Pan depth", "float", 0, 1, 0.5, 0.05, "", "How far the sound moves"),
+              Param("front", "Front L/R", "float", -60, 0, -12.0, 0.5, "dB", "Front L/R level when there is a centre"),
+              Param("surround_delay", "Surround delay", "float", 0, 500, 160.0, 1, "ms",
+                    "Echo delay of the surrounds (4-5 channels)"),
+              Param("surround", "Surround", "float", -60, 0, -9.0, 0.5, "dB", "Level of the surrounds"))),
+)
+BY_KEY = {b.key: b for b in BLOCKS}
+
+# Starting points (values set in place; the user tunes them by ear). Per block: {block: [(name, values)]}.
+PRESETS = {
+    "pitch": [("Down (Clayface)", {"shift": -5.0, "formant": 0.0, "drift": 0.0}),
+              ("Deep, same voice", {"shift": -3.0, "formant": 2.0, "drift": 0.0}),
+              ("Up", {"shift": 4.0, "formant": 0.0, "drift": 0.0}),
+              ("Drift (Mr. Freeze)", {"shift": 0.0, "formant": 0.0, "drift": 1.5, "drift_rate": 0.35})],
+    "octaves": [("One octave down", {"interval1": -12.0, "level1": -4.0, "level2": -60.0}),
+                ("Giant (two down)", {"interval1": -12.0, "level1": -3.0, "interval2": -24.0, "level2": -9.0}),
+                ("Fifth down", {"interval1": -7.0, "level1": -8.0, "level2": -60.0})],
+    "robot": [("Suit (Mr. Freeze)", {"ring_freq": 60.0, "ring_mix": 0.3, "comb_ms": 4.0, "comb_feedback": 0.4}),
+              ("Metallic", {"ring_freq": 30.0, "ring_mix": 0.1, "comb_ms": 9.0, "comb_feedback": 0.7}),
+              ("Dalek", {"ring_freq": 30.0, "ring_mix": 1.0, "comb_ms": 0.0})],
+    "glitch": [("Light", {"rate": 2.0, "length": 90.0, "repeats": 1, "pitch": 0.0}),
+               ("Hacked (Riddler)", {"rate": 4.0, "length": 120.0, "repeats": 2, "pitch": 5.0}),
+               ("Heavy", {"rate": 8.0, "length": 70.0, "repeats": 3, "pitch": -7.0})],
+    "eq": [("Radio", {"low_cut": 400.0, "high_cut": 3200.0, "slope": 24, "mid_freq": 1800.0, "mid_gain": 6.0,
+                      "mid_q": 1.0}),
+           ("Phone", {"low_cut": 300.0, "high_cut": 3400.0, "slope": 36, "mid_freq": 1500.0, "mid_gain": 4.0,
+                      "mid_q": 0.8}),
+           ("Phone mic (tape)", {"low_cut": 200.0, "high_cut": 5000.0, "slope": 18, "mid_freq": 2500.0,
+                                 "mid_gain": 5.0, "mid_q": 0.7}),
+           ("TV speaker", {"low_cut": 250.0, "high_cut": 6000.0, "slope": 18, "mid_freq": 1200.0, "mid_gain": 3.0,
+                           "mid_q": 0.7}),
+           ("Megaphone / PA", {"low_cut": 500.0, "high_cut": 3000.0, "slope": 30, "mid_freq": 2000.0,
+                               "mid_gain": 8.0, "mid_q": 1.2})],
+    "distortion": [("Speaker", {"drive": 8.0, "clip": 0.0, "bits": 16, "gsm": False}),
+                   ("Overdriven radio", {"drive": 18.0, "clip": -6.0, "bits": 16, "gsm": False}),
+                   ("Phone codec", {"drive": 0.0, "clip": 0.0, "bits": 16, "gsm": True}),
+                   ("Lo-fi", {"drive": 4.0, "clip": 0.0, "bits": 8, "gsm": False})],
+    "noise": [("Radio static", {"level": -26.0, "colour": 2500.0, "crackle": 0.1}),
+              ("Soft hiss", {"level": -36.0, "colour": 6000.0, "crackle": 0.0}),
+              ("Crackle", {"level": -40.0, "colour": 1500.0, "crackle": 0.5})],
+    "echo": [("PA announcement", {"time": 160.0, "feedback": 0.3, "mix": 0.35}),
+             ("Slapback", {"time": 90.0, "feedback": 0.0, "mix": 0.3}),
+             ("Short (width)", {"time": 12.0, "feedback": 0.0, "mix": 0.3}),
+             ("Canyon", {"time": 400.0, "feedback": 0.45, "mix": 0.3})],
+    "reverb": [("Small room", {"decay": 0.35, "mix": 0.15, "predelay": 3.0, "early": 0.6, "damping": 0.6}),
+               ("Room", {"decay": 0.7, "mix": 0.18, "predelay": 8.0, "early": 0.5, "damping": 0.5}),
+               ("Tiled / cell", {"decay": 1.0, "mix": 0.22, "predelay": 5.0, "early": 0.8, "damping": 0.15}),
+               ("Hall", {"decay": 2.0, "mix": 0.22, "predelay": 25.0, "early": 0.4, "damping": 0.5}),
+               ("Church", {"decay": 4.0, "mix": 0.26, "predelay": 40.0, "early": 0.3, "damping": 0.4}),
+               ("Outdoors (city)", {"decay": 1.5, "mix": 0.15, "predelay": 60.0, "early": 0.15, "damping": 0.7})],
+    "channels": [("PA layout", {"width": 0.0, "pan_rate": 0.0, "front": -12.0, "surround_delay": 160.0,
+                                "surround": -9.0}),
+                 ("Wide stereo", {"width": 12.0, "pan_rate": 0.0, "front": 0.0, "surround": -12.0,
+                                  "surround_delay": 20.0}),
+                 ("Slow pan (Mr. Freeze)", {"width": 0.0, "pan_rate": 0.25, "pan_depth": 0.6, "front": -6.0})],
+}
+# Whole presets (several blocks) for the sounds heard in the game (docs/history/mission4_dubbing_plan.md
+# section 9): every other block off. (name, {block: preset name of PRESETS or values})
+LOOKS = [
+    ("Radio", {"eq": "Radio", "distortion": "Speaker", "noise": "Radio static"}),
+    ("Phone", {"eq": "Phone", "distortion": "Phone codec"}),
+    ("Tape / phone mic", {"eq": "Phone mic (tape)", "noise": "Soft hiss"}),
+    ("TV in a room", {"eq": "TV speaker", "reverb": "Small room"}),
+    ("PA announcement", {"eq": "Megaphone / PA", "echo": "PA announcement", "channels": "PA layout"}),
+    ("Mr. Freeze", {"pitch": "Drift (Mr. Freeze)", "robot": "Suit (Mr. Freeze)", "channels": "Slow pan (Mr. Freeze)"}),
+    ("Clayface", {"pitch": "Down (Clayface)"}),
+    ("Riddler broadcast", {"glitch": "Hacked (Riddler)", "eq": "Phone", "echo": "Short (width)",
+                           "reverb": {"decay": 0.6, "mix": 0.11, "reverse": 0.6, "reverse_every": 0.3}}),
+    ("Giant (Ra's al Ghul)", {"octaves": "Giant (two down)", "pitch": {"formant": -3.0},
+                              "reverb": "Hall"}),
+    ("Car radio", {"eq": "Radio", "distortion": "Overdriven radio", "noise": "Radio static",
+                   "pitch": {"formant": 3.0}}),
+]
+
+
+def preset(block, name):
+    """Values of a block preset (without 'on')."""
+    return dict(next(v for n, v in PRESETS[block] if n == name))
+
+
+def with_preset(settings, block, name):
+    """Settings with `block` on and set to preset `name`."""
+    s = normalize(settings)
+    s[block].update(preset(block, name), on=True)
+    return normalize(s)
+
+
+def look(name):
+    """Settings of a whole preset (LOOKS): its blocks on (preset or values over the defaults), every other off."""
+    s = defaults()
+    for block, v in dict(LOOKS)[name].items():
+        s[block].update(preset(block, v) if isinstance(v, str) else v, on=True)
+    return normalize(s)
+LAYOUTS = {2: "L R", 3: "L R C", 4: "L R SL SR", 5: "L R C SL SR", 6: "L R C LFE SL SR"}   # WAVE channel masks
+
+
+def defaults():
+    """Every block off, every parameter at its default."""
+    return {b.key: dict({"on": False}, **{p.key: p.default for p in b.params}) for b in BLOCKS}
+
+
+RENAMED = {("reverb", "mix"): "wet"}           # parameter -> its name in settings of earlier versions
+
+
+def normalize(settings):
+    """Complete and clamp settings (unknown keys dropped), so old or hand-edited settings always render."""
+    out = defaults()
+    for b in BLOCKS:
+        s = (settings or {}).get(b.key) or {}
+        out[b.key]["on"] = bool(s.get("on", False))
+        for p in b.params:
+            v = s.get(p.key, s.get(RENAMED.get((b.key, p.key)), p.default))
+            try:
+                v = bool(v) if p.kind == "bool" else int(round(float(v))) if p.kind == "int" else float(v)
+            except (TypeError, ValueError):
+                v = p.default
+            out[b.key][p.key] = v if p.kind == "bool" else min(max(v, p.lo), p.hi)
+    return out
+
+
+def active(settings):
+    return any((settings or {}).get(b.key, {}).get("on") for b in BLOCKS)
+
+
+def summary(settings):
+    """Short text of the blocks that are on, e.g. 'Pitch, Band-pass EQ'."""
+    return ", ".join(b.label for b in BLOCKS if (settings or {}).get(b.key, {}).get("on"))
+
+
+# --- helpers -------------------------------------------------------------------------------------
+def _db(v):
+    return 10 ** (v / 20.0)
+
+
+def _pb(plugins, x, rate):
+    """Run pedalboard plugins on (frames, channels) audio."""
+    import pedalboard
+    y = pedalboard.Pedalboard(list(plugins))(np.ascontiguousarray(x.T, dtype=np.float32), rate)
+    return y.T
+
+
+def _shift(x, rate, semitones, formants=False):
+    """Pitch shift without changing the length (Rubber Band); `semitones` may be a per-frame curve."""
+    import pedalboard
+    st = semitones if np.ndim(semitones) else float(semitones)
+    y = pedalboard.time_stretch(np.ascontiguousarray(x.T, dtype=np.float32), rate,
+                                pitch_shift_in_semitones=st, preserve_formants=formants)
+    y = y.T
+    if len(y) != len(x):                          # same length as the input (Rubber Band may differ by a few frames)
+        y = y[:len(x)] if len(y) > len(x) else np.pad(y, ((0, len(x) - len(y)), (0, 0)))
+    return y
+
+
+def _rms(x):
+    return float(np.sqrt(np.mean(np.square(x)))) if len(x) else 0.0
+
+
+def _envelope_db(x, rate, hop_s=0.01):
+    hop = max(1, int(rate * hop_s))
+    m = x.mean(axis=1)
+    n = len(m) // hop
+    if n == 0:
+        return np.zeros(0)
+    return 10 * np.log10(np.mean(m[:n * hop].reshape(n, hop) ** 2, axis=1) + 1e-12)
+
+
+def _phrase_starts(x, rate, gap=0.15):
+    """Frame indices (in samples) where speech starts after at least `gap` seconds of quiet."""
+    env = _envelope_db(x, rate)
+    if not len(env):
+        return []
+    on = env > np.max(env) - 30
+    starts, quiet = [], int(gap / 0.01)
+    run = quiet                                   # the start of the audio counts as after a pause
+    for i, a in enumerate(on):
+        if a and run >= quiet:
+            starts.append(int(i * 0.01 * rate))
+        run = 0 if a else run + 1
+    return starts
+
+
+def _band(x, rate, low=None, high=None, order=4):
+    """Zero-phase Butterworth high-pass (`low`) and/or low-pass (`high`) magnitude applied in the frequency domain
+    (exact slope of 6 dB/octave per order, flat pass band; pedalboard's cuts are first order only)."""
+    pad = int(0.2 * rate)
+    n = 1 << int(np.ceil(np.log2(len(x) + 2 * pad)))
+    f = np.fft.rfftfreq(n, 1 / rate)
+    h = np.ones_like(f)
+    with np.errstate(divide="ignore"):
+        if low:
+            h /= np.sqrt(1 + (low / np.maximum(f, 1e-9)) ** (2 * order))
+        if high and high < rate / 2:
+            h /= np.sqrt(1 + (f / high) ** (2 * order))
+    spec = np.fft.rfft(np.pad(x, ((pad, 0), (0, 0))), n, axis=0) * h[:, None]
+    return np.fft.irfft(spec, n, axis=0)[pad:pad + len(x)].astype(np.float32)
+
+
+def _with_tail(fn, x, rate, seconds):
+    """Run `fn` on x padded with `seconds` of silence, then drop the silent end (never shorter than x)."""
+    pad = np.zeros((int(seconds * rate), x.shape[1]), np.float32)
+    y = fn(np.concatenate([x, pad]))
+    lvl = np.max(np.abs(y), axis=1)
+    loud = np.flatnonzero(lvl > max(np.max(lvl, initial=0.0), 1e-9) * 10 ** (-60 / 20))
+    end = max(len(x), (loud[-1] + 1) if len(loud) else len(x))
+    return y[:end]
+
+
+def _fade(seg, rate, ms=5):
+    n = min(len(seg) // 2, int(rate * ms / 1000))
+    if n:
+        r = np.linspace(0, 1, n, dtype=np.float32)[:, None]
+        seg[:n] *= r
+        seg[-n:] *= r[::-1]
+    return seg
+
+
+# --- blocks --------------------------------------------------------------------------------------
+def _pitch(x, rate, p):
+    if p["formant"]:
+        x = _shift(_shift(x, rate, p["formant"]), rate, -p["formant"], formants=True)
+    if p["drift"]:
+        t = np.arange(len(x)) / rate
+        return _shift(x, rate, p["shift"] + p["drift"] * np.sin(2 * np.pi * p["drift_rate"] * t))
+    return _shift(x, rate, p["shift"]) if p["shift"] else x
+
+
+def _octaves(x, rate, p):
+    y = x.copy()
+    for i in (1, 2):
+        if p["level%d" % i] > -60 and p["interval%d" % i]:
+            y += _shift(x, rate, p["interval%d" % i]) * _db(p["level%d" % i])
+    return y
+
+
+def _robot(x, rate, p):
+    t = (np.arange(len(x)) / rate)[:, None]
+    y = x * (1 - p["ring_mix"]) + x * np.sin(2 * np.pi * p["ring_freq"] * t) * p["ring_mix"]
+    if p["comb_ms"] > 0:
+        import pedalboard
+        y = _pb([pedalboard.Delay(p["comb_ms"] / 1000.0, p["comb_feedback"], 0.5)], y, rate)
+    return y.astype(np.float32)
+
+
+def _glitch(x, rate, p):
+    n = int(round(p["rate"] * len(x) / rate / 10))
+    seg = int(rate * p["length"] / 1000)
+    if n <= 0 or seg < 8 or len(x) <= seg:
+        return x
+    env = _envelope_db(x, rate)
+    voiced = np.flatnonzero(env > np.max(env) - 25) * int(rate * 0.01) if len(env) else np.zeros(0, int)
+    voiced = voiced[voiced + seg < len(x)]
+    if not len(voiced):
+        return x
+    rng = np.random.default_rng(p["seed"])
+    at = np.unique(rng.choice(voiced, size=min(n, len(voiced)), replace=False))
+    parts, last = [], 0
+    for a in at:
+        if a < last:
+            continue
+        piece = _fade(x[a:a + seg].copy(), rate)
+        parts.append(x[last:a + seg])
+        for k in range(p["repeats"]):
+            parts.append(_fade(_shift(piece, rate, p["pitch"] * (k + 1)), rate) if p["pitch"] else piece)
+        last = a + seg
+    parts.append(x[last:])
+    return np.concatenate(parts).astype(np.float32)
+
+
+def _eq(x, rate, p):
+    import pedalboard
+    y = _band(x, rate, p["low_cut"], p["high_cut"], max(1, int(p["slope"]) // 6))
+    if p["mid_gain"] and p["mid_freq"] < rate / 2 * 0.95:
+        y = _pb([pedalboard.PeakFilter(p["mid_freq"], p["mid_gain"], p["mid_q"])], y, rate)
+    return y
+
+
+def _distortion(x, rate, p):
+    import pedalboard
+    plugins = []
+    if p["drive"] > 0:
+        plugins.append(pedalboard.Distortion(p["drive"]))
+    if p["clip"] < 0:
+        plugins.append(pedalboard.Clipping(p["clip"]))
+    if p["bits"] < 16:
+        plugins.append(pedalboard.Bitcrush(p["bits"]))
+    if p["gsm"]:
+        plugins.append(pedalboard.GSMFullRateCompressor())
+    if not plugins:
+        return x
+    y = _pb(plugins, x, rate)
+    r = _rms(y)
+    return y * (_rms(x) / r) if r > 0 else y        # same loudness as before: only the colour changes
+
+
+def _noise(x, rate, p):
+    rng = np.random.default_rng(0)
+    n = _band(rng.standard_normal((len(x), 1)).astype(np.float32), rate, p["colour"] / 2, p["colour"] * 2, 2)
+    if p["crackle"] > 0:
+        clicks = (rng.random(len(x)) < p["crackle"] * 20 / rate) * rng.uniform(-1, 1, len(x))
+        n = n / (_rms(n) or 1) + clicks[:, None].astype(np.float32) * 8
+    r = _rms(n)
+    return x + (n / r if r else n) * _rms(x) * _db(p["level"])
+
+
+def _echo(x, rate, p):
+    import pedalboard
+    fb = p["feedback"]
+    tail = p["time"] / 1000 * (1 + (np.log(1e-3) / np.log(fb) if fb > 0.01 else 0))
+    return _with_tail(lambda a: _pb([pedalboard.Delay(p["time"] / 1000.0, fb, p["mix"])], a, rate),
+                      x, rate, min(tail, 6.0))
+
+
+REVERB_BANDS = ((0, 500), (500, 2000), (2000, 6000), (6000, None))
+REVERB_DARK = (0.0, 0.3, 0.6, 0.85)          # share of the decay a band loses at damping 1 (highs die first)
+
+
+def room_response(rate, channels, decay, predelay=10.0, early=0.5, damping=0.5, width=1.0, seed=7):
+    """Generated room impulse response -> float32 (frames, channels), energy 1 per channel: sparse early
+    reflections, then a dense exponentially decaying noise tail in 4 bands, the higher ones shorter with damping.
+    Each channel gets its own noise (decorrelated), mixed towards a common one as `width` goes to 0."""
+    rng = np.random.default_rng(seed)
+    n = int(rate * (decay * 1.2 + predelay / 1000)) + 1
+    t = np.arange(n) / rate
+    pre = int(rate * predelay / 1000)
+
+    def one(r):
+        tail = np.zeros(n, np.float32)
+        noise = r.standard_normal((n, 1)).astype(np.float32)
+        for (lo, hi), dark in zip(REVERB_BANDS, REVERB_DARK):
+            rt = max(0.05, decay * (1 - damping * dark))
+            tail += _band(noise, rate, lo or None, hi, 4)[:, 0] * 10 ** (-3 * t / rt)
+        tail *= np.minimum(1, t / 0.015)                      # the dense tail builds up over ~15 ms
+        er = np.zeros(n, np.float32)
+        span = 0.01 + 0.03 * min(decay, 2)                    # reflections within 10-70 ms
+        for k in range(12):
+            i = int(rate * r.uniform(0.002, span))
+            er[i] += r.choice((-1, 1)) * (1 - k / 14) * 3
+        h = tail / (np.sqrt(np.sum(tail ** 2)) or 1) + early * er / (np.sqrt(np.sum(er ** 2)) or 1)
+        return np.concatenate([np.zeros(pre, np.float32), h])[:n]
+
+    common = one(np.random.default_rng(seed + 1000))
+    out = np.stack([width * one(rng) + (1 - width) * common for _ in range(channels)], axis=1)
+    return (out / np.sqrt(np.sum(out ** 2, axis=0, keepdims=True)).clip(1e-9)).astype(np.float32)
+
+
+def _convolve(x, ir):
+    """Per channel FFT convolution (full length)."""
+    n = len(x) + len(ir) - 1
+    m = 1 << int(np.ceil(np.log2(n)))
+    return np.fft.irfft(np.fft.rfft(x, m, axis=0) * np.fft.rfft(ir, m, axis=0), m, axis=0)[:n].astype(np.float32)
+
+
+def _reverb(x, rate, p):
+    ir = room_response(rate, x.shape[1], p["decay"], p["predelay"], p["early"], p["damping"], p["width"])
+    y = x
+    if p["mix"] > 0:
+        wet = _convolve(x, ir)
+        wet *= _rms(x) / (_rms(wet) or 1)                     # the reverb alone is as loud as the sound
+        a = p["mix"] * np.pi / 2                              # equal-power crossfade: steady level at any mix
+        y = np.concatenate([x, np.zeros((len(wet) - len(x), x.shape[1]), np.float32)]) * np.cos(a)
+        y = y + wet * np.sin(a)
+        lvl = np.max(np.abs(y), axis=1)                       # drop the inaudible end of the tail
+        loud = np.flatnonzero(lvl > max(np.max(lvl, initial=0.0), 1e-9) * 10 ** (-60 / 20))
+        y = y[:max(len(x), loud[-1] + 1 if len(loud) else len(x))]
+    if p["reverse"] > 0:
+        rng = np.random.default_rng(p["seed"])
+        swell_ir = room_response(rate, x.shape[1], max(0.6, p["decay"]), 0, p["early"], p["damping"], p["width"])
+        y = y.copy()
+        for s in _phrase_starts(x, rate):
+            if rng.random() >= p["reverse_every"] or s < rate // 20:
+                continue
+            head = x[s:s + int(0.3 * rate)][::-1]
+            sw = _convolve(head, swell_ir)[::-1]
+            sw = sw[:len(sw) - len(head)]          # the swell ends where the phrase starts
+            a = max(0, s - len(sw))
+            sw = sw[len(sw) - (s - a):]
+            peak = float(np.max(np.abs(sw), initial=0.0))
+            if peak > 0:
+                y[a:s] += sw / peak * float(np.max(np.abs(x[s:s + int(0.3 * rate)]), initial=0.0)) * p["reverse"]
+    return y
+
+
+def _channels(x, rate, p, channels):
+    """Mono sound -> `channels` outputs in WAVE order (LAYOUTS)."""
+    v = x.mean(axis=1)
+    names = LAYOUTS.get(channels, " ".join(["L"] * channels)).split()
+    t = np.arange(len(v)) / rate
+    pan = p["pan_depth"] * np.sin(2 * np.pi * p["pan_rate"] * t) if p["pan_rate"] > 0 else np.zeros(len(v))
+    ang = (pan + 1) * np.pi / 4                    # equal power: centre = both sides at 1.0
+    g_left, g_right = np.cos(ang) * np.sqrt(2), np.sin(ang) * np.sqrt(2)
+
+    def delayed(sig, ms):
+        d = int(rate * ms / 1000)
+        return np.concatenate([np.zeros(d, np.float32), sig])[:len(sig)] if d else sig
+
+    front = _db(p["front"]) if "C" in names else 1.0
+    sur = delayed(v, p["surround_delay"]) * _db(p["surround"])
+    out = []
+    for nm in names:
+        out.append({"L": v * front * g_left, "R": delayed(v, p["width"]) * front * g_right, "C": v,
+                    "LFE": np.zeros_like(v), "SL": sur * g_left, "SR": sur * g_right}[nm])
+    return np.stack(out, axis=1).astype(np.float32)
+
+
+_ORDER = (("pitch", _pitch), ("octaves", _octaves), ("robot", _robot), ("glitch", _glitch), ("eq", _eq),
+          ("distortion", _distortion), ("noise", _noise), ("echo", _echo), ("reverb", _reverb))
+
+
+def render(x, rate, settings, channels=None):
+    """Apply the blocks that are on -> float32 (frames, channels). `channels` = the target sound's channel count:
+    with the Channels block on and 2+ channels the result has that many channels in the game's order; otherwise
+    the channel count of `x` is kept (the encoder up/down-mixes as before)."""
+    s = normalize(settings)
+    x = np.asarray(x, dtype=np.float32)
+    x = x[:, None] if x.ndim == 1 else x
+    if not len(x):
+        return x
+    for key, fn in _ORDER:
+        if s[key]["on"]:
+            x = np.asarray(fn(x, rate, s[key]), dtype=np.float32)
+    if s["channels"]["on"] and channels and channels >= 2:
+        x = _channels(x, rate, s["channels"], channels)
+    return np.nan_to_num(x).astype(np.float32)

ackit/game.py

@@ -11,7 +11,6 @@ import sys
 
 APPID = 200260
 FOLDER_NAMES = ("Batman Arkham City GOTY", "Batman Arkham City")
-LANGS = ("INT", "ITA", "FRA", "DEU", "ESN", "ESM", "POL", "RUS", "JPN", "KOR", "POR")
 
 
 class GameError(Exception):
@@ -124,6 +123,14 @@ class Game:
     def languages(self):
         return [a for a in self.archives if a != "SFX"]
 
+    def voice_archive(self, lang):
+        """The speech archive used for a language: its own, or INT for the languages the game has no voices for
+        (the game plays the INT voices with their own subtitles and menu text)."""
+        lang = (lang or "INT").upper()
+        if lang in self.languages:
+            return lang
+        return "INT" if "INT" in self.languages else self.languages[0]
+
     def wad(self, archive):
         try:
             return self.archives[archive.upper()]
@@ -154,8 +161,16 @@ class Game:
             dirs.append(os.path.join(pfx, docs, "WB Games", "Batman Arkham City GOTY", "BmGame", "Config"))
         return [d for d in dirs if os.path.isdir(d)]
 
+    def all_languages(self):
+        """Every language of the game: the voice languages (speech WADs) and the text languages (folders of
+        BmGame/Localization; the game plays the INT voices for those without their own)."""
+        loc = _iglob_one(self.root, "BmGame", "Localization")
+        texts = [d.upper() for d in os.listdir(loc) if os.path.isdir(os.path.join(loc, d))] if loc else []
+        return sorted(set(self.languages) | set(texts))
+
     def language(self):
-        """Speech language the game is set to (falls back to INT, then the first available)."""
+        """The language the game is set to (voices or texts only, e.g. JPN; `voice_archive` gives its speech
+        archive); falls back to INT, then the first voice language."""
         for d in self.config_dirs():
             for f in ("UserEngine.ini", "BmEngine.ini"):
                 p = os.path.join(d, f)
@@ -164,7 +179,7 @@ class Game:
                 raw = open(p, "rb").read()
                 txt = raw.decode("utf-16") if raw[:2] in (b"\xff\xfe", b"\xfe\xff") else raw.decode("latin-1")
                 m = re.search(r"^\s*Language\s*=\s*(\w+)", txt, re.M | re.I)
-                if m and m.group(1).upper() in self.archives:
+                if m and m.group(1).upper() in self.all_languages():
                     return m.group(1).upper()
         if "INT" in self.archives:
             return "INT"

ackit/lines.py

@@ -0,0 +1,224 @@
+"""Speech lines across languages, derived only from the game's own data (docs/formats/banks.md, "Lines").
+
+A voice recording exists once per language. The game ties them together itself:
+  - every recording of a speech WAD has exactly one row in its language's BKPK table (media ID, offset, size);
+    the table's language is the one its localized AkBank export declares (banks.py: localized_exports);
+  - every recording, streamed, prefetched or in memory, is the source of a Wwise Sound object. Sound objects
+    (and the events, actions and containers above them) have the same IDs in every language; only the media
+    differ (a few media IDs are shared by several languages, each at its own position in each WAD).
+So a *line* is the set of recordings whose media are the sources of the same Sound objects: one recording per
+language. Its identity (`Sound.line`) is the smallest Sound object ID (8 hex digits), its name is chosen once
+for all languages (`line_names`). A media stored in two entries of one WAD is one recording with a copy
+(`copy_of`), written together. Nothing here depends on file positions or on names.
+"""
+import collections
+from dataclasses import dataclass, field
+
+from . import banks
+from .wad import Wad
+
+
+@dataclass
+class Lines:
+    langs: list                                        # voice languages (speech WADs)
+    owners: dict = field(default_factory=dict)         # lang -> {media id: sound key} (WAD line or BANK sound)
+    line_of: dict = field(default_factory=dict)        # sound key -> line id
+    members: dict = field(default_factory=dict)        # line id -> {lang: sound key}
+    objects: dict = field(default_factory=dict)        # line id -> sorted Sound object ids
+    obj_keys: dict = field(default_factory=dict)       # Sound object id -> {lang: sound key}
+    names: dict = field(default_factory=dict)          # line id -> name (catalog.line_names)
+    unnamed: set = field(default_factory=set)          # line ids no named event plays (named after a recording)
+    copy_of: dict = field(default_factory=dict)        # WAD key -> key of the same media stored in another entry
+    issues: list = field(default_factory=list)         # (kind, detail): broken one-to-one relations
+    stats: dict = field(default_factory=dict)
+    _keys: dict = field(default_factory=dict, repr=False)  # lang -> set of sound keys
+
+    def key_langs(self, key):
+        return [L for L in self.langs if key in self._keys.get(L, ())]
+
+    def head_key(self, loc):
+        """Speech line (sound key) of a prefetch-head copy: the line of the copy's bank language that has
+        this media. A head in a language-neutral bank belongs to the one line that uses the media."""
+        if loc.lang:
+            return self.owners.get(loc.lang, {}).get(loc.media, "")
+        ks = {d[loc.media] for d in self.owners.values() if loc.media in d}
+        return ks.pop() if len(ks) == 1 else ""
+
+
+def wad_owners(game, sc, langs, walk):
+    """({lang: {media id: WAD sound key}}, {copy key: key}, issues, WEM counts) through each language's own BKPK
+    tables. Issues: a WEM without a table row, a row without a WEM. A media stored in two entries of one
+    language's WAD (a few lines, byte-identical) has one key; the other place is a copy written with it.
+    `walk(f, offset, size)` -> [(offset, length, header)] (catalog._walk_entry)."""
+    owners, issues, wems = {}, [], collections.Counter()
+    places = {}
+    for L in langs:
+        owners[L] = {}
+        places[L] = collections.defaultdict(list)
+        wad = Wad(game.wad(L))
+        with open(game.wad(L), "rb") as f:
+            for e in wad.entries:
+                tables = sc.packs.get(e.hash, {})
+                table = tables.get(L) if L in tables else tables.get("")
+                ws = walk(f, e.offset, e.size)
+                wems[L] += len(ws)
+                if table is None:
+                    issues.append(("no table", "%s:%08X: %d WEMs, no BKPK table" % (L, e.hash, len(ws))))
+                    continue
+                at = {(o, ln): i for i, (o, ln, _) in enumerate(ws)}
+                used = set()
+                for m, v in table.items():
+                    if v not in at:
+                        issues.append(("row without WEM", "%s:%08X: media %08X" % (L, e.hash, m)))
+                        continue
+                    used.add(at[v])
+                    places[L][m].append("%s:%08X" % (L, e.hash) + (":%d" % at[v] if len(ws) > 1 else ""))
+                for i in sorted(set(range(len(ws))) - used):
+                    issues.append(("WEM without row", "%s:%08X:%d" % (L, e.hash, i)))
+    copy_of = {}
+    for L in langs:
+        for m, ks in places[L].items():
+            ks = sorted(ks)
+            owners[L][m] = ks[0]
+            for k in ks[1:]:
+                copy_of[k] = ks[0]
+    return owners, copy_of, issues, dict(wems)
+
+
+def bank_langs(sc, m):
+    """Voice languages of a localized in-memory media: the languages of the banks holding it ('' = neutral)."""
+    ls = sc.media[m].get("langs", set())
+    return () if "" in ls else tuple(sorted(ls))
+
+
+def build(game, sc, walk):
+    """Lines of every voice language of the game (see module doc)."""
+    in_banks = sorted(set().union(*(e.get("langs", set()) for e in sc.media.values())) - {""} - set(game.languages))
+    langs = list(game.languages) + in_banks            # voice languages: speech WADs + localized banks
+    lines = Lines(langs)
+    owners, copy_of, issues, wems = wad_owners(game, sc, game.languages, walk)
+    for L in in_banks:
+        owners[L] = {}
+    lines.issues += issues
+    lines.copy_of = copy_of
+    for m, e in sc.media.items():                      # localized in-memory recordings (not prefetch heads)
+        if banks.PREFETCH_HEAD in e["kinds"]:
+            continue
+        for L in bank_langs(sc, m):
+            if L in owners:
+                owners[L].setdefault(m, "%s:%08X" % (banks.ARCHIVE, m))
+    lines.owners = owners
+    lines._keys = {L: set(d.values()) for L, d in owners.items()}
+
+    # Sound object -> its recording in each language (exactly one, or the relation is broken)
+    for o, ms in sc.sound_media.items():
+        per = {}
+        for L in langs:
+            ks = {owners[L][m] for m in ms if m in owners[L]}
+            if len(ks) > 1:
+                lines.issues.append(("object", "Sound %08X has %d recordings in %s: %s" % (o, len(ks), L, sorted(ks))))
+            elif ks:
+                per[L] = ks.pop()
+        if per:
+            lines.obj_keys[o] = per
+
+    # lines = connected recordings (union-find over the objects' recordings)
+    parent = {}
+
+    def find(x):
+        while parent.get(x, x) != x:
+            parent[x] = parent.get(parent[x], parent[x])
+            x = parent[x]
+        return x
+
+    for o, per in lines.obj_keys.items():
+        ks = sorted(set(per.values()))
+        for k in ks[1:]:
+            a, b = find(ks[0]), find(k)
+            if a != b:
+                parent[max(a, b)] = min(a, b)
+    groups = collections.defaultdict(set)
+    objs = collections.defaultdict(set)
+    for o, per in lines.obj_keys.items():
+        for k in per.values():
+            groups[find(k)].add(k)
+            objs[find(k)].add(o)
+    for root, ks in groups.items():
+        per = {}
+        ok = True
+        for k in ks:
+            for L in lines.key_langs(k):
+                if L in per and per[L] != k:
+                    ok = False
+                per.setdefault(L, k)
+        if not ok:
+            lines.issues.append(("line", "recordings %s share Sound objects but several are in one language" % sorted(ks)))
+            continue
+        lid = "%08X" % min(objs[root])
+        lines.members[lid] = per
+        lines.objects[lid] = sorted(objs[root])
+        for k in ks:
+            lines.line_of[k] = lid
+    for k, primary in copy_of.items():                 # a second place of the same recording: the same line
+        if primary in lines.line_of:
+            lines.line_of[k] = lines.line_of[primary]
+    recordings = set().union(*lines._keys.values()) if lines._keys else set()
+    all_keys = recordings | set(copy_of)               # + the second places of recordings stored twice
+    full = sum(1 for per in lines.members.values() if len(per) == len(langs))
+    lines.stats = {"languages": langs, "wad_wems": wems, "recordings": len(recordings),
+                   "in_a_line": len(lines.line_of), "lines": len(lines.members), "lines_in_every_language": full,
+                   "copies": sorted(copy_of),
+                   "lines_missing_a_language": {lid: sorted(set(langs) - set(per))
+                                                for lid, per in lines.members.items() if len(per) < len(langs)},
+                   "without_line": sorted(all_keys - set(lines.line_of))}
+    return lines
+
+
+def line_names(sc, lines, fallback):
+    """{line id: name}, the same in every language. A line is named after the game event that plays it: for
+    WAD lines the most specific `Speech_` event (fewest lines), for in-memory lines the most specific event
+    (`Play_` first), as the in-memory sounds always were. When an event plays several lines they are numbered
+    in the event's own order (the same in every language): NAME, NAME#2, ... (WAD) or NAME_1, NAME_2, ... (in
+    memory). A line no named event plays keeps the name its English recording has (`fallback(key)`: the
+    position-based name in the INT WAD, or the bank-numbered name), else the first voice language's."""
+    obj_line = {o: lid for lid, os_ in lines.objects.items() for o in os_}
+    is_wad = {lid: any(not k.startswith(banks.ARCHIVE + ":") for k in per.values()) for lid, per in lines.members.items()}
+    best = {}
+    reach = {}
+    for name, objs in banks.event_objects(sc).items():
+        order = []
+        for o in objs:
+            lid = obj_line.get(o)
+            if lid and lid not in order:
+                order.append(lid)
+        if order:
+            reach[name] = order
+    for name, order in reach.items():
+        low = name.lower()
+        for lid in order:
+            if is_wad[lid]:
+                if not low.startswith("speech_"):
+                    continue
+                rank = (len(order), low)
+            else:
+                rank = (0 if low.startswith("play") else 1, len(order), low)
+            if lid not in best or rank < best[lid][0]:
+                best[lid] = (rank, name)
+    named = collections.defaultdict(list)                     # event -> lines it names, in the event's order
+    for name, order in reach.items():
+        for lid in order:
+            if lid in best and best[lid][1] == name:
+                named[name].append(lid)
+    out = {}
+    for name, lids in named.items():
+        for i, lid in enumerate(lids):
+            if is_wad[lid]:
+                out[lid] = name if i == 0 else "%s#%d" % (name, i + 1)
+            else:
+                out[lid] = name if len(lids) == 1 else "%s_%d" % (name, i + 1)
+    lines.unnamed = set(lines.members) - set(out)
+    for lid in lines.unnamed:
+        per = lines.members[lid]
+        k = per.get("INT") or per[next(L for L in lines.langs if L in per)]
+        out[lid] = fallback(k)
+    return out

ackit/loctext.py

@@ -0,0 +1,122 @@
+"""Menu and UI text: the game's localization files BmGame/Localization/<LANG>/<File>.<lang>
+(UE3 .ini format; docs/formats/text.md).
+
+Files are UTF-16LE with a BOM and CRLF line ends (two small ones are plain ASCII). Lines are `[Section]`,
+`Key=Value` (the value may be in double quotes) or comments (`;`). A key can repeat inside a section, so an
+entry is identified by (file, section, key, occurrence). Writing changes only the value part of the edited
+lines; every other byte of the file (BOM, encoding, quotes, line ends, comments) stays as it was.
+"""
+import os
+import re
+from collections import namedtuple
+
+# key: "<LANG>:<File>:<Section>:<Key>" + ("#n" for the n-th repeat, n >= 2)
+Entry = namedtuple("Entry", "key lang file section name occurrence value quoted line")
+
+
+class TextError(ValueError):
+    pass
+
+
+def _decode(raw):
+    if raw[:2] == b"\xff\xfe":
+        return raw[2:].decode("utf-16-le"), "utf-16"
+    if raw[:3] == b"\xef\xbb\xbf":
+        return raw[3:].decode("utf-8"), "utf-8"
+    return raw.decode("cp1252"), "cp1252"
+
+
+def _encode(text, enc):
+    if enc == "utf-16":
+        return b"\xff\xfe" + text.encode("utf-16-le")
+    if enc == "utf-8":
+        return b"\xef\xbb\xbf" + text.encode("utf-8")
+    return text.encode("cp1252")
+
+
+_KV = re.compile(r"^([^=;\[\s][^=]*?)=(.*)$")
+
+
+def parse(raw, lang, fname):
+    """Entries of one file's bytes."""
+    text, _ = _decode(raw)
+    out = []
+    section = ""
+    seen = {}
+    for i, line in enumerate(re.split(r"(?<=\n)", text)):
+        body = line.rstrip("\r\n")
+        s = body.strip()
+        if s.startswith("[") and s.endswith("]"):
+            section = s[1:-1]
+            continue
+        m = _KV.match(body)
+        if not m or s.startswith(";"):
+            continue
+        name, value = m.group(1), m.group(2)
+        quoted = len(value) >= 2 and value[0] == '"' and value[-1] == '"'
+        if quoted:
+            value = value[1:-1]
+        k = (section.lower(), name.lower())
+        seen[k] = seen.get(k, 0) + 1
+        n = seen[k]
+        key = "%s:%s:%s:%s" % (lang, fname, section, name) + ("#%d" % n if n > 1 else "")
+        out.append(Entry(key, lang, fname, section, name, n, value, quoted, i))
+    return out
+
+
+def encoding(raw):
+    """Encoding of a file's bytes: 'utf-16', 'utf-8' or 'cp1252'."""
+    return "utf-16" if raw[:2] == b"\xff\xfe" else "utf-8" if raw[:3] == b"\xef\xbb\xbf" else "cp1252"
+
+
+def check_value(value, enc, fname):
+    """Raise TextError if `value` cannot be written into a file of encoding `enc` (what `write` refuses)."""
+    if "\r" in value or "\n" in value:
+        raise TextError("a menu text cannot contain line breaks (use the game's own markers, e.g. \\n)")
+    try:
+        _encode(value, enc)
+    except UnicodeEncodeError:
+        raise TextError("%s is stored as %s and cannot hold these characters" % (fname, enc)) from None
+
+
+def write(raw, lang, fname, values):
+    """New file bytes with {entry key: new value} applied (keys not in the file raise TextError)."""
+    text, enc = _decode(raw)
+    lines = re.split(r"(?<=\n)", text)
+    by_key = {e.key: e for e in parse(raw, lang, fname)}
+    for key, value in values.items():
+        e = by_key.get(key)
+        if e is None:
+            raise TextError("no text %s in %s" % (key, fname))
+        check_value(value, enc, fname)
+        line = lines[e.line]
+        end = line[len(line.rstrip("\r\n")):]
+        head = line[:line.index("=") + 1]
+        lines[e.line] = head + ('"%s"' % value if e.quoted else value) + end
+    out = "".join(lines)
+    try:
+        return _encode(out, enc)
+    except UnicodeEncodeError:
+        raise TextError("%s is stored as %s and cannot hold these characters" % (fname, enc)) from None
+
+
+def files(loc_root, lang):
+    """{file name: path} of one language folder (case-insensitive): localization files, without the
+    Version*.int build IDs (not UI text)."""
+    d = next((os.path.join(loc_root, x) for x in os.listdir(loc_root) if x.upper() == lang.upper()), None)
+    if not d:
+        return {}
+    return {f: os.path.join(d, f) for f in sorted(os.listdir(d))
+            if os.path.splitext(f)[1].lower() == "." + lang.lower() and not f.lower().startswith("version")}
+
+
+def languages(loc_root):
+    return sorted(x.upper() for x in os.listdir(loc_root) if os.path.isdir(os.path.join(loc_root, x)))
+
+
+def load(loc_root, lang):
+    """All entries of a language: [Entry]."""
+    out = []
+    for f, p in files(loc_root, lang).items():
+        out += parse(open(p, "rb").read(), lang.upper(), f)
+    return out

ackit/lzo.py

@@ -0,0 +1,142 @@
+"""LZO1X, as used by the game's compressed UE3 packages (docs/formats/upk.md).
+
+`decompress` is a pure-Python LZO1X decoder (same semantics as lzo1x_decompress_safe; the packages
+use 128 KiB blocks, so speed is fine for block-level access). `compress_stored` writes a valid LZO1X
+stream that holds the data as one literal run (no matches): bigger than real compression, but any
+LZO1X decoder (the game's included) reads it back exactly, and nothing else has to change.
+"""
+M2_MAX_OFFSET = 0x0800
+
+
+class LzoError(ValueError):
+    pass
+
+
+def _copy_match(out, dist, n):
+    start = len(out) - dist
+    if start < 0:
+        raise LzoError("match distance beyond output start")
+    if dist >= n:
+        out += out[start:start + n]
+    else:  # overlapping: repeat the last `dist` bytes
+        pat = bytes(out[start:])
+        out += (pat * (n // dist + 1))[:n]
+
+
+def decompress(src, out_len=None):
+    """LZO1X block -> bytes. `out_len` (if given) is checked."""
+    src = bytes(src)
+    n = len(src)
+    out = bytearray()
+    ip = 0
+
+    def ext(t, base):
+        nonlocal ip
+        while src[ip] == 0:
+            t += 255
+            ip += 1
+        t += base + src[ip]
+        ip += 1
+        return t
+
+    try:
+        state = "loop"
+        if src[0] > 17:
+            t = src[0] - 17
+            ip = 1
+            if t < 4:
+                state = "match_next"
+            else:
+                out += src[ip:ip + t]
+                ip += t
+                state = "first_literal_run"
+        while True:
+            if state == "loop":
+                t = src[ip]
+                ip += 1
+                if t >= 16:
+                    state = "match"
+                    continue
+                if t == 0:
+                    t = ext(0, 15)
+                out += src[ip:ip + t + 3]
+                ip += t + 3
+                state = "first_literal_run"
+                continue
+            if state == "first_literal_run":
+                t = src[ip]
+                ip += 1
+                if t >= 16:
+                    state = "match"
+                    continue
+                dist = 1 + M2_MAX_OFFSET + (t >> 2) + (src[ip] << 2)
+                ip += 1
+                _copy_match(out, dist, 3)
+                state = "match_done"
+                continue
+            if state == "match_next":
+                out += src[ip:ip + t]
+                ip += t
+                t = src[ip]
+                ip += 1
+                state = "match"
+                continue
+            if state == "match":
+                if t >= 64:                       # M2
+                    dist = 1 + ((t >> 2) & 7) + (src[ip] << 3)
+                    ip += 1
+                    _copy_match(out, dist, (t >> 5) + 1)
+                elif t >= 32:                     # M3
+                    t &= 31
+                    if t == 0:
+                        t = ext(0, 31)
+                    dist = 1 + ((src[ip] | (src[ip + 1] << 8)) >> 2)
+                    ip += 2
+                    _copy_match(out, dist, t + 2)
+                elif t >= 16:                     # M4 (distance 0 = end of stream)
+                    hi = (t & 8) << 11
+                    t &= 7
+                    if t == 0:
+                        t = ext(0, 7)
+                    dist = hi + ((src[ip] | (src[ip + 1] << 8)) >> 2)
+                    ip += 2
+                    if dist == 0:
+                        break
+                    _copy_match(out, dist + 0x4000, t + 2)
+                else:                             # M1 after a match: 2 bytes
+                    dist = 1 + (t >> 2) + (src[ip] << 2)
+                    ip += 1
+                    _copy_match(out, dist, 2)
+                state = "match_done"
+                continue
+            # match_done: the low 2 bits of the instruction's last byte = trailing literals
+            t = src[ip - 2] & 3
+            if t == 0:
+                state = "loop"
+            else:
+                state = "match_next"
+    except IndexError:
+        raise LzoError("truncated LZO stream") from None
+    if ip != n:
+        raise LzoError("%d bytes after the end of the LZO stream" % (n - ip))
+    if out_len is not None and len(out) != out_len:
+        raise LzoError("LZO block decoded to %d bytes, expected %d" % (len(out), out_len))
+    return bytes(out)
+
+
+EOF = b"\x11\x00\x00"
+
+
+def compress_stored(data):
+    """bytes -> LZO1X stream made of one literal run (decodes back to `data`)."""
+    data = bytes(data)
+    n = len(data)
+    if n == 0:
+        return EOF
+    if n <= 238:
+        return bytes([n + 17]) + data + EOF
+    # long literal run: 0x00, k zero bytes (255 each), last byte; length = 18 + 255*k + last
+    r = n - 18
+    k = (r - 1) // 255
+    last = r - 255 * k
+    return b"\x00" + b"\x00" * k + bytes([last]) + data + EOF

ackit/modpack.py

@@ -5,7 +5,10 @@ A package is a zip:
   items/NNN.wem      the exact WEM that was applied (used as-is when the target matches)
   items/NNN.flac     the same audio, decoded (used to re-encode when the target differs, e.g. another
                      language whose original has a different size)
-Each item records archive, target, name, group and the SHA-1 of the original it replaced.
+Each item records archive, target, name, group, the SHA-1 of the original it replaced and, for a voice line,
+its line identity (the same in every language, ackit/lines.py): that is how another language finds it.
+  "texts": [{"key", "text", "original"}]   edited subtitles / menu texts (keys: ackit/texts.py); optional,
+                                           so packages without texts stay valid
 """
 import io
 import json
@@ -22,15 +25,23 @@ from .originals import sha1_16
 FORMAT = 1
 
 
-def export_mod(ws, path, keys, name="", author="", description=""):
+def export_mod(ws, path, keys, name="", author="", description="", texts=()):
-    """Write the applied versions of `keys` (from the originals store) into a package."""
+    """Write the applied versions of sound `keys` and text keys `texts` (from the originals store) into a
+    package. Returns the number of entries."""
     items = []
+    recs = ws.texts.records()
+    text_items = []
+    for k in texts:
+        r = recs.get(k)
+        if not r or r.get("applied") is None:
+            raise ValueError("%s is not modified" % k)
+        text_items.append({"key": k, "text": r["applied"], "original": r["orig"]})
     with zipfile.ZipFile(path + ".tmp", "w", zipfile.ZIP_DEFLATED) as z:
         for i, key in enumerate(keys):
             e = ws.store.get(key)
             if not e or not e.get("applied"):
                 raise ValueError("%s is not modified" % key)
-            s = ws.catalog.by_key.get(key)
+            s = ws.sound_for_key(key)               # also a change made in another language (its name and line)
             wem = ws.store.get_blob(e["applied"])
             x, sr = audio.decode_wem(wem)
             buf = io.BytesIO()
@@ -39,20 +50,22 @@ def export_mod(ws, path, keys, name="", author="", description=""):
             z.writestr(base + ".wem", wem)
             z.writestr(base + ".flac", buf.getvalue())
             items.append({"archive": e["archive"], "target": e["target"], "name": s.name if s else "",
+                          "line": s.line if s else "",
                           "group": s.group if s else "", "original_sha1": e["orig_sha1"],
                           "wem": base + ".wem", "audio": base + ".flac", "meta": e.get("meta", {})})
         z.writestr("manifest.json", json.dumps({"format": FORMAT, "name": name, "author": author,
                                                 "description": description, "created": time.time(),
-                                                "game": "Batman: Arkham City (PC)", "items": items}, indent=1))
+                                                "game": "Batman: Arkham City (PC)", "items": items,
+                                                "texts": text_items}, indent=1, ensure_ascii=False))
     os.replace(path + ".tmp", path)
-    return len(items)
+    return len(items) + len(text_items)
 
 
 @dataclass
 class ImportItem:
     item: dict
     sound: object = None      # matched Sound or None
-    how: str = ""             # 'exact' | 'by name' | 'missing'
+    how: str = ""             # 'exact' | 'same line' | 'by name' | 'missing'
     wem: bytes = None         # ready-to-apply bytes (after `prepare_import`)
     note: str = ""
 
@@ -69,19 +82,34 @@ def match_mod(ws, path):
     """Match package items to sounds of this workspace. -> (manifest, [ImportItem])"""
     z, man = read_mod(path)
     out = []
+    by_line = {}
+    for c in ws.catalog.sounds:
+        if c.line:
+            by_line.setdefault(c.line, []).append(c)
     for it in man["items"]:
         key = "%s:%s" % (it["archive"], it["target"])
         s = ws.catalog.by_key.get(key)
-        if s and it.get("name") and s.name != it["name"]:
+        # the very same recording: same position and the same name or the same original bytes
-            s = None
+        if s and (s.name == it.get("name") or (s.sha1 and s.sha1 == it.get("original_sha1"))):
-        if s:
             out.append(ImportItem(it, s, "exact"))
             continue
+        if it.get("line"):                    # a voice line: its recording in the language loaded here
+            cands = sorted(by_line.get(it["line"], []), key=lambda c: (c.name != it.get("name"), c.key))
+            if cands:                         # several only for one recording stored twice (written together)
+                out.append(ImportItem(it, cands[0], "same line"))
+            else:
+                out.append(ImportItem(it, None, "missing", note="the line %r has no recording in the language "
+                                      "loaded here" % it.get("name")))
+            continue
         try:
             cands = [c for c in ws.catalog.search(it["name"]) if c.name == it["name"]] if it.get("name") else []
         except LookupError:
             cands = []
-        if len(cands) == 1:
+        if len(cands) == 1 and cands[0].line:
+            # a voice line from a package without line identities (older version): names alone are not proof
+            out.append(ImportItem(it, None, "missing", note="%r is a voice line from a package of an older "
+                                  "version; export the mod again to move it to another language" % it["name"]))
+        elif len(cands) == 1:
             out.append(ImportItem(it, cands[0], "by name"))
         else:
             out.append(ImportItem(it, None, "missing", note="no sound named %r in this game/language" % it.get("name")))
@@ -89,6 +117,20 @@ def match_mod(ws, path):
     return man, out
 
 
+def match_texts(ws, man):
+    """[(key, text, problem or '')] for the package's texts in this game (problem: why it cannot be applied)."""
+    out = []
+    for it in man.get("texts", []):
+        k, text = it["key"], it["text"]
+        try:
+            ws.texts.current(k)
+            ws.texts.check(k, text)
+            out.append((k, text, ""))
+        except Exception as e:  # missing key, too long, characters the file cannot hold
+            out.append((k, text, str(e)))
+    return out
+
+
 def prepare_import(ws, path, items, progress=None):
     """Fill `wem` for matched items: reuse the packaged WEM when it targets the very same original,
     otherwise re-encode the packaged audio for the local sound."""
@@ -98,8 +140,7 @@ def prepare_import(ws, path, items, progress=None):
             continue
         orig = ws.original_bytes(im.sound)
         wem = z.read(im.item["wem"])
-        lim = ws.limit(im.sound)
+        if im.how == "exact" and sha1_16(orig) == im.item.get("original_sha1") and ws.fits(im.sound, wem):
-        if im.how == "exact" and sha1_16(orig) == im.item.get("original_sha1") and (lim is None or len(wem) <= lim):
             im.wem, im.note = wem, "exact copy"
         else:
             x, sr = sf.read(io.BytesIO(z.read(im.item["audio"])), dtype="float32", always_2d=True)
@@ -110,11 +151,15 @@ def prepare_import(ws, path, items, progress=None):
 
 
 def import_mod(ws, path, force=False, progress=None):
-    """Match, prepare and apply a package. Returns the list of ImportItem (see .how/.note)."""
+    """Match, prepare and apply a package. Returns ([ImportItem] (see .how/.note), [(key, text, problem)])."""
-    _, items = match_mod(ws, path)
+    man, items = match_mod(ws, path)
     prepare_import(ws, path, items, progress)
-    changes = [(im.sound, im.wem, {"source": "mod:" + path.replace("\\", "/").split("/")[-1]})
+    source = {"source": "mod:" + path.replace("\\", "/").split("/")[-1]}
-               for im in items if im.wem]
+    changes = [(im.sound, im.wem, source) for im in items if im.wem]
     if changes:
         ws.apply(changes, force=force, progress=progress)
-    return items
+    texts = match_texts(ws, man)
+    ok = {k: v for k, v, problem in texts if not problem}
+    if ok:
+        ws.texts.apply(ok, force=force, progress=progress, meta=source)
+    return items, texts

ackit/paths.py

@@ -16,11 +16,6 @@ def resource_root():
     return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 
 
-def package_data(*parts):
-    """Files shipped inside the package (codebooks, known setups, hash list)."""
-    return os.path.join(os.path.dirname(os.path.abspath(__file__)), "data", *parts)
-
-
 def catalog_dir():
     return os.path.join(resource_root(), "data", "catalog")
 

ackit/settings.py

@@ -4,10 +4,10 @@ import os
 
 from .paths import user_dir
 
-DEFAULTS = {"game_dir": None, "store_root": None, "full_backup": True, "language": None,
+DEFAULTS = {"game_dir": None, "store_root": None, "full_backup": True,
             "input_device": None, "output_device": None, "last_dir": None,
             "background_enabled": True, "background_opacity": 0.1}
-OBSOLETE = ("theme", "background_image")
+OBSOLETE = ("theme", "background_image", "language")   # the editor starts in the game's language
 
 
 def path():

ackit/subtitles.py

@@ -0,0 +1,356 @@
+"""Subtitles: the RDialogueEvent objects inside the game's packages (docs/formats/subtitles.md).
+
+Each dialogue object references its speech event (an export named like the speech line, e.g.
+'Speech_CHURCH_B1_Ch2_POST_Explosion_BM_Cash_04') and holds the subtitle text of every language:
+  LocalizedSubtitles: array of {Language: FString, Subtitles: array of {Text: FString, (float @0x0c)}}
+in Rocksteady's compact tagged-property format (u16 type, u16 field offset, value; structs and arrays with
+a byte size; struct elements end with a u16 0).
+
+A dialogue object can be stored in several packages; a subtitle edit is written into every copy, in place:
+the new text replaces the bytes of the old FString and is padded with NUL characters (the game stops at the
+first NUL), so no size field, export or package layout changes. The text must therefore fit the original
+string's bytes (cp1252 strings: one byte per character; UTF-16 strings: two).
+"""
+import csv
+import gzip
+import io
+import os
+import re
+import struct
+from collections import defaultdict, namedtuple
+
+
+CLASS = "RDialogueEvent"
+SUBTITLE_CLASS, LOOKUP_CLASS = "RSubtitle", "RSubtitleLookup"   # scripted radio/encryption subtitles
+UNRESOLVED = "#lookup:%08X"     # speech of an RSubtitle until its lookup key is matched to a line name
+T_STRUCT, T_STRING, T_OBJECT = 0x09, 0x0D, 0x10
+
+# one subtitle string inside a package: pos = absolute uncompressed offset of the FString length field,
+# size = bytes of the whole FString (length field included), enc 'a' (cp1252) or 'w' (UTF-16)
+Field = namedtuple("Field", "lang index text pos size enc")
+# speech: name of the referenced speech event; speaker: the object's speaker field (RDialogueEvent only)
+Dialogue = namedtuple("Dialogue", "speech export fields speaker", defaults=("",))
+
+
+class SubtitleError(ValueError):
+    pass
+
+
+# Windows-1252 as Windows itself maps it: the 5 bytes Python's cp1252 leaves undefined stand for the same code
+# points (U+0081 ...). One line is UTF-8 bytes in a cp1252 FString in 4 languages (Speech_TAPE_J_Answer_1_00,
+# docs/formats/subtitles.md); the RUS and KOR ones contain such bytes. Read and written back losslessly, as stored.
+_CP1252_UNDEFINED = frozenset((0x81, 0x8D, 0x8F, 0x90, 0x9D))
+
+
+def _decode_cp1252(raw):
+    try:
+        return raw.decode("cp1252")
+    except UnicodeDecodeError:
+        return "".join(chr(x) if x in _CP1252_UNDEFINED else bytes([x]).decode("cp1252") for x in raw)
+
+
+def _encode_cp1252(text):
+    """UnicodeEncodeError for a character that is not in Windows-1252."""
+    try:
+        return text.encode("cp1252")
+    except UnicodeEncodeError:
+        return b"".join(bytes([ord(c)]) if ord(c) in _CP1252_UNDEFINED else c.encode("cp1252") for c in text)
+
+
+def read_fstring(b, o):
+    """-> (text, end offset, enc)"""
+    n = struct.unpack_from("<i", b, o)[0]
+    if n >= 0:
+        raw = bytes(b[o + 4:o + 4 + n])
+        return _decode_cp1252(raw.split(b"\0", 1)[0]), o + 4 + n, "a"
+    raw = bytes(b[o + 4:o + 4 - 2 * n])
+    return raw.decode("utf-16-le", "replace").split("\0", 1)[0], o + 4 - 2 * n, "w"
+
+
+def encode_fstring(text, size, enc):
+    """The FString bytes (length field included) for `text`, exactly `size` bytes long (NUL padded).
+    Raises SubtitleError when the text does not fit."""
+    if "\0" in text:
+        raise SubtitleError("the text must not contain NUL characters")
+    if enc == "a":
+        try:
+            raw = _encode_cp1252(text)
+        except UnicodeEncodeError:
+            raise SubtitleError("this subtitle is stored as Western European (Windows-1252) text; '%s' cannot be "
+                                "written"
+                                % next(c for c in text if not _cp1252(c))) from None
+        n = size - 4
+        if len(raw) + 1 > n:
+            raise SubtitleError("the text is %d characters long but this subtitle has room for %d"
+                                % (len(raw), n - 1))
+        return struct.pack("<i", n) + raw + bytes(n - len(raw))
+    raw = text.encode("utf-16-le")
+    n = (size - 4) // 2
+    if len(raw) // 2 + 1 > n:
+        raise SubtitleError("the text is %d characters long but this subtitle has room for %d" % (len(raw) // 2, n - 1))
+    return struct.pack("<i", -n) + raw + bytes(2 * n - len(raw))
+
+
+def _cp1252(c):
+    try:
+        _encode_cp1252(c)
+        return True
+    except UnicodeEncodeError:
+        return False
+
+
+def room(field):
+    """Maximum number of characters for a subtitle string."""
+    return (field.size - 4 - 1) if field.enc == "a" else (field.size - 4) // 2 - 1
+
+
+def _parse_lines(b, o, end, base, lang):
+    """Subtitles array content [o, end) -> [Field]."""
+    count = struct.unpack_from("<I", b, o)[0]
+    o += 4
+    out = []
+    for i in range(count):
+        # element: props until u16 0; the text is the string at field offset 0
+        while True:
+            t, fo = struct.unpack_from("<HH", b, o)
+            if t == 0:
+                o += 2
+                break
+            if t == T_STRING:
+                text, e, enc = read_fstring(b, o + 4)
+                if fo == 0:
+                    out.append(Field(lang, i, text, base + o + 4, e - (o + 4), enc))
+                o = e
+            elif t == 0x04:               # 4-byte value (the line's time)
+                o += 8
+            else:
+                raise SubtitleError("unexpected property type %#x in a subtitle line" % t)
+            if o > end:
+                raise SubtitleError("subtitle line overruns its array")
+    if o != end:
+        raise SubtitleError("subtitle array size mismatch")
+    return out
+
+
+def parse_dialogue(b, base, names_idx):
+    """Fields of one dialogue/subtitle export (bytes `b` starting at uncompressed offset `base`).
+    names_idx: (name indices of the per-language array: LocalizedSubtitles or ArraySubtitles, of Subtitles)."""
+    arrays, sub = names_idx
+    i = -1
+    for ls in arrays:
+        i = b.find(struct.pack("<II", ls, 0))
+        if i >= 4 and struct.unpack_from("<H", b, i - 4)[0] == T_STRUCT:
+            break
+    if i < 4 or struct.unpack_from("<H", b, i - 4)[0] != T_STRUCT:
+        return []
+    size = struct.unpack_from("<i", b, i + 8)[0]
+    o, end = i + 16, i + 16 + size
+    count = struct.unpack_from("<I", b, o)[0]
+    o += 4
+    fields = []
+    for _ in range(count):
+        lang = None
+        while True:
+            t, fo = struct.unpack_from("<HH", b, o)
+            if t == 0:
+                o += 2
+                break
+            if t == T_STRING:
+                lang, o, _ = read_fstring(b, o + 4)
+            elif t == T_STRUCT:
+                nm, _nn, ssize, _ai = struct.unpack_from("<iiii", b, o + 4)
+                c0 = o + 20
+                if nm == sub:
+                    fields += _parse_lines(b, c0, c0 + ssize, base, lang)
+                o = c0 + ssize
+            else:
+                raise SubtitleError("unexpected property type %#x in a localized subtitle" % t)
+    if o != end:
+        raise SubtitleError("localized subtitle size mismatch")
+    return fields
+
+
+def _speaker(b):
+    """The speaker FString of a dialogue object (property 0x0D at field offset 0x34), '' if none."""
+    m = re.search(rb"\x0d\x00\x34\x00", b)
+    try:
+        return read_fstring(b, m.start() + 4)[0] if m else ""
+    except (struct.error, UnicodeDecodeError, ValueError):
+        return ""
+
+
+def _speech_ref(b, exports, imports):
+    """Name of the object referenced by the dialogue's first object property (its speech event)."""
+    m = re.search(rb"\x10\x00\x2c\x00", b)
+    if not m:
+        return ""
+    ref = struct.unpack_from("<i", b, m.start() + 4)[0]
+    if ref > 0 and ref <= len(exports):
+        return exports[ref - 1].name
+    if ref < 0 and -ref <= len(imports):
+        return imports[-ref - 1][1]
+    return ""
+
+
+class Collector:
+    """`visit` hook for banks.scan: collects the dialogue objects of every package."""
+
+    def __init__(self):
+        self.dialogues = defaultdict(list)       # speech name -> [(file, Dialogue)]
+        self.scripted = set()                    # speech names of RSubtitle objects (resolve)
+        self.issues = []
+
+    def __call__(self, rel, pkg, start, data):
+        try:
+            for d in scan_package(pkg, data, start):
+                self.dialogues[d.speech].append((rel, d))
+        except (SubtitleError, struct.error, ValueError, IndexError) as e:
+            self.issues.append(("subtitles", "%s: %s" % (rel, e)))
+
+    PREFIXES = ("", "RADIO_SCAN_", "RADIO_", "TV_", "CH_ZERO_", "GCPD_", "CHAR_", "SEQUENCE_")
+
+    def resolve(self, line_names):
+        """Give RSubtitle objects their speech line name: the name whose lookup_key equals the object's key,
+        from `line_names` or built from the object's own name (Speech_ + prefix + name; exact hash match)."""
+        by_key = {lookup_key(n): n for n in line_names}
+        for sp in [s for s in self.dialogues if s.startswith("#lookup:")]:
+            key = int(sp.split(":")[1], 16)
+            name = by_key.get(key)
+            if not name:
+                exp = self.dialogues[sp][0][1].export
+                name = next(("Speech_" + p + exp for p in self.PREFIXES if lookup_key("Speech_" + p + exp) == key), None)
+            copies = self.dialogues.pop(sp)
+            if not name:
+                self.issues.append(("subtitles", "no speech line for subtitle %s" % copies[0][1].export))
+                name = copies[0][1].export
+            self.dialogues[name].extend((rel, d._replace(speech=name)) for rel, d in copies)
+            self.scripted.add(name)
+
+
+# --- catalog files: SUB_<LANG>.csv (texts) + SUB_locations.csv.gz (every copy) ------------------
+LOCATIONS = "SUB_locations.csv.gz"
+TEXT_FIELDS = ["speech", "line", "text", "room", "enc", "sound"]
+class SubLine(namedtuple("SubLine", "lang speech line text room enc sound")):
+    """A subtitle string. sound: the voice recording(s) its speech event plays in that language (sound keys,
+    ';'-separated in the event's order: an event can play one of two versions of a line)."""
+    __slots__ = ()
+
+    @property
+    def sounds(self):
+        return tuple(self.sound.split(";")) if self.sound else ()
+SubLoc = namedtuple("SubLoc", "lang speech line file pos size enc")
+
+
+def key(lang, speech, line):
+    return "SUB:%s:%s:%d" % (lang, speech, line)
+
+
+def rows(collector, links):
+    """({lang: [SubLine]}, [SubLoc]) from a Collector; links: {speech: {lang: sound key}}."""
+    texts = defaultdict(list)
+    locs = []
+    for speech in sorted(collector.dialogues):
+        copies = collector.dialogues[speech]
+        rel0, d0 = copies[0]
+        for f in d0.fields:
+            ln = links.get(speech, {})
+            snd = ln.get(f.lang) or ln.get("INT", "")
+            texts[f.lang].append(SubLine(f.lang, speech, f.index, f.text, room(f), f.enc, snd))
+        for rel, d in copies:
+            for f in d.fields:
+                locs.append(SubLoc(f.lang, speech, f.index, rel, f.pos, f.size, f.enc))
+    return dict(texts), locs
+
+
+def write_texts(path, lines):
+    with open(path + ".tmp", "w", encoding="utf-8", newline="") as fh:
+        w = csv.writer(fh, lineterminator="\n")
+        w.writerow(TEXT_FIELDS)
+        for s in lines:
+            w.writerow([s.speech, s.line, s.text, s.room, s.enc, s.sound])
+    os.replace(path + ".tmp", path)
+
+
+def read_texts(path, lang):
+    with open(path, encoding="utf-8", newline="") as fh:
+        return [SubLine(lang, r["speech"], int(r["line"]), r["text"], int(r["room"]), r["enc"], r["sound"])
+                for r in csv.DictReader(fh)]
+
+
+def write_locations(path, locs):
+    raw = io.StringIO()
+    w = csv.writer(raw, lineterminator="\n")
+    w.writerow(["lang", "speech", "line", "file", "pos", "size", "enc"])
+    for l in sorted(locs, key=lambda l: (l.lang, l.speech, l.line, l.file, l.pos)):
+        w.writerow([l.lang, l.speech, l.line, l.file, "%X" % l.pos, l.size, l.enc])
+    with open(path + ".tmp", "wb") as fh:
+        with gzip.GzipFile(fileobj=fh, mode="wb", mtime=0) as gz:
+            gz.write(raw.getvalue().encode("utf-8"))
+    os.replace(path + ".tmp", path)
+
+
+def read_locations(path, lang=None):
+    """{SUB key: [SubLoc]} (only `lang` if given)."""
+    out = defaultdict(list)
+    with gzip.open(path, "rt", encoding="utf-8", newline="") as fh:
+        for r in csv.DictReader(fh):
+            if lang and r["lang"] != lang:
+                continue
+            l = SubLoc(r["lang"], r["speech"], int(r["line"]), r["file"], int(r["pos"], 16), int(r["size"]), r["enc"])
+            out[key(l.lang, l.speech, l.line)].append(l)
+    return dict(out)
+
+
+def lookup_key(line_name):
+    """Key of a speech line in the RSubtitleLookup table: FNV-1 of its name without 'Speech_'."""
+    from .bnk import fnv1
+    return fnv1(line_name[7:] if line_name.lower().startswith("speech_") else line_name)
+
+
+def _lookup_table(b):
+    """RSubtitleLookup export data -> {export index: key}: array of {int key @0, object @4}."""
+    count = struct.unpack_from("<I", b, 24)[0]
+    out, o = {}, 28
+    for _ in range(count):
+        if b[o:o + 4] != b"\x02\x00\x00\x00" or b[o + 8:o + 12] != b"\x10\x00\x04\x00":
+            raise SubtitleError("unexpected subtitle lookup layout")
+        key, ref = struct.unpack_from("<Ii", b, o + 4)[0], struct.unpack_from("<i", b, o + 12)[0]
+        out[ref] = key
+        o += 18
+    return out
+
+
+def scan_package(pkg, data=None, start=None, tables=None):
+    """[Dialogue] of one package (empty if it has none): RDialogueEvent objects (speech = the referenced
+    speech event) and RSubtitle objects (speech = UNRESOLVED % lookup key, see Collector.resolve)."""
+    if data is None:
+        start, data = pkg.read_all()
+    names, imports, exports = tables or pkg.tables(data, start)
+    by_class = {}
+    for i, (c, o, _) in enumerate(imports):
+        if c == "Class" and o in (CLASS, SUBTITLE_CLASS, LOOKUP_CLASS):
+            by_class[-(i + 1)] = o
+    arrays = [names.index(a) for a in ("LocalizedSubtitles", "ArraySubtitles") if a in names]
+    if not by_class or not arrays or "Subtitles" not in names:
+        return []
+    idx = (arrays, names.index("Subtitles"))
+    keys = {}
+    for e in exports:
+        if by_class.get(e.cls) == LOOKUP_CLASS and e.size > 0:
+            keys.update(_lookup_table(data[e.offset - start:e.offset - start + e.size]))
+    out = []
+    for e in exports:
+        kind = by_class.get(e.cls)
+        if kind in (CLASS, SUBTITLE_CLASS) and e.size > 0:
+            b = data[e.offset - start:e.offset - start + e.size]
+            fields = parse_dialogue(b, e.offset, idx)
+            if not fields:
+                continue
+            if kind == CLASS:
+                speech = _speech_ref(b, exports, imports)
+            elif e.index in keys:
+                speech = UNRESOLVED % keys[e.index]
+            else:
+                speech = e.name                       # not in a lookup table: keep the object's name
+            out.append(Dialogue(speech, e.name, fields, _speaker(b) if kind == CLASS else ""))
+    return out

ackit/texts.py

@@ -0,0 +1,292 @@
+"""Subtitles and menu text: reading, states, apply and revert (shared by the CLI and the editor).
+
+Keys (also used in mod packages):
+  SUB:<LANG>:<speech event>:<line>                  a subtitle string (docs/formats/subtitles.md)
+  MENU:<LANG>:<File>:<Section>:<Key>[#n]            a menu/UI text (docs/formats/text.md)
+Records live in the originals store (data["texts"]): {key: {"orig", "applied", "files", ...}}. The files
+holding the texts are derived files of the Workspace: each is rebuilt from its original copy plus every edit
+applied now (sounds and texts), so reverting the last edit restores it byte-for-byte. Writes hold the
+workspace's lock (Workspace.lock).
+"""
+import os
+import time
+
+from . import banks, loctext, subtitles
+from .catalog import _shipped_index, catalog_dir, load_subtitles
+from .game import GameError
+
+ORIGINAL, MODIFIED, LOST, CHANGED, FOREIGN = "original", "modified", "lost", "changed", "foreign"
+
+
+def _locked(f):
+    """Run a Texts method holding the workspace's write lock."""
+    def run(self, *a, **kw):
+        with self.ws.lock:
+            return f(self, *a, **kw)
+    run.__name__, run.__doc__ = f.__name__, f.__doc__
+    return run
+
+
+class TextConflictError(GameError):
+    def __init__(self, keys):
+        self.keys = keys
+        super().__init__("These texts were changed outside this tool (Steam update or 'verify', or another mod "
+                         "tool): %s.\nRevert them or confirm to overwrite."
+                         % (", ".join(keys[:5]) + (" ..." if len(keys) > 5 else "")))
+
+
+class Texts:
+    def __init__(self, ws):
+        self.ws = ws
+        self._subs = {}          # lang -> ([SubLine], {key: [SubLoc]})
+        self._sub_text = {}      # lang -> {(speech, line): original text} (catalog)
+        self._menu = {}          # lang -> [Entry]
+        self._menu_paths = {}    # lang -> {file name: path}
+        self._menu_file = {}     # path -> ((size, mtime), encoding, {entry key: Entry})
+        self._shipped_subs = None  # (by file: [SubLoc], {(lang, speech, line): text}) of the shipped catalog
+        self.loc_root = _find_dir(ws.game.root, "BmGame", "Localization")
+
+    # --- languages and lists ---------------------------------------------------------------------
+    def menu_languages(self):
+        return loctext.languages(self.loc_root) if self.loc_root else []
+
+    def subtitle_languages(self):
+        langs = sorted(_shipped_index().get("SUB", {}).get("lines", {}))
+        return langs or self.menu_languages()
+
+    def subtitles(self, lang, progress=None):
+        """[SubLine] of a language (original texts from the catalog; use `current` for the game's text)."""
+        if lang not in self._subs:
+            self._subs[lang] = load_subtitles(self.ws.game, lang, progress)
+        return self._subs[lang][0]
+
+    def subtitle_locations(self, key):
+        lang = key.split(":")[1]
+        self.subtitles(lang)
+        return self._subs[lang][1].get(key, [])
+
+    def for_sound(self, sound, lang=None):
+        """Subtitle lines of a speech sound, in `lang` (default: the sound's own language). Languages without
+        their own voices link their subtitles to the INT sounds, so e.g. JPN subtitles are found for INT lines.
+        A sound shared by every language (an in-memory line stored once) defaults to the game's language."""
+        game = self.ws.game
+        lang = lang or (sound.languages[0] if sound.languages else
+                        sound.archive if sound.archive in game.languages else game.language())
+        if lang not in self.subtitle_languages():
+            return []
+        return [s for s in self.subtitles(lang) if sound.key in s.sounds]
+
+    def menu(self, lang):
+        """[loctext.Entry] of the game's current files (the values the game shows now)."""
+        if lang not in self._menu:
+            self._menu[lang] = loctext.load(self.loc_root, lang) if self.loc_root else []
+        return self._menu[lang]
+
+    def is_menu_file(self, path):
+        return bool(self.loc_root) and os.path.normcase(os.path.abspath(path)).startswith(
+            os.path.normcase(os.path.abspath(self.loc_root)) + os.sep)
+
+    # --- values and states ----------------------------------------------------------------------
+    def records(self):
+        return self.ws.store.data.setdefault("texts", {})
+
+    def modified(self):
+        return [k for k, r in self.records().items() if r.get("applied") is not None]
+
+    def _menu_path(self, key):
+        """Path of the file holding menu text `key` (MENU:<LANG>:<File>:...)."""
+        parts = key.split(":", 3)
+        if len(parts) < 4 or not self.loc_root:
+            raise GameError("no menu text %s" % key[len("MENU:"):])
+        lang = parts[1]
+        if lang not in self._menu_paths:
+            self._menu_paths[lang] = loctext.files(self.loc_root, lang)
+        path = self._menu_paths[lang].get(parts[2])
+        if not path:
+            raise GameError("no menu text %s" % key[len("MENU:"):])
+        return path
+
+    def _menu_entries(self, path, lang):
+        """(encoding, {entry key: Entry}) of one menu file as it is now (parsed again only when it changed)."""
+        st = os.stat(path)
+        stamp = (st.st_size, st.st_mtime_ns)
+        hit = self._menu_file.get(path)
+        if not hit or hit[0] != stamp:
+            raw = open(path, "rb").read()
+            hit = (stamp, loctext.encoding(raw),
+                   {e.key: e for e in loctext.parse(raw, lang.upper(), os.path.basename(path))})
+            self._menu_file[path] = hit
+        return hit[1], hit[2]
+
+    def _menu_entry(self, key):
+        path = self._menu_path(key)
+        e = self._menu_entries(path, key.split(":")[1])[1].get(key[len("MENU:"):])
+        if e is None:
+            raise GameError("no menu text %s" % key[len("MENU:"):])
+        return e
+
+    def files(self, key):
+        """Absolute paths of the files holding a text."""
+        if key.startswith("SUB:"):
+            return sorted({self.ws.bank_path(l.file) for l in self.subtitle_locations(key)})
+        self._menu_entry(key)
+        return [self._menu_path(key)]
+
+    def current(self, key):
+        """The text in the game files now."""
+        if key.startswith("SUB:"):
+            locs = self.subtitle_locations(key)
+            if not locs:
+                raise GameError("no subtitle %s" % key)
+            l = locs[0]
+            b = banks.open_file(self.ws.bank_path(l.file)).read(l.pos, l.size)
+            return subtitles.read_fstring(b, 0)[0]
+        return self._menu_entry(key).value
+
+    def catalog_text(self, key):
+        """Original text according to the catalog (subtitles only), or None."""
+        if not key.startswith("SUB:"):
+            return None
+        _, lang, rest = key.split(":", 2)
+        speech, line = rest.rsplit(":", 1)
+        if lang not in self._sub_text:
+            self._sub_text[lang] = {(s.speech, s.line): s.text for s in self.subtitles(lang)}
+        return self._sub_text[lang].get((speech, int(line)))
+
+    def original(self, key):
+        r = self.records().get(key)
+        return r["orig"] if r else self.current(key)
+
+    def state(self, key, current=None):
+        cur = self.current(key) if current is None else current
+        r = self.records().get(key)
+        if r:
+            if r.get("applied") is not None and cur == r["applied"]:
+                return MODIFIED
+            if cur == r["orig"]:
+                return LOST if r.get("applied") is not None else ORIGINAL
+            return CHANGED
+        cat = self.catalog_text(key)
+        if cat is not None and cur != cat:
+            return FOREIGN
+        return ORIGINAL
+
+    def limit(self, key):
+        """Maximum characters for a subtitle (None = no limit: menu text)."""
+        if not key.startswith("SUB:"):
+            return None
+        return min((subtitles.room(subtitles.Field(l.lang, l.line, "", l.pos, l.size, l.enc))
+                    for l in self.subtitle_locations(key)), default=0)
+
+    def check(self, key, text):
+        """Raise SubtitleError / TextError (ValueError) if `text` cannot be written for `key`, GameError if
+        there is no such text."""
+        if key.startswith("SUB:"):
+            for l in self.subtitle_locations(key):
+                subtitles.encode_fstring(text, l.size, l.enc)
+        else:
+            path = self._menu_path(key)
+            self._menu_entry(key)
+            loctext.check_value(text, self._menu_entries(path, key.split(":")[1])[0], os.path.basename(path))
+
+    def unoriginal_subtitles(self, rel):
+        """Subtitle strings in package `rel` (cooked-relative) that differ from the shipped catalog: [SUB key]
+        (none if the catalog is not the game's)."""
+        if self._shipped_subs is None:
+            self._shipped_subs = ({}, {})
+            shipped = _shipped_index().get("SUB", {})
+            loc = os.path.join(catalog_dir(), subtitles.LOCATIONS)
+            if shipped.get("fingerprint") == banks.game_fingerprint(self.ws.game.cooked) and os.path.exists(loc):
+                by_file, texts = self._shipped_subs
+                for ls in subtitles.read_locations(loc).values():
+                    for l in ls:
+                        by_file.setdefault(l.file, []).append(l)
+                for lang in shipped.get("lines", {}):
+                    p = os.path.join(catalog_dir(), "SUB_%s.csv" % lang)
+                    if os.path.exists(p):
+                        texts.update({(lang, s.speech, s.line): s.text for s in subtitles.read_texts(p, lang)})
+        by_file, texts = self._shipped_subs
+        out = []
+        locs = sorted(by_file.get(rel, []), key=lambda l: l.pos)
+        if locs:
+            r = banks.open_file(self.ws.bank_path(rel))
+            for l in locs:
+                want = texts.get((l.lang, l.speech, l.line))
+                if want is not None and subtitles.read_fstring(r.read(l.pos, l.size), 0)[0] != want:
+                    out.append(subtitles.key(l.lang, l.speech, l.line))
+        return out
+
+    # --- writing --------------------------------------------------------------------------------
+    @_locked
+    def apply(self, changes, force=False, progress=None, meta=None):
+        """changes: {key: new text}. Every file holding one of them is rebuilt once."""
+        for k, v in changes.items():
+            self.check(k, v)
+        conflicts = [k for k in changes if self.state(k) in (CHANGED, FOREIGN)]
+        files = {k: self.files(k) for k in changes}
+        paths = {p for ps in files.values() for p in ps}
+        changed = set(self.ws.changed_files(paths)) | set(self.ws.unoriginal_files(paths))
+        conflicts += [k for k in changes if k not in conflicts and set(files[k]) & changed]
+        if conflicts and not force:
+            raise TextConflictError(conflicts)
+        for p in paths:
+            self.ws._keep_original(p)
+        recs = self.records()
+        now = time.time()
+        for k, v in changes.items():
+            if k not in recs:
+                recs[k] = {"orig": self.current(k), "files": [self.ws.game.rel(p) for p in files[k]],
+                           "first_modified": now, "applied": None}
+            recs[k].update(applied=v, applied_at=now, meta=meta or {})
+        self.ws.store.save()
+        self._sync(paths, progress)
+
+    @_locked
+    def revert(self, keys, progress=None):
+        recs = self.records()
+        keys = [k for k in keys if k in recs]
+        paths = {p for k in keys for p in self.files(k)}
+        for k in keys:
+            recs.pop(k)
+        self.ws.store.save()
+        self._sync(paths, progress)
+        return keys
+
+    def revert_all(self, progress=None):
+        return self.revert(list(self.records()), progress)
+
+    def _sync(self, paths, progress):
+        self._menu.clear()
+        self.ws.sync_files(paths, progress)
+
+    # --- derived-file hooks (called by Workspace.sync_files) ------------------------------------
+    def subtitle_edits(self, rel):
+        out = []
+        for k, r in self.records().items():
+            if r.get("applied") is None or not k.startswith("SUB:"):
+                continue
+            for l in self.subtitle_locations(k):
+                if l.file == rel:
+                    out.append((l.pos, subtitles.encode_fstring(r["applied"], l.size, l.enc)))
+        return out
+
+    def menu_content(self, path, orig_raw):
+        fname = os.path.basename(path)
+        lang = os.path.basename(os.path.dirname(path)).upper()
+        values = {}
+        for k, r in self.records().items():
+            if r.get("applied") is None or not k.startswith("MENU:"):
+                continue
+            _, lg, f, _rest = k.split(":", 3)
+            if lg == lang and f == fname:
+                values[k[len("MENU:"):]] = r["applied"]
+        return loctext.write(orig_raw, lang, fname, values) if values else orig_raw
+
+
+def _find_dir(root, *parts):
+    d = root
+    for p in parts:
+        if not d or not os.path.isdir(d):
+            return None
+        d = next((os.path.join(d, x) for x in os.listdir(d) if x.lower() == p.lower()), None)
+    return d if d and os.path.isdir(d) else None

ackit/upk.py

@@ -0,0 +1,295 @@
+"""The game's UE3 packages (.upk): summary, LZO-compressed chunks, block-level reads and patches.
+
+Offsets used by the rest of ackit are positions in the *uncompressed* package (what Gildor's
+decompress.exe writes), so a sound's location does not depend on how the file is compressed.
+Format and rules: docs/formats/upk.md.
+
+Patching rewrites only the blocks that contain changed bytes (stored as one LZO literal run, see
+lzo.compress_stored) and moves the chunks after them; every other byte of the file stays identical.
+"""
+import struct
+import threading
+from collections import OrderedDict, namedtuple
+
+from . import lzo
+
+TAG = 0x9E2A83C1
+COMPRESS_LZO = 2
+
+
+class PackageError(ValueError):
+    pass
+
+
+Export = namedtuple("Export", "index cls outer name size offset entry")   # entry: table position (uncompressed)
+
+
+class Block:
+    __slots__ = ("chunk", "uoff", "usize", "coff", "csize")
+
+    def __init__(self, chunk, uoff, usize, coff, csize):
+        self.chunk, self.uoff, self.usize, self.coff, self.csize = chunk, uoff, usize, coff, csize
+
+
+class Package:
+    """Read access to one package file. `blocks` covers the uncompressed range [header_size, size)."""
+
+    def __init__(self, path, cache_blocks=8):
+        self.path = path
+        with open(path, "rb") as f:
+            head = f.read(0x10000)
+            self._parse_summary(head, f)
+            self.file_size = f.seek(0, 2)
+        self._cache = OrderedDict()
+        self._cache_n = cache_blocks
+        self._lock = threading.Lock()
+
+    # summary -----------------------------------------------------------------------------------
+    def _parse_summary(self, b, f):
+        if len(b) < 64 or struct.unpack_from("<I", b, 0)[0] != TAG:
+            raise PackageError("%s is not an Unreal package" % self.path)
+        self.version, self.licensee = struct.unpack_from("<HH", b, 4)
+        o = 12
+        n = struct.unpack_from("<i", b, o)[0]
+        o += 4 + max(n, 0)
+        # PackageFlags, NameCount/Offset, ExportCount/Offset, ImportCount/Offset, DependsOffset,
+        # ImportExportGuidsOffset, ImportGuidsCount, ExportGuidsCount, ThumbnailTableOffset
+        f12 = struct.unpack_from("<12i", b, o)
+        self.name_count, self.name_offset = f12[1], f12[2]
+        self.export_count, self.export_offset = f12[3], f12[4]
+        self.import_count, self.import_offset = f12[5], f12[6]
+        o += 48
+        self.guid = b[o:o + 16]
+        o += 16
+        ng = struct.unpack_from("<i", b, o)[0]
+        o += 4 + 12 * ng
+        self.engine, self.cooker, self.compression = struct.unpack_from("<3I", b, o)
+        o += 12
+        nc = struct.unpack_from("<i", b, o)[0]
+        o += 4
+        if nc and len(b) < o + 16 * nc:
+            f.seek(0)
+            b = f.read(o + 16 * nc)
+        self.chunk_table_offset = o
+        self.chunks = [list(struct.unpack_from("<4i", b, o + 16 * i)) for i in range(nc)]  # uoff usize coff csize
+        if nc and self.compression != COMPRESS_LZO:
+            raise PackageError("%s: unsupported compression %d" % (self.path, self.compression))
+        self.blocks = []
+        if not nc:
+            return
+        for ci, (uoff, usize, coff, csize) in enumerate(self.chunks):
+            f.seek(coff)
+            h = f.read(16)
+            tag, bsize, sum_c, sum_u = struct.unpack("<4I", h)
+            if tag != TAG or sum_u != usize:
+                raise PackageError("%s: bad chunk header at 0x%X" % (self.path, coff))
+            nb = (sum_u + bsize - 1) // bsize
+            tbl = f.read(8 * nb)
+            p = coff + 16 + 8 * nb
+            u = uoff
+            for i in range(nb):
+                c, uu = struct.unpack_from("<2I", tbl, 8 * i)
+                self.blocks.append(Block(ci, u, uu, p, c))
+                p += c
+                u += uu
+            if p != coff + csize or u != uoff + usize:
+                raise PackageError("%s: inconsistent chunk %d" % (self.path, ci))
+        self.block_size = bsize
+
+    @property
+    def compressed(self):
+        return bool(self.chunks)
+
+    @property
+    def header_size(self):
+        """Uncompressed size of the summary area (= start of the first chunk's data)."""
+        return self.chunks[0][0] if self.chunks else 0
+
+    @property
+    def size(self):
+        """Uncompressed package size."""
+        if not self.chunks:
+            return self.file_size
+        return self.chunks[-1][0] + self.chunks[-1][1]
+
+    # tables (need the uncompressed data: `read_all`) ---------------------------------------------
+    def tables(self, data=None, start=None):
+        """(names, imports, exports) of the package. imports: [(class name, object name, outer)],
+        exports: [Export]. Layouts: docs/formats/upk.md (licensee 101 adds one u32 to each export)."""
+        if data is None:
+            start, data = self.read_all()
+        buf = memoryview(data)
+        names = []
+        o = self.name_offset - start
+        for _ in range(self.name_count):
+            n = struct.unpack_from("<i", buf, o)[0]
+            o += 4
+            if n >= 0:
+                names.append(bytes(buf[o:o + n - 1]).decode("latin-1"))
+                o += n
+            else:
+                names.append(bytes(buf[o:o - 2 * n - 2]).decode("utf-16-le", "replace"))
+                o += -2 * n
+            o += 8                                   # name flags
+        imports = []
+        o = self.import_offset - start
+        for _ in range(self.import_count):
+            _pkg, _pn, cls, _cn, outer, nm, nn = struct.unpack_from("<7i", buf, o)
+            imports.append((names[cls], names[nm], outer))
+            o += 28
+        exports = []
+        o = self.export_offset - start
+        for i in range(self.export_count):
+            cls, sup, outer, nm, nn, arch, _x = struct.unpack_from("<7i", buf, o)
+            size, off = struct.unpack_from("<ii", buf, o + 36)
+            ngen = struct.unpack_from("<i", buf, o + 48)[0]
+            exports.append(Export(i + 1, cls, outer, names[nm] + ("_%d" % (nn - 1) if nn else ""), size, off,
+                                  o + start))
+            o += 72 + 4 * ngen
+        return names, imports, exports
+
+    def fingerprint(self):
+        """Cheap identity of the package contents: guid + chunk table (changes with any re-cook/patch)."""
+        import hashlib
+        h = hashlib.sha1(self.guid)
+        h.update(struct.pack("<q", self.file_size))
+        for c in self.chunks:
+            h.update(struct.pack("<4i", *c))
+        return h.hexdigest()[:16]
+
+    # reading -----------------------------------------------------------------------------------
+    def _block(self, i, f=None):
+        with self._lock:                        # the editor reads one package from several background jobs
+            data = self._cache.get(i)
+            if data is not None:
+                self._cache.move_to_end(i)
+                return data
+        bl = self.blocks[i]
+        own = f is None
+        f = f or open(self.path, "rb")
+        try:
+            f.seek(bl.coff)
+            data = lzo.decompress(f.read(bl.csize), bl.usize)
+        finally:
+            if own:
+                f.close()
+        with self._lock:
+            self._cache[i] = data
+            if len(self._cache) > self._cache_n:
+                self._cache.popitem(last=False)
+        return data
+
+    def _block_index(self, off):
+        lo, hi = 0, len(self.blocks) - 1
+        while lo <= hi:
+            mid = (lo + hi) // 2
+            bl = self.blocks[mid]
+            if off < bl.uoff:
+                hi = mid - 1
+            elif off >= bl.uoff + bl.usize:
+                lo = mid + 1
+            else:
+                return mid
+        raise PackageError("offset 0x%X is outside the package data" % off)
+
+    def read(self, off, n):
+        """Uncompressed bytes [off, off+n)."""
+        if not self.chunks:
+            with open(self.path, "rb") as f:
+                f.seek(off)
+                return f.read(n)
+        if off < self.header_size:
+            raise PackageError("reading the package summary area is not supported")
+        out = bytearray()
+        i = self._block_index(off)
+        with open(self.path, "rb") as f:
+            while len(out) < n:
+                bl = self.blocks[i]
+                d = self._block(i, f)
+                s = off + len(out) - bl.uoff
+                out += d[s:s + n - len(out)]
+                i += 1
+        return bytes(out)
+
+    def iter_data(self):
+        """(uncompressed offset, bytes) per block, in order (whole-package scans)."""
+        if not self.chunks:
+            with open(self.path, "rb") as f:
+                yield 0, f.read()
+            return
+        with open(self.path, "rb") as f:
+            for bl in self.blocks:
+                f.seek(bl.coff)
+                yield bl.uoff, lzo.decompress(f.read(bl.csize), bl.usize)
+
+    def read_all(self):
+        """The whole uncompressed data area as one bytes object (offset `data_start`)."""
+        parts = list(self.iter_data())
+        return (parts[0][0] if parts else 0), b"".join(p for _, p in parts)
+
+    # patching ----------------------------------------------------------------------------------
+    def patched(self, edits):
+        """edits: [(uncompressed offset, bytes)] (same length as what they overwrite) -> new file bytes.
+
+        Uncompressed packages are patched in place. For compressed ones the touched blocks are
+        re-encoded and the chunk headers and the summary's chunk table are updated."""
+        with open(self.path, "rb") as f:
+            raw = f.read()
+        if not self.chunks:
+            out = bytearray(raw)
+            for off, data in edits:
+                if off + len(data) > len(out):
+                    raise PackageError("patch beyond the end of %s" % self.path)
+                out[off:off + len(data)] = data
+            return bytes(out)
+        touched = {}
+        for off, data in edits:
+            if off < self.header_size or off + len(data) > self.size:
+                raise PackageError("patch outside the package data (0x%X)" % off)
+            pos = 0
+            while pos < len(data):
+                i = self._block_index(off + pos)
+                bl = self.blocks[i]
+                if i not in touched:
+                    touched[i] = bytearray(lzo.decompress(raw[bl.coff:bl.coff + bl.csize], bl.usize))
+                s = off + pos - bl.uoff
+                k = min(len(data) - pos, bl.usize - s)
+                touched[i][s:s + k] = data[pos:pos + k]
+                pos += k
+        # rebuild chunk by chunk
+        head = bytearray(raw[:self.chunks[0][2]])
+        body = bytearray()
+        by_chunk = {}
+        for i, bl in enumerate(self.blocks):
+            by_chunk.setdefault(bl.chunk, []).append(i)
+        for ci, (uoff, usize, coff, csize) in enumerate(self.chunks):
+            new_coff = len(head) + len(body)
+            idx = by_chunk[ci]
+            if not any(i in touched for i in idx):
+                body += raw[coff:coff + csize]
+                self._set_chunk(head, ci, uoff, usize, new_coff, csize)
+                continue
+            datas = []
+            for i in idx:
+                bl = self.blocks[i]
+                if i in touched:
+                    datas.append((lzo.compress_stored(touched[i]), bl.usize))
+                else:
+                    datas.append((raw[bl.coff:bl.coff + bl.csize], bl.usize))
+            hdr = struct.pack("<4I", TAG, self.block_size, sum(len(d) for d, _ in datas), usize)
+            hdr += b"".join(struct.pack("<2I", len(d), u) for d, u in datas)
+            chunk = hdr + b"".join(d for d, _ in datas)
+            body += chunk
+            self._set_chunk(head, ci, uoff, usize, new_coff, len(chunk))
+        return bytes(head + body)
+
+    def _set_chunk(self, head, ci, uoff, usize, coff, csize):
+        struct.pack_into("<4i", head, self.chunk_table_offset + 16 * ci, uoff, usize, coff, csize)
+
+
+def is_package(path):
+    try:
+        with open(path, "rb") as f:
+            return struct.unpack("<I", f.read(4))[0] == TAG
+    except (OSError, struct.error):
+        return False

ackit/verify/catalogcheck.py

@@ -0,0 +1,454 @@
+"""Cross-check the catalog against the installed game (`ackit verify --catalog`).
+
+For every archive the catalog in use (shipped CSV or the user-cache scan) is compared with a fresh scan:
+  missing     a sound of the game that is not in the catalog
+  extra       a catalog row that is not in the game
+  duplicate   a target or a name used twice
+  size/sha1   the catalog's original size or checksum differs from the game (game modified or other build)
+  category    the row's group/category is not what the classification rules give for its name
+  character   the verified speaker differs from data/catalog/characters.csv
+  language    the archive code does not match the WAD's language folder
+  copies      (BANK) stored copies or prefetch heads differ from the location index
+Run it on unmodified game files (`ackit status`), otherwise modified sounds show up as size/sha1 issues.
+"""
+import collections
+import hashlib
+import os
+import random
+
+from .. import banks, classify, subtitles
+from ..catalog import (Catalog, apply_lines, available_archives, bank_names, bank_sounds, build_lines, character_of,
+                       is_shipped, line_names, load_archive, load_banks, load_characters, mark_heads, read_csv,
+                       scan_archive, sfx_names, whole_recordings)
+from ..inject import read_target
+from ..paths import catalog_dir
+from ..wad import Wad
+
+FOLDER_LANG = {"ENGLISH(US)": "INT", "ITALIAN": "ITA", "FRENCH(FRANCE)": "FRA", "GERMAN": "DEU",
+               "SPANISH(SPAIN)": "ESN"}
+
+
+class Report:
+    def __init__(self):
+        self.checked = collections.Counter()
+        self.issues = []                       # (archive, kind, key, detail)
+        self.facts = []                        # what was established about the game's relations (full check)
+
+    def add(self, archive, kind, key, detail=""):
+        self.issues.append((archive, kind, key, detail))
+
+    def counts(self):
+        return collections.Counter((a, k) for a, k, _, _ in self.issues)
+
+    @property
+    def ok(self):
+        return not self.issues
+
+
+def _expected_category(s, names):
+    if s.archive == banks.ARCHIVE:
+        return classify.classify(s.name, bank=s.bank)
+    pack = None
+    if s.is_pack_media:
+        pack = names.get(int(s.target.split("/")[0], 16), s.target.split("/")[0])
+    return classify.classify(s.name, pack=pack)
+
+
+def _check_rows(rep, archive, rows, characters, names):
+    seen_t, seen_n = set(), {}
+    for s in rows:
+        rep.checked[archive] += 1
+        if s.archive != archive:
+            rep.add(archive, "language", s.key, "row says archive %s" % s.archive)
+        if s.target in seen_t:
+            rep.add(archive, "duplicate", s.key, "target")
+        seen_t.add(s.target)
+        n = (s.language, s.name.lower())          # BANK: names are unique per language
+        if n in seen_n or (s.language and ("", n[1]) in seen_n):
+            rep.add(archive, "duplicate", s.key, "name %s (also %s)" % (s.name, seen_n.get(n) or seen_n[("", n[1])]))
+        seen_n[n] = s.key
+        g, c = _expected_category(s, names)
+        if (s.group, s.category) != (g, c):
+            rep.add(archive, "category", s.key, "%s/%s, rules give %s/%s" % (s.group, s.category, g, c))
+        want = character_of(s, characters)
+        if s.character != want:
+            rep.add(archive, "character", s.key, "%r, curated list says %r" % (s.character, want))
+
+
+def check_wad(rep, game, archive, rows, characters, lines=None):
+    """Rows vs a fresh scan of the WAD. With `lines` (full check) the fresh sounds get the game's line
+    identities and names, so names, aliases and lines are compared too."""
+    path = game.wad(archive)
+    if archive != "SFX":
+        folder = os.path.basename(os.path.dirname(path)).upper()
+        if FOLDER_LANG.get(folder, archive) != archive:
+            rep.add(archive, "language", archive, "WAD in folder %s" % folder)
+    _check_rows(rep, archive, rows, characters, bank_names(game.cooked))
+    fresh_list = scan_archive(archive, path, characters=characters)
+    if lines is not None and archive in lines.langs:
+        apply_lines(fresh_list, lines, lines.names)
+    fresh = {s.target: s for s in fresh_list}
+    if lines is None and archive in game.languages:    # quick check: line names need the banks, not rescanned
+        byk = {s.target: s for s in rows}
+        for t, s in fresh.items():
+            if t in byk:
+                s.name, s.aliases, s.line = byk[t].name, byk[t].aliases, byk[t].line
+    byk = {s.target: s for s in rows}
+    for t in sorted(set(fresh) & set(byk)):
+        f, s = fresh[t], byk[t]
+        if (f.name, tuple(f.aliases), f.line) != (s.name, tuple(s.aliases), s.line):
+            rep.add(archive, "name", s.key, "catalog %s %s line %s, game %s %s line %s"
+                    % (s.name, list(s.aliases), s.line or "-", f.name, list(f.aliases), f.line or "-"))
+    cat = {s.target: s for s in rows}
+    for t in sorted(set(fresh) - set(cat)):
+        rep.add(archive, "missing", "%s:%s" % (archive, t), fresh[t].name)
+    for t in sorted(set(cat) - set(fresh)):
+        rep.add(archive, "extra", "%s:%s" % (archive, t), cat[t].name)
+    wad = Wad(path)
+    for t in sorted(set(cat) & set(fresh)):
+        s = cat[t]
+        if s.size and s.size != fresh[t].size and not s.is_pack_media:
+            rep.add(archive, "size", s.key, "catalog %d, game %d" % (s.size, fresh[t].size))
+        if s.sha1:
+            h = hashlib.sha1(read_target(path, t, wad)).hexdigest()[:16]
+            if h != s.sha1:
+                rep.add(archive, "sha1", s.key, "game file differs from the catalog's original")
+
+
+def check_banks(rep, game, rows, locations, characters, quick=False, sample=40, progress=None):
+    """BANK rows, copies, prefetch heads and subtitles vs a scan of the packages. The full check (not quick)
+    also derives the game's lines from that scan and returns them (else None)."""
+    _check_rows(rep, banks.ARCHIVE, rows, characters, bank_names(game.cooked))
+    files = banks.bank_files(game.cooked)
+    if quick:
+        pk = [f for f in files if f.lower().endswith(".upk")]
+        rnd = random.Random(1)
+        files = [f for f in files if f.startswith("SFX/")] + sorted(rnd.sample(pk, min(sample, len(pk))))
+    col = subtitles.Collector()
+    sc = banks.scan(game.cooked, progress, files=files, visit=col)
+    known = {n.split("#")[0] for s in rows for n in s.names()}           # BANK names
+    for a in game.languages:                                               # + speech line names of the catalogs
+        p = os.path.join(catalog_dir(), "%s.csv" % a)
+        if os.path.exists(p):
+            known.update(n.split("#")[0] for s in read_csv(p) for n in s.names())
+    col.resolve({n for n in sc.names | known if n.lower().startswith("speech_")})
+    check_subtitles(rep, game, col, set(files), quick)
+    lines = None
+    index = collections.defaultdict(set)
+    for m, ls in locations.items():
+        for l in ls:
+            if l.file in set(files):
+                index[(l.media, l.kind)].add((l.file, l.offset, l.size, l.sound if l.kind == banks.PREFETCH_HEAD else ""))
+    if not quick:
+        sc.names.update(col.dialogues)        # as the build does: subtitle objects name speech events too
+        lines = build_lines(game, sc)
+        mark_heads(sc, whole_recordings(sc, lines))       # WAD recordings also stored whole in banks
+    else:                                     # a sample cannot derive them: the index says which they are
+        mark_heads(sc, {m for m, k in index if k == banks.PREFETCH_HEAD} - sc.prefetch)
+    found = collections.defaultdict(set)
+    for l in sc.locations:
+        head = l.kind == banks.PREFETCH_HEAD
+        found[(l.media, l.kind)].add((l.file, l.offset, l.size, lines.head_key(l) if head and lines else ""))
+    if lines is None:                       # quick: a subset of the packages cannot resolve the heads' lines
+        index = {k: {(f, o, s, "") for f, o, s, _ in v} for k, v in index.items()}
+    for k in sorted(set(found) | set(index)):
+        if found.get(k, set()) != index.get(k, set()):
+            rep.add(banks.ARCHIVE, "copies", "%08X" % k[0], "%s: index %d, game %d"
+                    % ("head" if k[1] == banks.PREFETCH_HEAD else "copies", len(index.get(k, ())), len(found.get(k, ()))))
+    cat = {s.target: s for s in rows}
+    fresh_list = bank_sounds(sc, characters, sfx_names(game))
+    if lines is not None:
+        lines.names = line_names(game, sc, lines, {s.key: s.name for s in fresh_list})
+        apply_lines(fresh_list, lines, lines.names)
+        for s in fresh_list:
+            s.character = character_of(s, characters)
+    fresh = {s.target: s for s in fresh_list}
+    for t in sorted(set(fresh) - set(cat)):
+        rep.add(banks.ARCHIVE, "missing", "BANK:" + t, fresh[t].bank)
+    if not quick:
+        for t in sorted(set(cat) - set(fresh)):
+            rep.add(banks.ARCHIVE, "extra", "BANK:" + t, cat[t].name)
+    for t in sorted(set(cat) & set(fresh)):
+        s, f = cat[t], fresh[t]
+        if s.sha1 and f.sha1 and s.sha1 != f.sha1:
+            rep.add(banks.ARCHIVE, "sha1", s.key, "game file differs from the catalog's original")
+        if f.size < s.size:
+            rep.add(banks.ARCHIVE, "size", s.key, "catalog %d, game %d" % (s.size, f.size))
+        if lines is not None and (f.name, f.language, f.line) != (s.name, s.language, s.line):
+            rep.add(banks.ARCHIVE, "name", s.key, "catalog %s [%s] line %s, game %s [%s] line %s"
+                    % (s.name, s.language, s.line or "-", f.name, f.language, f.line or "-"))
+    for m, ls in locations.items():
+        for l in ls:
+            if l.kind == banks.PREFETCH_HEAD and not l.sound:
+                rep.add(banks.ARCHIVE, "copies", "%08X" % m, "prefetch head without a speech line")
+    return sc, lines, col
+
+
+def check_subtitles(rep, game, col, files, quick):
+    """Subtitle copies found in the scanned packages vs SUB_locations + the SUB_<LANG>.csv texts."""
+    from ..catalog import _shipped_index, catalog_dir
+    loc_path = os.path.join(catalog_dir(), subtitles.LOCATIONS)
+    if not os.path.exists(loc_path) or not is_shipped(game, banks.ARCHIVE):
+        return
+    index = collections.defaultdict(set)
+    for k, ls in subtitles.read_locations(loc_path).items():
+        for l in ls:
+            if l.file in files:
+                index[k].add((l.file, l.pos, l.size))
+    found, texts = collections.defaultdict(set), {}
+    for speech, copies in col.dialogues.items():
+        for rel, d in copies:
+            for f in d.fields:
+                k = subtitles.key(f.lang, speech, f.index)
+                found[k].add((rel, f.pos, f.size))
+                texts[k] = f.text
+    for k in sorted(set(found) | set(index)):
+        if found.get(k, set()) != index.get(k, set()):
+            rep.add("SUB", "copies", k, "index %d, game %d" % (len(index.get(k, ())), len(found.get(k, ()))))
+    langs = sorted(_shipped_index().get("SUB", {}).get("lines", {}))
+    bank_lang = {s.key: s.language for s in read_csv(os.path.join(catalog_dir(), "BANK.csv"))}
+    for lang in langs:
+        voice = lang if lang in game.languages else "INT"      # languages without own voices: INT voices
+        p = os.path.join(catalog_dir(), "SUB_%s.csv" % lang)
+        rows = subtitles.read_texts(p, lang)
+        rep.checked["SUB_" + lang] += len(rows)
+        seen = set()
+        for s in rows:
+            k = subtitles.key(lang, s.speech, s.line)
+            if k in seen:
+                rep.add("SUB", "duplicate", k)
+            seen.add(k)
+            if k in texts and texts[k] != s.text:
+                rep.add("SUB", "text", k, "catalog %r, game %r" % (s.text[:40], texts[k][:40]))
+            for snd in s.sounds:
+                if snd.startswith(banks.ARCHIVE + ":"):
+                    b = bank_lang.get(snd)
+                    if b and voice not in b.split(";"):             # '' = stored for every language
+                        rep.add("SUB", "language", k, "linked to %s (%s)" % (snd, b))
+                elif not snd.startswith(voice + ":"):
+                    rep.add("SUB", "language", k, "linked to %s, not a %s voice line" % (snd, voice))
+
+
+def check_lines(rep, game, sc, lines, locations):
+    """The game's relations, re-derived from the game files (ackit/lines.py) and compared with the catalogs:
+      links   a broken one-to-one relation in the game data (a WEM without its table row, a Sound object with
+              two recordings in one language, a line without a recording in some language, ...)
+      line    a catalog row whose line identity / name differs from the game's
+      subtitle  a subtitle linked to a different line in one language than in INT
+    Facts (rep.facts) give the counts, prefetch heads compared byte by byte with the start of their WAD
+    recording, and the length agreement of a line's recordings across languages (independent evidence)."""
+    for kind, detail in lines.issues:
+        rep.add("LINES", "links", kind, detail)
+    st = lines.stats
+    for lid, miss in sorted(st["lines_missing_a_language"].items()):
+        rep.facts.append("line %s (%s) has no recording in %s" % (lid, lines.names.get(lid, "?"), ", ".join(miss)))
+    for k in st["without_line"]:
+        rep.add("LINES", "links", k, "a recording of no line")
+    # independent of the BKPK tables: each WAD recording is the source of a Sound object in a bank of the
+    # language the tables give it (localized banks: one per declared language)
+    named = total = 0
+    for L in lines.langs:
+        for m, k in lines.owners.get(L, {}).items():
+            if not k.startswith(banks.ARCHIVE + ":"):
+                total += 1
+                if m in sc.lang_media.get(L, ()):
+                    named += 1
+                else:
+                    rep.add("LINES", "links", k, "media %08X is not a source in any %s bank" % (m, L))
+    rep.facts.append("WAD recordings named as a source by a Sound object of their own language's bank: %d of %d"
+                     % (named, total))
+    rep.facts.append("voice languages %s: %d recordings (WAD WEMs %s, %d of them second places of a recording "
+                     "stored twice, + localized in-memory), %d lines, %d of them with exactly one recording in every "
+                     "language" % ("/".join(lines.langs), st["recordings"],
+                                   "/".join(str(st["wad_wems"].get(L, 0)) for L in lines.langs), len(st["copies"]),
+                                   st["lines"], st["lines_in_every_language"]))
+    # catalog rows vs the game's lines
+    rows = {}
+    for L in lines.langs + [banks.ARCHIVE]:
+        p = os.path.join(catalog_dir(), "%s.csv" % L)
+        if os.path.exists(p):
+            for s in read_csv(p):
+                rows[s.key] = s
+    for lid, per in lines.members.items():
+        for L, k in per.items():
+            s = rows.get(k)
+            if s is None:
+                rep.add("LINES", "line", k, "recording of line %s is not in the catalog" % lid)
+            elif s.line != lid or s.name != lines.names[lid]:
+                rep.add("LINES", "line", k, "catalog %s/%s, game %s/%s" % (s.line, s.name, lid, lines.names[lid]))
+    for k, s in rows.items():
+        if s.line and lines.line_of.get(k) != s.line:
+            rep.add("LINES", "line", k, "catalog says line %s, the game data %s" % (s.line, lines.line_of.get(k)))
+    # prefetch heads: every stored head is the start of the WAD recording it is mapped to
+    wads = {L: (game.wad(L), Wad(game.wad(L))) for L in lines.langs}
+    heads = [l for ls in locations.values() for l in ls if l.kind == banks.PREFETCH_HEAD]
+    by_file = collections.defaultdict(list)
+    for l in heads:
+        by_file[l.file].append(l)
+    same = 0
+    for rel, ls in by_file.items():
+        r = banks.open_file(banks.file_path(game.cooked, rel))
+        for l in ls:
+            L = l.sound.split(":")[0]
+            if L not in wads:
+                rep.add("LINES", "head", "%08X" % l.media, "head in %s maps to %r" % (rel, l.sound))
+                continue
+            w = read_target(wads[L][0], l.sound.split(":", 1)[1], wads[L][1])
+            if r.read(l.offset, l.size) == (w + bytes(l.size))[:l.size]:
+                same += 1
+            else:
+                rep.add("LINES", "head", "%08X" % l.media, "head in %s is not the start of %s" % (rel, l.sound))
+    same_copies = 0
+    for k, primary in sorted(lines.copy_of.items()):          # one recording stored in two WAD entries
+        L = k.split(":")[0]
+        a = read_target(wads[L][0], k.split(":", 1)[1], wads[L][1])
+        b = read_target(wads[L][0], primary.split(":", 1)[1], wads[L][1])
+        if a == b:
+            same_copies += 1
+        else:
+            rep.add("LINES", "copies", k, "differs from %s (same media ID)" % primary)
+    rep.facts.append("recordings stored twice in a WAD (same media ID, written together): %d, byte-identical: %d"
+                     % (len(lines.copy_of), same_copies))
+    rep.facts.append("prefetch heads and whole bank copies: %d of %d are byte-identical to the start of their WAD "
+                     "recording"
+                     % (same, len(heads)))
+    # independent evidence: the recordings of a line have similar lengths in every language
+    ratios = []
+    for lid, per in lines.members.items():
+        secs = [rows[k].seconds for k in per.values() if k in rows and rows[k].seconds > 0]
+        if len(secs) > 1:
+            ratios.append((max(secs) / min(secs), lid))
+    if ratios:
+        ratios.sort()
+        within = sum(1 for r, _ in ratios if r <= 1.5)
+        rep.facts.append("lengths across languages: %d of %d lines within a factor 1.5, median %.2f, longest ratio "
+                         "%.1f (%s)" % (within, len(ratios), ratios[len(ratios) // 2][0], ratios[-1][0],
+                                        lines.names.get(ratios[-1][1], "")))
+    return rows
+
+
+def check_subtitle_lines(rep, rows):
+    """A subtitle line (speech event + line index) must point to the same line in every language."""
+    from ..catalog import _shipped_index
+    langs = sorted(_shipped_index().get("SUB", {}).get("lines", {}))
+    ref = {}
+    p = os.path.join(catalog_dir(), "SUB_INT.csv")
+    if os.path.exists(p):
+        for s in subtitles.read_texts(p, "INT"):
+            if s.sounds and all(k in rows for k in s.sounds):
+                ref[(s.speech, s.line)] = [rows[k].line for k in s.sounds]
+    linked = total = 0
+    for lang in langs:
+        for s in subtitles.read_texts(os.path.join(catalog_dir(), "SUB_%s.csv" % lang), lang):
+            total += 1
+            if not s.sound:
+                continue
+            linked += 1
+            unknown = [k for k in s.sounds if k not in rows]
+            want = ref.get((s.speech, s.line))
+            if unknown:
+                rep.add("SUB", "link", subtitles.key(lang, s.speech, s.line), "linked to unknown %s" % unknown)
+            elif want and [rows[k].line for k in s.sounds] != want:
+                rep.add("SUB", "link", subtitles.key(lang, s.speech, s.line),
+                        "lines %s, INT subtitle is on lines %s" % ([rows[k].line for k in s.sounds], want))
+    rep.facts.append("subtitles: %d strings in %d languages, %d linked to a voice line, each to the same line as in INT"
+                     % (total, len(langs), linked))
+
+
+SAME_PERSON = {("mr freeze", "mister freeze"), ("batman", "bruce wayne"), ("two face", "twoface"),
+               ("azrael", "mysterious stranger")}
+
+
+def _person(s):
+    return " ".join(s.lower().replace("'", "").replace(".", "").replace("_", " ").replace("-", " ").split())
+
+
+def check_speakers(rep, rows, col):
+    """Independent evidence for the curated speaker labels: the dialogue object of a subtitled line names its
+    speaker. Facts only (the lists are curated by listening; differences are for a person to judge)."""
+    speaker = {}
+    for speech, copies in col.dialogues.items():
+        for _, d in copies:
+            if d.speaker:
+                speaker.setdefault(speech, d.speaker)
+    p = os.path.join(catalog_dir(), "SUB_INT.csv")
+    if not os.path.exists(p):
+        return
+    agree, unknown, differ = 0, 0, []
+    for s in subtitles.read_texts(p, "INT"):
+        spk = speaker.get(s.speech, "")
+        if s.line != 0 or not spk:
+            continue
+        for k in s.sounds:
+            r = rows.get(k)
+            if not r or not r.character:
+                continue
+            who = [_person(c) for c in r.character.split("; ")]
+            g = _person(spk)
+            if any(g in w or w in g or (w, g) in SAME_PERSON for w in who):
+                agree += 1
+            elif who == ["unknown"]:
+                unknown += 1
+            else:
+                differ.append("%s (list: %s, game: %s)" % (r.name, r.character, spk))
+    rep.facts.append("speakers: %d labelled voice lines agree with the speaker field of their dialogue object, "
+                     "%d listed as Unknown are named there, %d differ%s"
+                     % (agree, unknown, len(differ), (": " + "; ".join(differ)) if differ else ""))
+
+
+def check_menu(rep, game):
+    """Menu texts: the same keys in every language (facts; differences are the game's own)."""
+    from .. import loctext
+    from ..texts import _find_dir
+    root = _find_dir(game.root, "BmGame", "Localization")       # case-insensitive (Linux)
+    if not root:
+        return
+    keys = {}
+    for lang in loctext.languages(root):
+        try:
+            keys[lang] = {(os.path.splitext(e.file)[0].upper(), e.section, e.name, e.occurrence)
+                          for e in loctext.load(root, lang)}
+        except Exception as e:  # unreadable folder: report, do not stop
+            rep.add("MENU", "read", lang, str(e))
+    if "INT" not in keys:
+        return
+    for lang, ks in sorted(keys.items()):
+        extra, miss = ks - keys["INT"], keys["INT"] - ks
+        rep.facts.append("menu text %s: %d keys%s" % (lang, len(ks), "" if not (extra or miss) else
+                         " (%d not in INT, %d of INT missing)" % (len(extra), len(miss))))
+
+
+def run(game, archives=None, quick=False, log=print, progress=None):
+    """Check the given archives (default: all). Returns a Report."""
+    rep = Report()
+    archives = archives or available_archives(game)
+    cat = Catalog(game, archives, progress)
+    by_arch = collections.defaultdict(list)
+    for s in cat.sounds:
+        by_arch[s.archive].append(s)
+    lines = None
+    if banks.ARCHIVE in archives:           # first: the full check derives the game's lines from this scan
+        log("checking %s ..." % banks.ARCHIVE)
+        rows = load_banks(game, progress)[0]              # every language, not only the loaded one
+        sc, lines, col = check_banks(rep, game, rows, cat.locations, load_characters(lang=None), quick=quick,
+                               progress=progress)
+    for a in archives:
+        if a == banks.ARCHIVE:
+            continue
+        log("checking %s ..." % a)
+        rows_a = by_arch[a] if a in cat.archives else load_archive(game, a, progress)
+        check_wad(rep, game, a, rows_a, load_characters(lang=a), lines)
+    if lines is not None and is_shipped(game, banks.ARCHIVE):
+        log("checking the relations ...")
+        rows_all = check_lines(rep, game, sc, lines, cat.locations)
+        check_subtitle_lines(rep, rows_all)
+        check_speakers(rep, rows_all, col)
+        check_menu(rep, game)
+    characters = load_characters(lang=game.voice_archive(game.language()))
+    known = {n.lower() for s in cat.sounds for n in s.names()}
+    # speaker names are the same in every language; only meaningful for the build the lists were made for
+    if any(a in game.languages and is_shipped(game, a) for a in archives):
+        for n in sorted(characters):
+            if n not in known:
+                rep.add("characters.csv", "extra", n, "no sound with this name")
+    return rep

ackit/verify/debugger.py

@@ -61,48 +61,57 @@ def read_mem(pid, addr, n):
         k32.CloseHandle(h)
 
 
-u32 = ctypes.WinDLL("user32", use_last_error=True)
+class _MBI(ctypes.Structure):
-WM_KEYDOWN, WM_KEYUP = 0x0100, 0x0101
+    _fields_ = [("BaseAddress", ctypes.c_void_p), ("AllocationBase", ctypes.c_void_p),
-VK = dict(enter=0x0D, space=0x20, esc=0x1B, up=0x26, down=0x28, left=0x25, right=0x27, e=0x45)
+                ("AllocationProtect", wt.DWORD), ("RegionSize", ctypes.c_size_t), ("State", wt.DWORD),
+                ("Protect", wt.DWORD), ("Type", wt.DWORD)]
 
 
-def game_windows(pid):
+k32.VirtualQueryEx.argtypes = [wt.HANDLE, ctypes.c_void_p, ctypes.POINTER(_MBI), ctypes.c_size_t]
-    """Top-level windows of the game process."""
+k32.VirtualQueryEx.restype = ctypes.c_size_t
-    found = []
+MEM_COMMIT, PAGE_NOACCESS, PAGE_GUARD = 0x1000, 0x01, 0x100
-    cb_t = ctypes.WINFUNCTYPE(wt.BOOL, wt.HWND, wt.LPARAM)
 
-    def cb(hwnd, _):
+
-        p = wt.DWORD()
+def search_memory(pid, needles, limit=1 << 32):
-        u32.GetWindowThreadProcessId(hwnd, ctypes.byref(p))
+    """Which of `needles` (bytes) occur anywhere in the committed memory of the process -> set."""
-        if p.value == pid and u32.IsWindowVisible(hwnd):
+    h = k32.OpenProcess(PROCESS_VM_READ | PROCESS_QUERY_INFORMATION, False, pid)
-            found.append(hwnd)
+    if not h:
-        return True
+        return set()
-    u32.EnumWindows(cb_t(cb), 0)
+    found = set()
+    try:
+        addr, mbi = 0, _MBI()
+        while addr < limit and len(found) < len(needles):
+            if not k32.VirtualQueryEx(h, ctypes.c_void_p(addr), ctypes.byref(mbi), ctypes.sizeof(mbi)):
+                break
+            size = mbi.RegionSize
+            if mbi.State == MEM_COMMIT and not mbi.Protect & (PAGE_NOACCESS | PAGE_GUARD):
+                buf = ctypes.create_string_buffer(size)
+                got = ctypes.c_size_t()
+                if k32.ReadProcessMemory(h, ctypes.c_void_p(addr), buf, size, ctypes.byref(got)):
+                    data = buf.raw[:got.value]
+                    for n in needles:
+                        if n not in found and n in data:
+                            found.add(n)
+            addr = (mbi.BaseAddress or 0) + size
+    finally:
+        k32.CloseHandle(h)
     return found
 
 
-def post_key(pid, key, hold=0.08):
-    """Post a key press to the game's window only (never global SendInput)."""
-    vk = VK[key] if isinstance(key, str) else key
-    sc = u32.MapVirtualKeyW(vk, 0)
-    n = 0
-    for hwnd in game_windows(pid):
-        u32.PostMessageW(hwnd, WM_KEYDOWN, vk, 1 | (sc << 16))
-        time.sleep(hold)
-        u32.PostMessageW(hwnd, WM_KEYUP, vk, 1 | (sc << 16) | (3 << 30))
-        n += 1
-    return n
-
-
 def running_game_pids():
     out = subprocess.run(["tasklist", "/FI", "IMAGENAME eq BatmanAC.exe", "/FO", "CSV", "/NH"],
                          capture_output=True, text=True).stdout
     return [int(m.group(1)) for m in re.finditer(r'"BatmanAC\.exe","(\d+)"', out)]
 
 
-def kill_game():
+def kill_game(timeout=20):
+    """Kill the game and wait until it has exited: it keeps the WADs and packages open until then, so a revert
+    right after taskkill would find them locked."""
     for pid in running_game_pids():
         subprocess.run(["taskkill", "/F", "/PID", str(pid)], capture_output=True)
+    t0 = time.time()
+    while running_game_pids() and time.time() - t0 < timeout:
+        time.sleep(0.25)
 
 
 def launch_game(exe, extra_args=()):

ackit/verify/ingame.py

@@ -69,6 +69,8 @@ def run(ws, key=DEFAULT_TARGET, wem=None, mode="modded", seconds=40.0, threshold
     report = dict(target=s.key, name=s.name, mode=mode)
     before = {f: _sha1_file(f) for f in files} if mode == "modded" else {}
 
+    rec = runner = None
+    try:                                   # from the first write on: whatever happens, the finally block reverts
         if mode == "modded":
             ws.apply([(s, wem, {"source": "ingame-test"})], progress=log)
         expect = ws.current_bytes(s)
@@ -76,11 +78,9 @@ def run(ws, key=DEFAULT_TARGET, wem=None, mode="modded", seconds=40.0, threshold
         from ..wem import Wem
         report["expect_hash"] = "%08x" % Wem.parse(expect).vorb["codebook_hash"]
         ref_expect, ref_orig = _mono48(expect), _mono48(orig)
-
         rec = LoopbackRecorder(seconds + 10)
         rec.start()
         runner = Run(ws.game.exe, logp)
-    try:
         report.update(runner.start())
         while time.time() - runner.t0 < seconds and runner.alive():
             time.sleep(0.5)
@@ -88,7 +88,9 @@ def run(ws, key=DEFAULT_TARGET, wem=None, mode="modded", seconds=40.0, threshold
                 break
         report["alive_at_end"] = runner.alive()
     finally:
+        if runner:
             runner.stop()
+        if rec:
             rec.stop_flag = True
             rec.join(5)
         if mode == "modded" and revert_after:
@@ -119,3 +121,155 @@ def run(ws, key=DEFAULT_TARGET, wem=None, mode="modded", seconds=40.0, threshold
     report["result"] = "PASS" if ok else "FAIL"
     json.dump(report, open(logp.replace(".log", ".json"), "w"), indent=1)
     return report
+
+
+# --- texts --------------------------------------------------------------------------------------
+def run_texts(ws, seconds=45.0, menu_files=("FRONTEND", "GFXUI"), n_menu=12, n_subs=12, logdir=None, log=print):
+    """Write markers into menu texts of the game's language (front-end files, loaded at startup), into the
+    subtitles stored in Startup.upk (RSubtitle objects, loaded at startup) and into some other subtitles, start
+    the game, then check: no crash, the menu and Startup subtitle markers are in the game's memory (it loaded the
+    edited files), and after the revert every touched file is byte-identical. The other subtitles are not loaded
+    on the title screen; for them the check is that the patched packages do not break the game."""
+    import random
+    from .debugger import Run, kill_game, search_memory
+    tx = ws.texts
+    lang = ws.game.language()
+    menu = [e for e in tx.menu(lang) if e.file.split(".")[0].upper() in menu_files and len(e.value) >= 4]
+    rnd = random.Random(3)
+    picks = rnd.sample(menu, min(n_menu, len(menu)))
+    changes = {"MENU:" + e.key: "ACKIT%02d %s" % (i, e.value) for i, e in enumerate(picks)}
+    subs = [s for s in tx.subtitles(lang) if s.room >= 12] if lang in tx.subtitle_languages() else []
+    startup = [s for s in subs if {l.file for l in tx.subtitle_locations("SUB:%s:%s:%d" % (lang, s.speech, s.line))}
+               == {"Startup.upk"}]
+    for i, s in enumerate(startup):
+        changes["SUB:%s:%s:%d" % (lang, s.speech, s.line)] = ("AKST%02d " % i + s.text)[:s.room]
+    others = [s for s in subs if s not in startup]
+    for i, s in enumerate(rnd.sample(others, min(n_subs, len(others)))):
+        changes["SUB:%s:%s:%d" % (lang, s.speech, s.line)] = ("AKS%02d " % i + s.text)[:s.room]
+    files = sorted({p for k in changes for p in tx.files(k)})
+    logdir = logdir or os.path.join(os.getcwd(), "logs")
+    os.makedirs(logdir, exist_ok=True)
+    logp = os.path.join(logdir, "ingame_texts_%s.log" % time.strftime("%Y%m%d_%H%M%S"))
+    report = dict(mode="texts", lang=lang, menu_texts=len(picks), startup_subtitles=len(startup),
+                  subtitles=len(changes) - len(picks), files=len(files))
+    kill_game()
+    before = {f: _sha1_file(f) for f in files}
+    needles = {("ACKIT%02d " % i).encode("utf-16-le") for i in range(len(picks))}
+    sub_needles = {}                       # marker -> its byte forms (the game may keep cp1252 or UTF-16)
+    for i in range(len(startup)):
+        m = "AKST%02d " % i
+        sub_needles[m] = {m.encode("utf-16-le"), m.encode("cp1252")}
+    needles |= {n for v in sub_needles.values() for n in v}
+    runner = Run(ws.game.exe, logp)
+    found = set()
+    try:                                   # from the first write on: the finally block reverts
+        tx.apply(changes, progress=log)
+        report.update(runner.start())
+        while time.time() - runner.t0 < seconds and runner.alive():
+            time.sleep(0.5)
+            if re.search(r"^ACKIT_CRASH", runner.log(), re.M):
+                break
+        if runner.alive():
+            found = search_memory(report["pid"], needles)
+        report["alive_at_end"] = runner.alive()
+    finally:
+        runner.stop()
+        tx.revert(list(changes), progress=log)
+        report["reverted_byte_identical"] = all(_sha1_file(f) == h for f, h in before.items())
+    report["crash"] = bool(re.search(r"^ACKIT_CRASH", runner.log(), re.M))
+    report["menu_markers_in_memory"] = sum(1 for i in range(len(picks)) if ("ACKIT%02d " % i).encode("utf-16-le") in found)
+    report["startup_subtitles_in_memory"] = sum(1 for v in sub_needles.values() if v & found)
+    ok = (not report["crash"] and report["alive_at_end"] and report["menu_markers_in_memory"] > 0
+          and (not startup or report["startup_subtitles_in_memory"] > 0) and report["reverted_byte_identical"])
+    report["result"] = "PASS" if ok else "FAIL"
+    json.dump(report, open(logp.replace(".log", ".json"), "w"), indent=1)
+    return report
+
+
+# --- in-memory sounds (soundbanks inside the packages) ------------------------------------------
+PROBE_RATE, PROBE_Q = 28000, 4          # a codebook setup the game's own WEMs do not use (E10)
+
+
+def bank_probes(ws, sounds):
+    """[(sound, probe WEM)]: the same probe format for every sound, shortened to fit its slot."""
+    from ..encoder import encode
+    x, sr = probe_signal(10.0)
+    out = []
+    for s in sounds:
+        lim = ws.limit(s)
+        secs = min(10.0, max(0.4, s.seconds or 1.0))
+        while secs >= 0.1:
+            wem, _ = encode(x[:int(secs * sr)], sr, PROBE_RATE, 1, PROBE_Q)
+            if ws.fits(s, wem):                 # within the slot, and able to fill it exactly (wem.fill)
+                out.append((s, wem))
+                break
+            secs *= 0.7 if len(wem) > lim else 0.98
+    return out
+
+
+def run_banks(ws, keys=None, seconds=45.0, logdir=None, log=print):
+    """Replace in-memory sounds (default: every sound stored in the Frontend*/Startup packages) with probes, start the game,
+    then check: no crash, the probe data is in the game's memory (the modified packages were loaded), and
+    after the revert every touched file is byte-identical. Probes decoded (codec setup log) and heard
+    (loopback correlation) are reported too, but no in-memory sound plays on the title screen without input,
+    so they are not required. Returns a report dict (report['result'] PASS/FAIL)."""
+    from .debugger import Run, kill_game, search_memory
+    from .loopback import LoopbackRecorder, detect_chunks
+    from ..wem import Wem
+    if keys:
+        sounds = [ws.catalog.resolve(k) for k in keys]
+    else:  # everything stored in the packages loaded for the title screen (rain ambience plays without input)
+        sounds = [s for s in ws.catalog.sounds if s.archive == "BANK" and any(
+            l.file.lower().startswith(("frontend", "startup")) for l in ws.locations(s))]
+    probes = bank_probes(ws, sounds)
+    files = sorted({f for s, _ in probes for f in ws.files(s)})
+    logdir = logdir or os.path.join(os.getcwd(), "logs")
+    os.makedirs(logdir, exist_ok=True)
+    logp = os.path.join(logdir, "ingame_banks_%s.log" % time.strftime("%Y%m%d_%H%M%S"))
+    report = dict(mode="banks", sounds=len(sounds), probes=len(probes), files=len(files))
+    kill_game()
+    before = {f: _sha1_file(f) for f in files}
+    hashes = {"%08x" % Wem.parse(w).vorb["codebook_hash"] for _, w in probes}
+    needles = set()
+    for _, w in probes:
+        if len(w) > 400:
+            needles.add(w[len(w) // 2:len(w) // 2 + 32])
+    report.update(probe_hashes=sorted(hashes), needles=len(needles))
+    from .debugger import ADDR
+    # also log every setup parse (in-memory sources may not go through the stream setup cache)
+    extra = 'bp %#x ".printf \\"ACKIT_PARSE2 hash=%%08x\\\\n\\", poi(poi(@esp+4)+0x9c); gc"' % ADDR["parse_setup"]
+    runner = Run(ws.game.exe, logp, script_extra=extra)
+    found = set()
+    rec = LoopbackRecorder(seconds + 10)
+    rec.start()
+    try:                                   # from the first write on: the finally block reverts
+        ws.apply([(s, w, {"source": "ingame-test"}) for s, w in probes], progress=log)
+        report.update(runner.start())
+        pid = report["pid"]
+        while time.time() - runner.t0 < seconds and runner.alive():
+            time.sleep(0.5)
+            if re.search(r"^ACKIT_CRASH", runner.log(), re.M):
+                break
+        if runner.alive():
+            found = search_memory(pid, needles)
+        report["alive_at_end"] = runner.alive()
+    finally:
+        runner.stop()
+        rec.stop_flag = True
+        rec.join(5)
+        ws.revert([s for s, _ in probes], progress=log)
+        report["reverted_byte_identical"] = all(_sha1_file(f) == h for f, h in before.items())
+    text = runner.log()
+    report["crash"] = bool(re.search(r"^ACKIT_CRASH", text, re.M))
+    ref = probe_signal(0.35)[0][:, 0]
+    report["capture_error"] = rec.error
+    report["ncc_probe"] = round(detect_chunks(rec.audio(), ref)[0], 3)
+    seen = re.findall(r"ACKIT_(?:SETUP|PARSE2) hash=([0-9a-f]+)", text)
+    report["probe_streams_decoded"] = sum(1 for h in seen if h in hashes)
+    report["setups_logged"] = len(seen)
+    report["probes_in_memory"] = len(found)
+    ok = (not report["crash"] and report["alive_at_end"] and report["probes_in_memory"] > 0
+          and report["reverted_byte_identical"])
+    report["result"] = "PASS" if ok else "FAIL"
+    json.dump(report, open(logp.replace(".log", ".json"), "w"), indent=1)
+    return report

ackit/verify/selfcheck.py

@@ -65,12 +65,26 @@ def run(ws, sample=300, log=print):
                 if n >= sample // max(1, len(ws.game.archives)):
                     break
         log("%-4s layout %s, rebuilt %d/%d sample WEMs byte-identical" % (arch, "OK" if not probs else probs[:2], good, n))
-        ok &= not probs and good >= n - 2  # 4 looping music tracks are known exceptions
+        # known exceptions (docs/formats/wem.md): a few SFX WEMs (music seek tables and sample counts, 4-channel
+        # ambience with a loop chunk) and one INT speech WEM (avg-bytes/s field) are not reproduced byte-for-byte
+        ok &= not probs and good >= n - 2
+    bank = [s for s in ws.catalog.sounds if s.archive == "BANK"]
+    if bank:
+        n = good = 0
+        for s in random.Random(1).sample(bank, min(len(bank), sample)):
+            n += 1
+            try:
+                good += rebuild_identical(ws.current_bytes(s))
+            except Exception:
+                pass
+        log("BANK in-memory sounds: rebuilt %d/%d sample WEMs byte-identical" % (good, n))
+        ok &= good == n
     # 3
     t = np.arange(48000 * 2) / 48000.0
     sig = (0.3 * np.sin(2 * np.pi * 330 * t) * np.sin(2 * np.pi * 1.5 * t)).astype(np.float32)[:, None]
     for s in [x for x in ws.catalog.sounds if x.group == "Speech"][:1] + \
-             [x for x in ws.catalog.sounds if x.group == "Music"][:1]:
+             [x for x in ws.catalog.sounds if x.group == "Music"][:1] + \
+             [x for x in ws.catalog.sounds if x.archive == "BANK" and x.seconds >= 1.5][:1]:
         wem, info = ws.prepare(s, sig, 48000)
         y, sr = audio.decode_wem(wem)
         ref = audio.to_channels(audio.resample(sig, 48000, sr), y.shape[1])

ackit/wad.py

@@ -58,12 +58,6 @@ class Wad:
             f.seek(e.offset)
             return f.read(e.size)
 
-    def iter_entries(self):
-        with open(self.path, "rb") as f:
-            for e in self.entries:
-                f.seek(e.offset)
-                yield e, f.read(e.size)
-
     def check(self):
         """Validate structural invariants; returns list of problems (empty = OK)."""
         probs = []

ackit/wem.py

@@ -66,10 +66,6 @@ class Wem:
     def data(self) -> bytes:
         return self.get(b"data")
 
-    def seek_table(self):
-        n = self.vorb["seek_table_size"] // 4
-        return [struct.unpack_from("<HH", self.data, 4 * i) for i in range(n)]
-
     def setup_packet(self) -> bytes:
         d = self.data
         off = self.vorb["seek_table_size"]
@@ -105,6 +101,60 @@ class Wem:
         return b"RIFF" + struct.pack("<I", len(body)) + body
 
 
+# ---- filling an in-memory slot exactly ----
+# The game ships no in-memory WEM with bytes after its RIFF inside its slot, and plays a replacement that is
+# followed by zero padding there as silence (findings 24). A smaller replacement is therefore grown to the
+# slot's exact size with a label list chunk before `data`, the form the game's own WEMs use for markers:
+# LIST{'adtl', labl{u32 cue id 0, zero text}}. Even sizes only (the game's chunks before `data` are even).
+FILL_MIN = 8 + 4 + 8 + 4 + 2
+
+
+def fillable(length, size):
+    """True if a WEM of `length` bytes can fill a slot of `size` bytes exactly."""
+    pad = size - length
+    return pad == 0 or (pad >= FILL_MIN and pad % 2 == 0)
+
+
+def fill(wem: bytes, size: int) -> bytes:
+    """`wem` grown to exactly `size` bytes (see above). ValueError if `fillable` is False."""
+    if not fillable(len(wem), size):
+        raise ValueError("a %d-byte WEM cannot fill a %d-byte slot exactly" % (len(wem), size))
+    pad = size - len(wem)
+    if not pad:
+        return wem
+    i = _data_chunk(wem)
+    text = pad - 8 - 4 - 8 - 4
+    chunk = (b"LIST" + struct.pack("<I", pad - 8) + b"adtl" + b"labl" + struct.pack("<II", text + 4, 0)
+             + bytes(text))
+    out = wem[:i] + chunk + wem[i:]
+    return out[:4] + struct.pack("<I", len(out) - 8) + out[8:]
+
+
+def unfill(wem: bytes) -> bytes:
+    """The WEM without a filler chunk added by `fill` (a game WEM is returned unchanged)."""
+    i = _data_chunk(wem)
+    if i is None:
+        return wem
+    for j in range(12, i, 2):
+        if wem[j:j + 4] == b"LIST" and wem[j + 8:j + 16] == b"adtllabl":
+            n = 8 + struct.unpack_from("<I", wem, j + 4)[0]
+            if j + n == i and struct.unpack_from("<I", wem, j + 20)[0] == 0 and not wem[j + 24:i].strip(b"\0"):
+                out = wem[:j] + wem[i:]
+                return out[:4] + struct.pack("<I", len(out) - 8) + out[8:]
+    return wem
+
+
+def _data_chunk(wem: bytes):
+    """Offset of the `data` chunk header (walking the chunks), or None."""
+    pos = 12
+    while pos + 8 <= len(wem):
+        tag, ln = struct.unpack_from("<4sI", wem, pos)
+        if tag == b"data":
+            return pos
+        pos += 8 + ln
+    return None
+
+
 SUB_ALIGN = 0x800
 _NONZERO = re.compile(rb"[^\x00]")
 

ackit/workspace.py

@@ -6,19 +6,32 @@ Safety rules (docs/decisions.md):
   * each WAD is rewritten atomically; several sounds of one WAD are written in one pass;
   * if a sound in the game is neither its original nor what we applied (Steam update, "verify
     integrity", another tool), we refuse to write it unless forced, and report it.
+In-memory sounds (archive BANK) and the prefetch heads of speech lines live in loose banks and .upk
+packages (ackit/banks.py); a replacement fills each in-memory slot exactly (wem.fill). Before the first
+change to such a file a verified full copy is kept, and the
+file is always rebuilt as "original copy + every replacement currently applied", so reverting the last
+change restores it byte-for-byte and any later operation repairs an interrupted one. That copy is taken only
+from a file that is original according to the catalog (`unoriginal_files`), else the write is a conflict.
+Writes are serialized (`Workspace.lock`): the editor runs them as background jobs, and two rebuilds of one
+file at the same time would lose one of the edits.
 """
+import functools
+import hashlib
 import os
+import threading
 from collections import defaultdict
 
-from . import audio
+from . import audio, banks
-from .catalog import Catalog
+from .catalog import Catalog, available_archives, is_shipped, load_archive
-from .encoder import encode_fit
+from .encoder import EncodeError, encode_fit
 from .game import Game, GameError
-from .inject import inject, read_target, parse_target, pack_bnk_path
+from .inject import inject, read_target, parse_target, pack_bnk_path, SizeLimitError
 from .originals import OriginalsStore, sha256, sha1_16
 from .pack import Pack
+from .texts import Texts
 from .paths import user_dir
 from .wad import Wad
+from .wem import FILL_MIN, fill, fillable, unfill
 
 # sound states
 ORIGINAL = "original"      # untouched
@@ -33,28 +46,86 @@ class ConflictError(GameError):
         self.sounds = sounds
         super().__init__("These sounds were changed outside this tool (Steam update or 'verify', or "
                          "another mod tool): %s.\nRevert them or confirm to overwrite."
-                         % ", ".join(s.name for s in sounds[:5]) + (" ..." if len(sounds) > 5 else ""))
+                         % (", ".join(s.name for s in sounds[:5]) + (" ..." if len(sounds) > 5 else "")))
 
 
 def default_store_root(game):
     return os.path.join(user_dir("data"), "stores", game.store_id)
 
 
+def locked(f):
+    """Run a Workspace method holding its write lock (Texts has the same wrapper: texts._locked)."""
+    @functools.wraps(f)
+    def run(self, *a, **kw):
+        with self.lock:
+            return f(self, *a, **kw)
+    return run
+
+
 class Workspace:
     def __init__(self, game_dir, store_root=None, archives=None, full_backup=True, progress=None):
         self.game = game_dir if isinstance(game_dir, Game) else Game(game_dir)
         self.store = OriginalsStore(store_root or default_store_root(self.game), self.game.root)
         self.catalog = Catalog(self.game, archives, progress)
         self.full_backup = full_backup
+        self.lock = threading.RLock()       # one write (apply, revert, file rebuild) at a time
+        self._by_file = None                # cooked-relative file -> [Location] (unoriginal_files)
+        self._shipped = None                # the bank catalog is the shipped one (has checksums)
+        self._wad_rows = {}                 # voice archive -> {key: Sound} (heads' recordings)
+        self._other = {}                    # archive not loaded here -> Catalog (sound_for_key)
+        self.texts = Texts(self)
 
     # reading -----------------------------------------------------------------------------------
     def sound(self, name_or_key):
         return self.catalog.resolve(name_or_key)
 
+    def sound_for_key(self, key):
+        """The Sound of a key, also from an archive not loaded here (a change made in another language), or
+        None."""
+        if key in self.catalog.by_key:
+            return self.catalog.by_key[key]
+        archive = key.split(":")[0]
+        if archive not in self._other:
+            self._other[archive] = Catalog(self.game, [archive]) if archive in available_archives(self.game) \
+                else None
+        return self._other[archive].by_key.get(key) if self._other[archive] else None
+
     def wad_path(self, s):
         return self.game.wad(s.archive)
 
+    def bank_path(self, rel):
+        return banks.file_path(self.game.cooked, rel)
+
+    def locations(self, s_or_key):
+        """Bank copies written together with a sound: every copy of a BANK sound, or the prefetch heads
+        of a WAD speech line (usually none)."""
+        key = s_or_key if isinstance(s_or_key, str) else s_or_key.key
+        archive, _, target = key.partition(":")
+        if archive == banks.ARCHIVE:
+            return self.catalog.locations.get(int(target, 16), [])
+        return self.catalog.heads.get(key, [])
+
+    def copies(self, s):
+        """Other places of the same recording in the WAD (written and reverted together with `s`)."""
+        return self.catalog.copies.get(s.key, [])
+
+    def files(self, s):
+        """Absolute paths of the game files that writing `s` changes."""
+        out = []
+        if s.archive != banks.ARCHIVE:
+            out.append(self.wad_path(s))
+            if s.is_pack_media:
+                out.append(pack_bnk_path(self.wad_path(s), parse_target(s.target)[0]))
+        out += sorted({self.bank_path(l.file) for x in [s] + self.copies(s) for l in self.locations(x)})
+        return out
+
     def current_bytes(self, s):
+        if s.archive == banks.ARCHIVE:
+            locs = self.locations(s)
+            if not locs:
+                raise GameError("no copy of %s was found in the game's banks" % s.name)
+            l = locs[0]
+            return unfill(banks.riff_trim(banks.open_file(self.bank_path(l.file)).read(l.offset, l.size)))
         return read_target(self.wad_path(s), s.target)
 
     def original_bytes(self, s):
@@ -81,6 +152,9 @@ class Workspace:
         """Max size in bytes for a replacement (None = any size)."""
         if s.is_pack_media:
             return None
+        if s.archive == banks.ARCHIVE:
+            locs = self.locations(s)
+            return min(l.size for l in locs) if locs else s.size
         e = self.store.get(s.key)
         if e:
             return e["orig_size"]
@@ -91,18 +165,34 @@ class Workspace:
     def original_audio(self, s):
         return audio.decode_wem(self.original_bytes(s))
 
-    def current_audio(self, s):
+    def fill_sizes(self, s):
-        return audio.decode_wem(self.current_bytes(s))
+        """Slot sizes a replacement of `s` must fill exactly: its in-memory copies, and for a WAD recording the
+        bank copies that hold it whole (wem.fill: the game plays a padded in-memory WEM as silence)."""
+        if s.archive == banks.ARCHIVE:
+            return sorted({l.size for l in self.locations(s) if l.kind == banks.MEMORY})
+        orig = self.limit(s)
+        return sorted({l.size for l in self.locations(s) if orig is not None and l.size >= orig})
+
+    def fits(self, s, wem):
+        """True if `wem` can be written for `s`: within its size limit and able to fill its in-memory slots."""
+        lim = self.limit(s)
+        return (lim is None or len(wem) <= lim) and all(fillable(len(wem), n) for n in self.fill_sizes(s))
 
     # encoding ----------------------------------------------------------------------------------
     def prepare(self, s, x, src_rate, quality="auto", progress=None):
-        """Encode PCM for sound `s` (its rate/channels, fitted to its size limit) -> (wem, info)."""
+        """Encode PCM for sound `s` (its rate/channels, fitted to its size limit) -> (wem, info). A sound with
-        return encode_fit(x, src_rate, s.rate or src_rate, s.channels or x.shape[1], self.limit(s),
+        in-memory slots gets a WEM that fills them exactly (`fits`): at least FILL_MIN bytes smaller, and an
-                          quality, progress)
+        even number of bytes smaller; the end is shortened by a few samples until the size allows that."""
-
+        sizes = self.fill_sizes(s)
-    def prepare_file(self, s, path, quality="auto", progress=None):
+        lim = self.limit(s)
-        x, sr = audio.load(path)
+        if sizes:
-        return self.prepare(s, x, sr, quality, progress)
+            lim = min(([lim] if lim is not None else []) + sizes) - FILL_MIN
+        rate, ch = s.rate or src_rate, s.channels or x.shape[1]
+        for k in range(32):
+            wem, info = encode_fit(x[:len(x) - 16 * k] if k else x, src_rate, rate, ch, lim, quality, progress)
+            if self.fits(s, wem):
+                return wem, info
+        raise EncodeError("could not encode a size that fills this sound's slots exactly; shorten the audio")
 
     # writing -----------------------------------------------------------------------------------
     def _backup(self, wad, pack_hashes):
@@ -115,6 +205,11 @@ class Workspace:
 
     def _write(self, archive, items, progress=None):
         """items: [(sound, new_bytes, meta, is_revert)] all in one archive."""
+        if archive == banks.ARCHIVE:
+            return self._write_banks(items, progress)
+        heads = sorted({self.bank_path(l.file) for s, _, _, _ in items for l in self.locations(s)})
+        for path in heads:
+            self._keep_original(path)
         wad = self.game.wad(archive)
         self.game.check_writable(wad)
         packs = {parse_target(s.target)[0] for s, _, _, _ in items if s.is_pack_media}
@@ -140,13 +235,182 @@ class Workspace:
             else:
                 self.store.record_applied(s.key, data, meta)
         self.store.save()
+        self.sync_files(heads, progress)
 
+    # derived files: bank files and packages (in-memory sounds, prefetch heads, subtitles) and the
+    # localization files (menu text). Each is rebuilt from its original copy + every edit applied now.
+    def _derived(self):
+        return self.store.data.setdefault("derived", {})
+
+    def _fingerprint(self, path):
+        """Content hash of a file (whole file: any change by another tool is detected)."""
+        h = hashlib.sha1()
+        with open(path, "rb") as f:
+            for c in iter(lambda: f.read(1 << 22), b""):
+                h.update(c)
+        return h.hexdigest()
+
+    def _keep_original(self, path):
+        """Before the first change to a derived file: verified full copy + its fingerprint."""
+        rel = self.game.rel(path)
+        rec = self._derived().get(rel)
+        if rec:
+            return rec
+        self.game.check_writable(path)
+        self.store.full_backup(path, rel)
+        rec = self._derived()[rel] = {"orig_fp": self._fingerprint(path), "written_fp": None}
+        self.store.save()
+        return rec
+
+    def changed_files(self, paths):
+        """Derived files changed since we last wrote them (Steam update/verify, another tool)."""
+        out = []
+        for path in sorted(set(paths)):
+            rec = self._derived().get(self.game.rel(path))
+            if rec and self._fingerprint(path) not in (rec["orig_fp"], rec["written_fp"]):
+                out.append(path)
+        return out
+
+    def unoriginal_files(self, paths):
+        """Bank files and packages this store has never written that are not the game's original according to
+        the shipped catalog: an in-memory sound, the bank copy of a voice recording or a subtitle in them differs
+        (another originals store or another tool changed them). Their current content must not become this
+        store's "original copy". Menu files and other game builds have no catalog checksums: not checked."""
+        out = []
+        for path in sorted(set(paths)):
+            if self._derived().get(self.game.rel(path)) or self.texts.is_menu_file(path):
+                continue
+            rel = os.path.relpath(path, self.game.cooked).replace(os.sep, "/")
+            if not self._bank_copies_original(rel) or self.texts.unoriginal_subtitles(rel):
+                out.append(path)
+        return out
+
+    def _bank_copies_original(self, rel):
+        if self._by_file is None:
+            self._by_file = defaultdict(list)
+            for ls in self.catalog.locations.values():
+                for l in ls:
+                    self._by_file[l.file].append(l)
+        locs = sorted(self._by_file.get(rel, []), key=lambda l: l.offset)
+        if self._shipped is None:
+            self._shipped = is_shipped(self.game, banks.ARCHIVE)
+        if not locs or not self._shipped:
+            return True
+        r = banks.open_file(self.bank_path(rel))
+        for l in locs:
+            cur = r.read(l.offset, l.size)
+            if l.kind == banks.MEMORY:
+                want = self.catalog.bank_sha1.get(l.media)
+                if want and sha1_16(cur) != want:
+                    return False
+            elif l.sound:                     # a bank copy of a WAD recording: the start of the original
+                orig = self._wad_original(l.sound)
+                if orig is None or cur != (orig + bytes(l.size))[:l.size]:
+                    return False
+        return True
+
+    def _wad_original(self, key):
+        """Original bytes of a WAD recording (this store's record, or the game's when it matches the catalog)."""
+        if self.store.get(key):
+            return self.store.original(key)
+        archive, _, target = key.partition(":")
+        if archive not in self._wad_rows:
+            self._wad_rows[archive] = {s.key: s for s in load_archive(self.game, archive)}
+        s = self._wad_rows[archive].get(key)
+        cur = read_target(self.game.wad(archive), target)
+        return None if s and s.sha1 and sha1_16(cur) != s.sha1 else cur
+
+    def _audio_edits(self, rel):
+        """[(offset, bytes)] that the sound replacements applied now need in bank file `rel` (cooked-relative)."""
+        out = []
+        for key, e in self.store.entries.items():
+            if not e.get("applied"):
+                continue
+            locs = [l for l in self.locations(key) if l.file == rel]
+            if not locs:
+                continue
+            data = unfill(self.store.get_blob(e["applied"]))
+            for l in locs:
+                whole = l.kind == banks.MEMORY or l.size >= e["orig_size"]    # a whole WEM in memory
+                if whole and fillable(len(data), l.size):
+                    payload = fill(data, l.size)
+                elif whole:     # recorded by a version before 3.2.2 (zero padding: the game plays silence)
+                    payload = data + bytes(l.size - len(data))
+                else:           # prefetch head: the start of the WAD recording as it is stored there
+                    full = data + bytes(max(0, e["orig_size"] - len(data)))
+                    payload = (full + bytes(l.size))[:l.size]
+                out.append((l.offset, payload))
+        return out
+
+    def _build(self, path, orig):
+        """Content of derived file `path` from its original copy `orig`."""
+        if self.texts.is_menu_file(path):
+            return self.texts.menu_content(path, open(orig, "rb").read())
+        rel = os.path.relpath(path, self.game.cooked).replace(os.sep, "/")
+        edits = self._audio_edits(rel) + self.texts.subtitle_edits(rel)
+        return banks.open_file(orig).patched(edits) if edits else open(orig, "rb").read()
+
+    @locked
+    def sync_files(self, paths, progress=None):
+        """Rebuild each derived file from its original copy and the edits applied now (atomic writes)."""
+        for path in sorted(set(paths)):
+            rec = self._keep_original(path)
+            orig = self.store.full_copy(self.game.rel(path))
+            if orig is None:
+                raise GameError("the original copy of %s is missing from the store" % self.game.rel(path))
+            if progress:
+                progress("writing %s" % os.path.basename(path))
+            data = self._build(path, orig)
+            self.game.check_writable(path)
+            tmp = path + ".ackit-tmp"
+            with open(tmp, "wb") as f:
+                f.write(data)
+            os.replace(tmp, path)
+            same = data == open(orig, "rb").read()
+            rec["written_fp"] = None if same else self._fingerprint(path)
+        self.store.save()
+
+    def _write_banks(self, items, progress=None):
+        for s, data, meta, is_revert in items:
+            locs = self.locations(s)
+            if not locs:
+                raise GameError("no copy of %s was found in the game's banks" % s.name)
+            if not is_revert and len(data) > min(l.size for l in locs):
+                raise SizeLimitError(s.target, min(l.size for l in locs), len(data))
+        files = sorted({self.bank_path(l.file) for s, _, _, _ in items for l in self.locations(s)})
+        for path in files:
+            self._keep_original(path)
+        for s, data, meta, is_revert in items:
+            if is_revert:
+                self.store.forget(s.key)
+            else:
+                self.store.remember_original(s.key, s.archive, s.target, self.locations(s)[0].file,
+                                             self.current_bytes(s))
+                self.store.record_applied(s.key, data, meta)
+        self.store.save()  # the store says what the files must contain before they change
+        self.sync_files(files, progress)
+
+    @locked
     def apply(self, changes, force=False, progress=None):
-        """changes: [(sound, wem_bytes, meta dict)]. One atomic rewrite per WAD."""
+        """changes: [(sound, wem_bytes, meta dict)]. One atomic rewrite per WAD / bank file. A recording stored
+        in two places of a WAD gets the same bytes in both (Workspace.copies)."""
+        keys = {s.key for s, _, _ in changes}
+        changes = [(s, unfill(data), meta) for s, data, meta in changes]
+        for s, data, _ in changes:
+            if not self.fits(s, data) and len(data) <= (self.limit(s) or len(data)):
+                raise GameError("%s: a %d-byte WEM cannot fill this sound's in-memory slots (%s bytes) exactly; "
+                                "encode it for this sound (Workspace.prepare / the editor / ackit replace)"
+                                % (s.name, len(data), ", ".join(map(str, self.fill_sizes(s)))))
+        changes = list(changes) + [(c, data, meta) for s, data, meta in changes for c in self.copies(s)
+                                   if c.key not in keys]
         conflicts = []
         for s, _, _ in changes:
             if self.state(s) in (CHANGED, FOREIGN):
                 conflicts.append(s)
+        paths = [self.bank_path(l.file) for s, _, _ in changes for l in self.locations(s)]
+        changed = set(self.changed_files(paths)) | set(self.unoriginal_files(paths))
+        conflicts += [s for s, _, _ in changes
+                      if s not in conflicts and {self.bank_path(l.file) for l in self.locations(s)} & changed]
         if conflicts and not force:
             raise ConflictError(conflicts)
         by_arch = defaultdict(list)
@@ -155,10 +419,13 @@ class Workspace:
         for a, items in by_arch.items():
             self._write(a, items, progress)
 
+    @locked
     def revert(self, sounds, progress=None):
         """Put the stored originals back (sounds without a record are skipped). Returns reverted."""
         by_arch = defaultdict(list)
         done = []
+        keys = {s.key for s in sounds}
+        sounds = list(sounds) + [c for s in sounds for c in self.copies(s) if c.key not in keys]
         for s in sounds:
             orig = self.store.original(s.key)
             if orig is None:
@@ -170,28 +437,50 @@ class Workspace:
         self.store.gc()
         return done
 
+    @locked
     def revert_all(self, progress=None):
-        sounds = [self.catalog.by_key[k] for k in list(self.store.entries) if k in self.catalog.by_key]
+        """Revert every sound this store has changed (texts: Texts.revert_all)."""
-        missing = [k for k in self.store.entries if k not in self.catalog.by_key]
+        # records of archives not loaded in this catalog (e.g. another language) included
-        if missing:
+        sounds = [s for s in map(self.sound_for_key, list(self.store.entries)) if s]
-            # records of archives not loaded in this catalog (e.g. another language)
-            other = Catalog(self.game, sorted({k.split(":")[0] for k in missing}))
-            sounds += [other.by_key[k] for k in missing if k in other.by_key]
         return self.revert(sounds, progress)
 
-    def restore_from_full_backup(self):
+    @locked
-        """Last-resort fallback: copy back every full backup taken by this store."""
+    def forget(self, keys=(), text_keys=()):
+        """Stop tracking changes (the game files are kept as they are)."""
+        for k in keys:
+            self.store.forget(k)
+        for k in text_keys:
+            self.texts.records().pop(k, None)
+        self.store.save()
+        self.store.gc()
+
+    @locked
+    def restore_from_full_backup(self, progress=None):
+        """Last resort when the originals store is damaged (`ackit restore-backup`): put back the full copies.
+        A WAD (and its pack indexes) is copied back and its sounds are forgotten; bank files, packages and menu
+        files (always copied) are rebuilt from their copies with the edits of the sounds still recorded, i.e.
+        those of WADs without a full copy (full backups off), which stay modified and revertible. Returns the
+        restored files (game-relative)."""
+        import shutil
+        full = self.store.data.get("full", {})
+        derived = self._derived()
         out = []
-        for rel in self.store.data.get("full", {}):
+        for rel in full:
-            src = self.store.full_copy(rel)
+            if rel in derived:
+                continue
             dst = os.path.join(self.game.root, rel)
             self.game.check_writable(dst)
             tmp = dst + ".ackit-tmp"
-            import shutil
+            shutil.copy2(self.store.full_copy(rel), tmp)
-            shutil.copy2(src, tmp)
             os.replace(tmp, dst)
             out.append(rel)
-        self.store.data["entries"] = {}
+        entries = self.store.data["entries"]
+        for k, e in list(entries.items()):
+            if e["archive"] == banks.ARCHIVE or e.get("wad") in full:
+                del entries[k]
+        self.store.data["texts"] = {}
         self.store.save()
+        paths = [os.path.join(self.game.root, rel) for rel in derived if rel in full]
+        self.sync_files(paths, progress)
         self.store.gc()
-        return out
+        return sorted(out + [rel for rel in derived if rel in full])

app/audioio.py

@@ -163,7 +163,8 @@ class Recorder(QObject):
         self.chunks = []
         self.rate = 48000
         self.channels = 1
-        self.fmt_float = False
+        self.sample = "<i2"            # numpy type of the device's samples
+        self._rest = b""               # bytes of an incomplete frame, completed by the next read
         self._sim = None
 
     def set_device(self, device):
@@ -173,7 +174,7 @@ class Recorder(QObject):
         return self.source is not None or self._sim is not None
 
     def start(self):
-        self.chunks = []
+        self.chunks, self._rest = [], b""
         dev = self.device if (self.device is not None and not self.device.isNull()) else (
             QMediaDevices.defaultAudioInput() if audio_enabled() else None)
         if dev is None or dev.isNull():
@@ -186,7 +187,8 @@ class Recorder(QObject):
         if not dev.isFormatSupported(fmt):
             fmt = dev.preferredFormat()
         self.rate, self.channels = fmt.sampleRate(), fmt.channelCount()
-        self.fmt_float = fmt.sampleFormat() == QAudioFormat.SampleFormat.Float
+        F = QAudioFormat.SampleFormat
+        self.sample = {F.Float: "<f4", F.Int32: "<i4", F.UInt8: "u1"}.get(fmt.sampleFormat(), "<i2")
         self.source = QAudioSource(dev, fmt, self)
         self.io = self.source.start()
         if self.io is None:
@@ -202,13 +204,20 @@ class Recorder(QObject):
         self._ingest(bytes(self.io.readAll()))
 
     def _ingest(self, raw):
+        raw = self._rest + raw
+        frame = np.dtype(self.sample).itemsize * self.channels
+        n = len(raw) // frame * frame
+        raw, self._rest = raw[:n], raw[n:]
         if not raw:
             return
-        if self.fmt_float:
+        x = np.frombuffer(raw, dtype=self.sample)
-            x = np.frombuffer(raw, dtype="<f4")
+        if self.sample == "<i2":
-        else:
+            x = x.astype(np.float32) / 32768.0
-            x = np.frombuffer(raw[: len(raw) // 2 * 2], dtype="<i2").astype(np.float32) / 32768.0
+        elif self.sample == "<i4":
-        x = x[: len(x) // self.channels * self.channels].reshape(-1, self.channels)
+            x = (x / 2147483648.0).astype(np.float32)
+        elif self.sample == "u1":
+            x = (x.astype(np.float32) - 128.0) / 128.0
+        x = x.reshape(-1, self.channels)
         self.chunks.append(x)
         self.level.emit(float(np.max(np.abs(x))) if len(x) else 0.0)
         self.elapsed.emit(sum(len(c) for c in self.chunks) / float(self.rate))
@@ -216,8 +225,8 @@ class Recorder(QObject):
     # simulated input (tests / no microphone) ----------------------------------------------------
     def start_simulated(self, x, rate, realtime=True):
         """Feed `x` as if it came from a microphone (20 ms blocks)."""
-        self.chunks = []
+        self.chunks, self._rest = [], b""
-        self.rate, self.channels, self.fmt_float = rate, x.shape[1], True
+        self.rate, self.channels, self.sample = rate, x.shape[1], "<f4"
         blocks = [x[i:i + rate // 50] for i in range(0, len(x), rate // 50)]
         if not realtime:
             for b in blocks:

app/dialogs.py

@@ -133,7 +133,7 @@ class ExportModDialog(QDialog):
 class ImportModDialog(QDialog):
     """Shows how each item of a package matches this game before installing."""
 
-    def __init__(self, manifest, items, parent=None):
+    def __init__(self, manifest, items, parent=None, texts=()):
         super().__init__(parent)
         self.setWindowTitle("Import mod package")
         lay = QVBoxLayout(self)
@@ -147,13 +147,20 @@ class ImportModDialog(QDialog):
         self.tree.setHeaderLabels(["Sound", "Match", "Note"])
         n_ok = 0
         for im in items:
-            note = {"exact": "same sound", "by name": "matched by name (other language) - will be re-encoded",
+            note = {"exact": "same sound", "same line": "the same line in this language - will be re-encoded",
+                    "by name": "matched by name - will be re-encoded",
                     "missing": im.note}[im.how]
             QTreeWidgetItem(self.tree, [im.item.get("name") or im.item["target"], im.how, note])
             n_ok += im.how != "missing"
+        t_ok = 0
+        for key, _text, problem in texts:
+            QTreeWidgetItem(self.tree, [key, "text", problem or "will be applied"])
+            t_ok += not problem
         self.tree.resizeColumnToContents(0)
         lay.addWidget(self.tree)
-        lay.addWidget(QLabel("%d of %d sounds will be installed." % (n_ok, len(items))))
+        lay.addWidget(QLabel("%d of %d sounds and %d of %d texts will be installed."
+                             % (n_ok, len(items), t_ok, len(texts))))
+        n_ok += t_ok
         bb = QDialogButtonBox(QDialogButtonBox.StandardButton.Ok | QDialogButtonBox.StandardButton.Cancel)
         bb.button(QDialogButtonBox.StandardButton.Ok).setText("Install")
         bb.button(QDialogButtonBox.StandardButton.Ok).setEnabled(n_ok > 0)

app/edit.py

@@ -0,0 +1,455 @@
+"""Edit tab: the one place where sounds are replaced. The sounds come from the Sounds tab (Edit / Add to edit
+list) into an edit list; each keeps its replacement (recorded or loaded), its trim/level values and its effects
+(ackit/fx.py) in the user data folder until you apply it to the game. Layout: edit list | editor (subtitle,
+listen, replacement, actions) | effects. Plan: docs/history/mission4_dubbing_plan.md."""
+import json
+import os
+import re
+import shutil
+
+from PySide6.QtCore import QAbstractTableModel, QModelIndex, Qt, QTimer, Signal
+from PySide6.QtGui import QFont, QKeySequence, QShortcut
+from PySide6.QtWidgets import (QAbstractItemView, QHBoxLayout, QHeaderView, QLabel, QLineEdit, QPushButton,
+                               QScrollArea, QSplitter, QTableView, QVBoxLayout, QWidget)
+
+from ackit import audio, fx
+from ackit.paths import user_dir
+from . import icons
+from .editor import EditorPanel
+from .effects import EffectsRack
+
+
+class EditStore:
+    """The edit list and, per sound key, its replacement audio (WAV) and settings: <user data>/edits/index.json +
+    replacements/*.wav. (Data of the first versions, <user data>/dubs/ and edits/takes/, is moved here once.)"""
+
+    def __init__(self, root=None):
+        self.root = root or os.path.join(user_dir("data"), "edits")
+        old = os.path.join(os.path.dirname(self.root), "dubs")
+        if root is None and not os.path.exists(self.root) and os.path.isdir(old):
+            shutil.move(old, self.root)
+        takes = os.path.join(self.root, "takes")
+        if os.path.isdir(takes) and not os.path.exists(os.path.join(self.root, "replacements")):
+            os.rename(takes, os.path.join(self.root, "replacements"))
+        os.makedirs(os.path.join(self.root, "replacements"), exist_ok=True)
+        self.path = os.path.join(self.root, "index.json")
+        try:
+            data = json.load(open(self.path, encoding="utf-8"))
+        except (OSError, ValueError):
+            data = {}
+        if "items" not in data:                          # first version: {key: item}, no list
+            data = {"list": [k for k, v in data.items() if v.get("take")], "items": data}
+        self.list, self.items = list(data.get("list", [])), dict(data.get("items", {}))
+        for v in self.items.values():                   # first versions: "take" = the source's name
+            if "take" in v:
+                v.setdefault("source", v.pop("take"))
+
+    def save(self):
+        tmp = self.path + ".tmp"
+        with open(tmp, "w", encoding="utf-8") as f:
+            json.dump({"list": self.list, "items": self.items}, f, indent=1, ensure_ascii=False)
+        os.replace(tmp, self.path)
+
+    def get(self, key):
+        return self.items.get(key, {})
+
+    def audio_path(self, key):
+        return os.path.join(self.root, "replacements", re.sub(r"[^A-Za-z0-9_.-]", "_", key) + ".wav")
+
+    def has_replacement(self, key):
+        return bool(self.get(key).get("source")) and os.path.exists(self.audio_path(key))
+
+    def load_replacement(self, key):
+        return audio.load(self.audio_path(key)) if self.has_replacement(key) else None
+
+    def save_replacement(self, key, x, rate, name):
+        audio.save_wav(self.audio_path(key), x, rate)
+        self.update(key, source=name)
+
+    def update(self, key, **fields):
+        self.items[key] = dict(self.get(key), **fields)
+        self.save()
+
+    def add(self, keys):
+        new = [k for k in keys if k not in self.list]
+        self.list += new
+        if new:
+            self.save()
+        return new
+
+    def remove(self, keys):
+        """Off the list; the replacements and settings stay (adding the sound again brings them back)."""
+        self.list = [k for k in self.list if k not in set(keys)]
+        self.save()
+
+
+# --- edit list -----------------------------------------------------------------------------------
+LCOLS = ["Sound", "", "FX", ""]              # Saved and Game have icons in the header
+L_NAME, L_SAVED, L_FX, L_GAME = range(len(LCOLS))
+
+
+class EditListModel(QAbstractTableModel):
+    def __init__(self, store, parent=None):
+        super().__init__(parent)
+        self.store = store
+        self.sounds = []
+        self.modified = set()
+
+    def reset(self, sounds, modified):
+        self.beginResetModel()
+        self.sounds, self.modified = list(sounds), set(modified)
+        self.endResetModel()
+
+    def refresh(self, modified=None):
+        if modified is not None:
+            self.modified = set(modified)
+        if self.sounds:
+            self.dataChanged.emit(self.index(0, 0), self.index(len(self.sounds) - 1, len(LCOLS) - 1))
+
+    def rowCount(self, parent=QModelIndex()):
+        return 0 if parent.isValid() else len(self.sounds)
+
+    def columnCount(self, parent=QModelIndex()):
+        return len(LCOLS)
+
+    def headerData(self, section, orientation, role=Qt.ItemDataRole.DisplayRole):
+        if orientation != Qt.Orientation.Horizontal:
+            return None
+        if role == Qt.ItemDataRole.DisplayRole:
+            return LCOLS[section]
+        if role == Qt.ItemDataRole.DecorationRole:
+            return {L_SAVED: icons.icon("record"), L_GAME: icons.icon("apply", icons.ACCENT)}.get(section)
+        if role == Qt.ItemDataRole.ToolTipRole:
+            return {L_SAVED: "A replacement is saved (user data folder)", L_FX: "Number of effects on",
+                    L_GAME: "Applied: the replacement is in the game"}.get(section)
+        return None
+
+    def state(self, s):
+        if s.key in self.modified:
+            return "Applied"
+        return "Replacement saved" if self.store.has_replacement(s.key) else "Not started"
+
+    def data(self, index, role=Qt.ItemDataRole.DisplayRole):
+        if not index.isValid():
+            return None
+        s, c = self.sounds[index.row()], index.column()
+        rec = self.store.get(s.key)
+        n_fx = sum(1 for b in fx.BLOCKS if (rec.get("fx") or {}).get(b.key, {}).get("on"))
+        if role == Qt.ItemDataRole.DisplayRole:
+            if c == L_NAME:
+                return s.name
+            if c == L_FX:
+                return str(n_fx) if n_fx else ""
+            return ""
+        if role == Qt.ItemDataRole.DecorationRole:
+            if c == L_SAVED and self.store.has_replacement(s.key):
+                return icons.icon("record")
+            if c == L_GAME and s.key in self.modified:
+                return icons.icon("apply", icons.ACCENT)
+        if role == Qt.ItemDataRole.FontRole and c == L_NAME and s.key in self.modified:
+            f = QFont()
+            f.setBold(True)
+            return f
+        if role == Qt.ItemDataRole.ToolTipRole:
+            tip = "%s\n%s  •  %s\n%s" % (s.name, s.group + " / " + s.category, s.character or "—", self.state(s))
+            if n_fx:
+                tip += "\nEffects: " + fx.summary(rec.get("fx"))
+            return tip
+        return None
+
+
+# --- the tab -------------------------------------------------------------------------------------
+class EditPanel(QWidget):
+    message = Signal(str)
+
+    def __init__(self, ctx, parent=None, store=None):
+        super().__init__(parent)
+        self.ctx = ctx                    # main window: .ws, .runner, .player, .settings, ._after_change
+        self.store = store or EditStore()
+        self._restoring = False
+        self._build()
+
+    def _build(self):
+        lay = QVBoxLayout(self)
+        lay.setContentsMargins(0, 0, 0, 0)
+
+        left = QWidget()
+        ll = QVBoxLayout(left)
+        ll.setContentsMargins(6, 6, 0, 6)
+        ll.setSpacing(6)
+        title = QLabel("EDIT LIST")
+        f = title.font()
+        f.setBold(True)
+        title.setFont(f)
+        ll.addWidget(title)
+        self.filter = QLineEdit()
+        self.filter.setPlaceholderText("Filter the list…")
+        self.filter.setClearButtonEnabled(True)
+        self.filter.textChanged.connect(lambda _: self._apply_filter())
+        ll.addWidget(self.filter)
+        self.model = EditListModel(self.store, self)
+        self.table = QTableView()
+        self.table.setModel(self.model)
+        self.table.setSelectionBehavior(QAbstractItemView.SelectionBehavior.SelectRows)
+        self.table.setSelectionMode(QAbstractItemView.SelectionMode.ExtendedSelection)
+        self.table.verticalHeader().hide()
+        self.table.verticalHeader().setDefaultSectionSize(22)
+        self.table.setAlternatingRowColors(True)
+        self.table.setWordWrap(False)
+        self.table.setTextElideMode(Qt.TextElideMode.ElideLeft)        # names differ at their end
+        h = self.table.horizontalHeader()
+        h.setSectionResizeMode(L_NAME, QHeaderView.ResizeMode.Stretch)
+        for c in (L_SAVED, L_FX, L_GAME):
+            h.setSectionResizeMode(c, QHeaderView.ResizeMode.ResizeToContents)
+        self.table.selectionModel().currentRowChanged.connect(
+            lambda cur, _: self.load(self.model.sounds[cur.row()]) if cur.isValid() else None)
+        ll.addWidget(self.table, 1)
+        self.lbl_count = QLabel()
+        self.lbl_count.setObjectName("Muted")
+        self.lbl_count.setWordWrap(True)
+        ll.addWidget(self.lbl_count)
+        row = QHBoxLayout()
+        self.b_remove = QPushButton("Remove")
+        self.b_remove.setToolTip("Take the selected sounds off the list (their replacements and settings are kept)")
+        self.b_remove.clicked.connect(self.remove_selected)
+        self.b_clear = QPushButton("Remove applied")
+        self.b_clear.setToolTip("Take every sound whose replacement is in the game off the list")
+        self.b_clear.clicked.connect(self.remove_applied)
+        row.addWidget(self.b_remove, 1)
+        row.addWidget(self.b_clear, 1)
+        ll.addLayout(row)
+        self.b_use = QPushButton("Use these effects for the selected")
+        self.b_use.setToolTip("Give every selected sound of the list the effects shown on the right (their "
+                              "replacements stay)")
+        self.b_use.clicked.connect(self.use_for_selected)
+        ll.addWidget(self.b_use)
+
+        self.editor = EditorPanel(self.ctx)
+        self.editor.message.connect(self.message)
+        self.editor.source_changed.connect(self._source_changed)
+        self.editor.edited.connect(lambda: self._save_timer.start())
+        self.editor.applied.connect(self._after_write)
+        self.editor.reverted.connect(self._after_write)
+        self.editor.b_prev.clicked.connect(lambda: self.step(-1))
+        self.editor.b_next.clicked.connect(lambda: self.step(1))
+        es = QScrollArea()
+        es.setWidget(self.editor)
+        es.setWidgetResizable(True)
+        mid = QWidget()                                # the editor scrolls; its action bar stays visible
+        ml = QVBoxLayout(mid)
+        ml.setContentsMargins(0, 0, 0, 0)
+        ml.setSpacing(0)
+        ml.addWidget(es, 1)
+        self.editor.layout().removeWidget(self.editor.action_bar)
+        ml.addWidget(self.editor.action_bar)
+        self.rack = EffectsRack()
+        self.rack.changed.connect(self._effects_changed)
+        rs = QScrollArea()
+        rs.setWidget(self.rack)
+        rs.setWidgetResizable(True)
+        rs.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
+        rs.setMinimumWidth(self.rack.minimumSizeHint().width() + rs.verticalScrollBar().sizeHint().width() + 4)
+
+        split = QSplitter(Qt.Orientation.Horizontal)
+        split.addWidget(left)
+        split.addWidget(mid)
+        split.addWidget(rs)
+        split.setStretchFactor(1, 1)
+        split.setSizes([250, 100000, 360])
+        split.setChildrenCollapsible(False)
+        lay.addWidget(split)
+        self.split = split
+
+        ctx = Qt.ShortcutContext.WidgetWithChildrenShortcut
+        QShortcut(QKeySequence("Alt+Left"), self, lambda: self.step(-1), context=ctx)
+        QShortcut(QKeySequence("Alt+Right"), self, lambda: self.step(1), context=ctx)
+        QShortcut(QKeySequence(Qt.Key.Key_Space), self.table, self.editor.toggle_play,
+                  context=Qt.ShortcutContext.WidgetShortcut)
+        QShortcut(QKeySequence(Qt.Key.Key_Delete), self.table, self.remove_selected,
+                  context=Qt.ShortcutContext.WidgetShortcut)
+
+        self._save_timer = QTimer(self)
+        self._save_timer.setSingleShot(True)
+        self._save_timer.setInterval(300)
+        self._save_timer.timeout.connect(self._save_settings)
+        self._nav()
+
+    # --- list ------------------------------------------------------------------------------------
+    def populate(self, ws):
+        """The list's sounds that the loaded catalog has (sounds of another language stay on the list)."""
+        cur = self.editor.sound
+        sounds = [ws.catalog.by_key[k] for k in self.store.list if k in ws.catalog.by_key]
+        self.model.reset(sounds, ws.modified_keys())
+        self.editor.refresh_devices()
+        self._apply_filter()
+        if cur and cur.key in {s.key for s in sounds}:
+            self._select(ws.catalog.by_key[cur.key])
+        elif cur:
+            self.editor.set_sound(None)
+        self._nav()
+
+    def refresh(self, modified=None):
+        self.model.refresh(modified)
+        self._count()
+
+    def _count(self):
+        n, applied = len(self.model.sounds), sum(1 for s in self.model.sounds if s.key in self.model.modified)
+        other = len(self.store.list) - n
+        self.lbl_count.setText("%d sound(s), %d applied%s" % (n, applied, "; %d of another language hidden" % other
+                                                                if other else "") if n or other else
+                               "Empty: select sounds in the Sounds tab and press Edit.")
+
+    def _apply_filter(self):
+        words = self.filter.text().lower().split()
+        for r, s in enumerate(self.model.sounds):
+            hay = (s.name + " " + s.character + " " + s.category).lower()
+            self.table.setRowHidden(r, not all(w in hay for w in words))
+        self._count()
+
+    def add(self, sounds, open_first=False):
+        """Put sounds on the list (once); with open_first, show the first of them."""
+        ws = self.ctx.ws
+        self.store.add([s.key for s in sounds])
+        known = {s.key for s in self.model.sounds}
+        if any(s.key not in known for s in sounds):
+            self.model.reset([ws.catalog.by_key[k] for k in self.store.list if k in ws.catalog.by_key],
+                             ws.modified_keys())
+            self._apply_filter()
+        if open_first and sounds:
+            self._select(sounds[0])
+        self._nav()
+
+    def _row(self, s):
+        return next((i for i, x in enumerate(self.model.sounds) if x.key == s.key), -1)
+
+    def _select(self, s):
+        r = self._row(s)
+        if r < 0:
+            return False
+        idx = self.model.index(r, L_NAME)
+        if self.table.isRowHidden(r):
+            self.filter.clear()
+        if self.table.currentIndex().row() == r and self.editor.sound and self.editor.sound.key == s.key:
+            return True
+        self.table.setCurrentIndex(idx)
+        self.table.scrollTo(idx)
+        return True
+
+    def select_name(self, name):
+        s = next((x for x in self.model.sounds if x.name == name), None)
+        return bool(s) and self._select(s)
+
+    def step(self, d):
+        """Previous / next visible sound of the list."""
+        rows = [r for r in range(len(self.model.sounds)) if not self.table.isRowHidden(r)]
+        if not rows:
+            return
+        cur = self.table.currentIndex().row()
+        nxt = [r for r in rows if (r > cur if d > 0 else r < cur)]
+        if nxt:
+            self._select(self.model.sounds[nxt[0] if d > 0 else nxt[-1]])
+
+    def _nav(self):
+        rows = [r for r in range(len(self.model.sounds)) if not self.table.isRowHidden(r)]
+        cur = self.table.currentIndex().row()
+        self.editor.b_prev.setEnabled(any(r < cur for r in rows))
+        self.editor.b_next.setEnabled(any(r > cur for r in rows))
+
+    def selected_sounds(self):
+        return [self.model.sounds[i.row()] for i in self.table.selectionModel().selectedRows()]
+
+    def remove_selected(self):
+        self._remove(self.selected_sounds())
+
+    def remove_applied(self):
+        self._remove([s for s in self.model.sounds if s.key in self.model.modified])
+
+    def _remove(self, sounds):
+        if not sounds:
+            return
+        self.store.remove([s.key for s in sounds])
+        gone = {s.key for s in sounds}
+        if self.editor.sound and self.editor.sound.key in gone:
+            self.editor.set_sound(None)
+        self.model.reset([s for s in self.model.sounds if s.key not in gone], self.model.modified)
+        self._apply_filter()
+        self._nav()
+
+    def use_for_selected(self):
+        settings = self.rack.settings()
+        sounds = self.selected_sounds()
+        for s in sounds:
+            self.store.update(s.key, fx=settings)
+        self.refresh()
+        self.message.emit("Effects set for %d sound(s)" % len(sounds))
+
+    # --- one sound -------------------------------------------------------------------------------
+    def flush(self):
+        """Write pending edit values now (switching sounds, window closing)."""
+        if self._save_timer.isActive():
+            self._save_timer.stop()
+            self._save_settings()
+
+    def load(self, s):
+        """Show `s` with its saved replacement, edit values and effects."""
+        self.flush()
+        self._restoring = True
+        try:
+            rec = self.store.get(s.key) if s else {}
+            settings = fx.normalize(rec.get("fx"))
+            self.rack.set_settings(settings)
+            self.rack.set_channels(s.channels if s else 1)
+            self.editor.effects = settings
+            self.editor.set_sound(s)
+            ed = self.editor
+            saved = self.store.load_replacement(s.key) if s else None
+            if saved:
+                ed.set_source(saved[0], saved[1], rec.get("source") or "replacement", restoring=True)
+                a, b = rec.get("trim") or (0, 0)
+                if b > a:
+                    ed.sp_a.setValue(a)
+                    ed.sp_b.setValue(b)
+                ed.sp_gain.setValue(rec.get("gain", 0.0))
+                ed.cb_norm.setChecked(rec.get("normalize", False))
+                ed.cb_match.setChecked(rec.get("match", s.group == "Speech"))
+            else:
+                ed.cb_match.setChecked(bool(s) and s.group == "Speech")   # voices: as loud as the original
+        finally:
+            self._restoring = False
+            self._save_timer.stop()
+        self._nav()
+
+    def _source_changed(self):
+        s, ed = self.editor.sound, self.editor
+        if self._restoring or not s or not ed.src:
+            return
+        self.store.save_replacement(s.key, ed.src[0], ed.src[1], ed.src_name)
+        self._save_settings()
+        self.refresh()
+
+    def _effects_changed(self, settings):
+        if self._restoring:
+            return
+        self.editor.set_effects(settings)
+        self._save_timer.start()
+
+    def _save_settings(self):
+        s, ed = self.editor.sound, self.editor
+        if self._restoring or not s:
+            return
+        rec = {"fx": self.rack.settings(), "gain": ed.sp_gain.value(), "normalize": ed.cb_norm.isChecked(),
+               "match": ed.cb_match.isChecked()}
+        if ed.src:
+            rec["trim"] = [ed.sp_a.value(), ed.sp_b.value()]
+        old = self.store.get(s.key)
+        if not ed.src and not fx.active(rec["fx"]) and not old:
+            return                                      # nothing to keep for an untouched sound
+        if any(old.get(k) != v for k, v in rec.items()):
+            self.store.update(s.key, **rec)
+            self.refresh()
+
+    def _after_write(self, key):
+        self.ctx._after_change(key)
+        s = self.editor.sound
+        if s and s.key == key:
+            QTimer.singleShot(0, lambda: self.load(s) if self.editor.sound is s else None)

app/editor.py

@@ -1,14 +1,16 @@
-"""Single-sound editor: listen, substitute (file or recording), process, apply / revert."""
+"""Single-sound editor of the Edit tab: listen, replace (file or recording), process with effects (`effects`,
+ackit/fx.py, between the trim and the loudness steps), apply / revert."""
+import json
 import os
 
 import numpy as np
 from PySide6.QtCore import Qt, QTimer, Signal
 from PySide6.QtGui import QFont
 from PySide6.QtWidgets import (QCheckBox, QComboBox, QDoubleSpinBox, QFileDialog, QGridLayout, QGroupBox,
-                               QHBoxLayout, QLabel, QMessageBox, QProgressBar, QPushButton, QSlider,
+                               QHBoxLayout, QLabel, QMessageBox, QProgressBar, QPushButton, QSizePolicy, QSlider,
                                QVBoxLayout, QWidget)
 
-from ackit import audio, workspace as W
+from ackit import audio, fx, workspace as W
 from . import icons, theme
 from .audioio import Recorder, input_devices, find_device
 from .spinner import Spinner
@@ -29,6 +31,8 @@ class EditorPanel(QWidget):
     applied = Signal(str)     # sound key
     reverted = Signal(str)
     message = Signal(str)
+    source_changed = Signal()  # a new replacement source (file, recording)
+    edited = Signal()          # trim / gain / normalize / loudness changed
 
     def __init__(self, ctx, parent=None):
         super().__init__(parent)
@@ -39,6 +43,10 @@ class EditorPanel(QWidget):
         self.src = None                # replacement source (x, rate) before processing
         self.src_name = ""
         self.playing_what = None
+        self.effects = None            # effects settings (ackit.fx) or None
+        self._src_gen = 0              # identity of the current source (effects cache key)
+        self._fx_cache = (None, None)  # (key, rendered audio)
+        self._fx_pending = None
         self.recorder = Recorder(self)
         self._build()
         self.set_sound(None)
@@ -54,15 +62,29 @@ class EditorPanel(QWidget):
             b.setToolTip(tip)
         return b
 
+    @staticmethod
+    def _fill(*items):
+        """A row of buttons filling the column's width: [(widget, share)], equal edges in every column."""
+        row = QHBoxLayout()
+        row.setSpacing(6)
+        for w, share in items:
+            row.addWidget(w, share)
+        return row
+
     def _wave(self, kind, selectable=False):
         w = Waveform(selectable=selectable)
         w.accent, w.bg = theme.WAVE[kind], theme.WAVE_BG
+        w.setMinimumHeight(44)                 # three rows + the rest fit the Edit tab without scrolling
+        w.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Expanding)   # and grow with the window
         return w
 
     def _build(self):
+        """Top to bottom: header, SUBTITLE (when the sound has one), LISTEN (original, installed and replacement,
+        each with its play button), REPLACEMENT (record / load, trim, level), then the action bar."""
         self.setObjectName("EditorPanel")
         lay = QVBoxLayout(self)
-        lay.setContentsMargins(10, 8, 10, 8)
+        lay.setContentsMargins(8, 4, 8, 2)
+        lay.setSpacing(3)
         self.title = QLabel()
         self.title.setObjectName("EditorTitle")
         self.title.setTextInteractionFlags(Qt.TextInteractionFlag.TextSelectableByMouse)
@@ -71,136 +93,187 @@ class EditorPanel(QWidget):
         f.setBold(True)
         f.setLetterSpacing(QFont.SpacingType.PercentageSpacing, 104)
         self.title.setFont(f)
-        lay.addWidget(self.title)
+        self.state = QLabel()
+        self.state.setWordWrap(True)
+        head = QHBoxLayout()
+        head.addWidget(self.title, 1)
+        head.addWidget(self.state, 0, Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter)
+        lay.addLayout(head)
         self.meta = QLabel()
         self.meta.setWordWrap(True)
         self.meta.setObjectName("Muted")
-        self.state = QLabel()
-        self.state.setWordWrap(True)
         lay.addWidget(self.meta)
-        lay.addWidget(self.state)
 
-        # listen ----------------------------------------------------------------------------------
+        # the subtitle, large ------------------------------------------------------------------------
+        from .textpanel import SubtitlePanel
+        self.subtitle_box = QGroupBox("SUBTITLE")
+        sl = QVBoxLayout(self.subtitle_box)
+        sl.setContentsMargins(8, 8, 8, 8)
+        self.subtitle = SubtitlePanel(self.ctx, self.subtitle_box)
+        sl.addWidget(self.subtitle)
+        lay.addWidget(self.subtitle_box)
+
+        # listen: one row per version, its play button on the left -----------------------------------
         g = QGroupBox("LISTEN")
         gl = QGridLayout(g)
+        gl.setContentsMargins(8, 8, 8, 8)
+        gl.setVerticalSpacing(4)
+        gl.setHorizontalSpacing(8)
         self.wf_orig = self._wave("orig")
         self.wf_inst = self._wave("inst")
-        self.b_play_orig = self._btn("Original", "play", lambda: self.play("orig"), "Play the original (Space)", icons.STEEL)
+        self.wf_new = self._wave("new", selectable=True)
-        self.b_play_inst = self._btn("Installed", "play", lambda: self.play("inst"), "Play what the game has now", "#8f4a4e")
+        self.wf_new.setToolTip("Click: play from here. Right-drag (or Shift+drag): select the part to keep.")
+        self.b_play_orig = self._btn("Original", "play", lambda: self.play("orig"),
+                                     "Play the game's original (Space plays the replacement once there is one)", icons.STEEL)
+        self.b_play_inst = self._btn("Installed", "play", lambda: self.play("inst"), "Play what the game has now",
+                                     "#8f4a4e")
+        self.b_play_new = self._btn("Replacement", "play", lambda: self.play("new"),
+                                    "Play the replacement as it will be applied (trim, effects, level)", icons.ACCENT)
+        for i, (b, w) in enumerate(((self.b_play_orig, self.wf_orig), (self.b_play_inst, self.wf_inst),
+                                    (self.b_play_new, self.wf_new))):
+            b.setMinimumWidth(110)
+            gl.addWidget(b, i, 0)
+            gl.addWidget(w, i, 1)
         self.b_stop = self._btn("Stop", "stop", self.stop, "Stop (Esc)")
-        self.b_ab = self._btn("A/B", "ab", self.ab_switch, "Switch between original and replacement at the same position",
+        self.b_ab = self._btn("A/B", "ab", self.ab_switch, "Switch between the original and the replacement at the same "
-                              icons.ACCENT)
+                              "position, to compare", icons.ACCENT)
         self.cb_loop = self._btn("Loop", "loop", None, "Repeat playback")
         self.cb_loop.setCheckable(True)
-        self.vol_icon = QLabel()
-        self.vol_icon.setPixmap(icons.icon("play", icons.MUTED).pixmap(14, 14))
         self.sl_vol = QSlider(Qt.Orientation.Horizontal)
         self.sl_vol.setRange(0, 100)
         self.sl_vol.setValue(80)
-        self.sl_vol.setMaximumWidth(120)
+        self.sl_vol.setMinimumWidth(80)
         self.sl_vol.setToolTip("Volume")
         self.sl_vol.valueChanged.connect(lambda v: self.ctx.player.set_volume(v / 100.0))
         self.lbl_time = QLabel("0:00.00")
         self.lbl_time.setObjectName("Muted")
-        gl.addWidget(QLabel("Original"), 0, 0)
+        self.lbl_time.setAlignment(Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter)
-        gl.addWidget(self.wf_orig, 0, 1, 1, 6)
+        self.lbl_time.setMinimumWidth(self.lbl_time.fontMetrics().horizontalAdvance("00:00.00") + 4)
-        self.lbl_inst = QLabel("Installed")
+        tr = QHBoxLayout()
-        gl.addWidget(self.lbl_inst, 1, 0)
+        tr.setSpacing(6)
-        gl.addWidget(self.wf_inst, 1, 1, 1, 6)
+        for w in (self.b_stop, self.b_ab, self.cb_loop):
-        row = QHBoxLayout()
+            tr.addWidget(w)
-        for w in (self.b_play_orig, self.b_play_inst, self.b_stop, self.b_ab, self.cb_loop, QLabel("  Volume"),
+        tr.addSpacing(10)
-                  self.sl_vol, self.lbl_time):
+        tr.addWidget(QLabel("Volume"))
-            row.addWidget(w)
+        tr.addWidget(self.sl_vol, 1)
-        row.addStretch(1)
+        tr.addWidget(self.lbl_time)
-        gl.addLayout(row, 2, 0, 1, 7)
+        gl.addLayout(tr, 3, 0, 1, 2)
         gl.setColumnStretch(1, 1)
-        lay.addWidget(g)
         self.wf_orig.seekRequested.connect(lambda t: self.play("orig", t))
         self.wf_inst.seekRequested.connect(lambda t: self.play("inst", t))
+        self.wf_new.seekRequested.connect(lambda t: self.play("new", t))
+        self.wf_new.selectionChanged.connect(self._selection)
+        for r in range(3):
+            gl.setRowStretch(r, 1)             # the waveforms take the free height of the window
+        lay.addWidget(g, 1)
 
-        # replace ---------------------------------------------------------------------------------
+        # replacement: source, trim, level -----------------------------------------------------------
-        g = QGroupBox("REPLACE")
+        g = QGroupBox("REPLACEMENT")
         rl = QVBoxLayout(g)
+        rl.setContentsMargins(8, 8, 8, 8)
+        rl.setSpacing(6)
         src = QHBoxLayout()
-        self.b_load = self._btn("Load file…", "open", self.load_file_dialog,
+        src.setSpacing(6)
-                                "WAV, FLAC, OGG, OPUS, MP3, AIFF (M4A/WMA need ffmpeg installed)  (Ctrl+O)")
-        self.cmb_in = QComboBox()
-        self.cmb_in.setMinimumWidth(180)
-        self.cmb_in.setToolTip("Recording device")
         self.b_rec = self._btn("Record", "record", self.toggle_record, "Record from the selected input")
+        self.b_rec.setObjectName("Primary")
+        self.b_rec.setMinimumWidth(150)
+        self.cmb_in = QComboBox()
+        self.cmb_in.setMinimumWidth(110)
+        self.cmb_in.setToolTip("Recording device")
         self.meter = QProgressBar()
         self.meter.setRange(0, 100)
         self.meter.setTextVisible(False)
-        self.meter.setMaximumWidth(120)
+        self.meter.setFixedWidth(80)
         self.meter.setToolTip("Input level")
         self.lbl_rec = QLabel("")
         self.lbl_rec.setObjectName("Muted")
-        for w in (self.b_load, QLabel("  or"), self.cmb_in, self.b_rec, self.meter, self.lbl_rec):
+        self.b_load = self._btn("Load file…", "open", self.load_file_dialog,
-            src.addWidget(w)
+                                "WAV, FLAC, OGG, OPUS, MP3, AIFF (M4A/WMA need ffmpeg installed)  (Ctrl+O)")
-        src.addStretch(1)
+        src.addWidget(self.b_rec)
+        src.addWidget(self.cmb_in, 1)
+        src.addWidget(self.meter)
+        src.addWidget(self.lbl_rec)
+        src.addSpacing(10)
+        src.addWidget(self.b_load)
         rl.addLayout(src)
-        self.wf_new = self._wave("new", selectable=True)
-        self.wf_new.setToolTip("Click: play from here. Right-drag (or Shift+drag): select the part to keep.")
-        self.wf_new.seekRequested.connect(lambda t: self.play("new", t))
-        self.wf_new.selectionChanged.connect(self._selection)
-        rl.addWidget(self.wf_new)
-        proc = QHBoxLayout()
         self.sp_a = QDoubleSpinBox()
         self.sp_b = QDoubleSpinBox()
         for s_ in (self.sp_a, self.sp_b):
             s_.setDecimals(2)
             s_.setSuffix(" s")
             s_.setRange(0, 36000)
+            s_.setMaximumWidth(90)
             s_.valueChanged.connect(self._changed)
         self.sp_gain = QDoubleSpinBox()
         self.sp_gain.setRange(-40, 40)
         self.sp_gain.setSuffix(" dB")
+        self.sp_gain.setMaximumWidth(90)
         self.sp_gain.valueChanged.connect(self._changed)
         self.cb_norm = QCheckBox("Normalize")
-        self.cb_match = QCheckBox("Match original loudness")
+        self.cb_match = QCheckBox("Match loudness")
+        self.cb_match.setToolTip("Match the loudness of the original sound")
         self.cb_norm.toggled.connect(self._changed)
         self.cb_match.toggled.connect(self._changed)
         self.b_silence = self._btn("Trim silence", None, self.trim_silence, "Cut silence at the start and end")
-        self.b_play_new = self._btn("Replacement", "play", lambda: self.play("new"), "Play the processed replacement",
+        proc = QGridLayout()                   # labels in one column, the fields aligned
-                                    icons.ACCENT)
+        proc.setHorizontalSpacing(6)
-        for w in (QLabel("Keep from"), self.sp_a, QLabel("to"), self.sp_b, self.b_silence, QLabel("  Gain"),
+        proc.setVerticalSpacing(6)
-                  self.sp_gain, self.cb_norm, self.cb_match, self.b_play_new):
+        proc.addWidget(QLabel("Keep"), 0, 0)
-            proc.addWidget(w)
+        proc.addWidget(self.sp_a, 0, 1)
-        proc.addStretch(1)
+        proc.addWidget(QLabel("to"), 0, 2, Qt.AlignmentFlag.AlignCenter)
+        proc.addWidget(self.sp_b, 0, 3)
+        proc.addWidget(self.b_silence, 0, 4)
+        proc.addWidget(QLabel("Gain"), 1, 0)
+        proc.addWidget(self.sp_gain, 1, 1)
+        checks = QHBoxLayout()
+        checks.setSpacing(12)
+        checks.addWidget(self.cb_norm)
+        checks.addWidget(self.cb_match)
+        checks.addStretch(1)
+        proc.addLayout(checks, 1, 2, 1, 3)
+        proc.setColumnStretch(5, 1)
         rl.addLayout(proc)
         info = QHBoxLayout()
         self.lbl_new = QLabel()
-        self.lbl_new.setWordWrap(True)
         self.lbl_new.setObjectName("Muted")
+        self.lbl_new.setWordWrap(True)
         self.warn_icon = QLabel()
         self.warn_icon.setPixmap(icons.icon("warning", icons.ACCENT).pixmap(18, 18))
         self.warn_icon.hide()
         self.lbl_warn = QLabel()
         self.lbl_warn.setWordWrap(True)
         self.lbl_warn.setStyleSheet("color: %s;" % icons.ACCENT)
-        self.lbl_new.setWordWrap(False)
+        info.addWidget(self.lbl_new, 1, Qt.AlignmentFlag.AlignTop)
-        info.addWidget(self.lbl_new, 0, Qt.AlignmentFlag.AlignTop)
-        info.addSpacing(12)
         info.addWidget(self.warn_icon, 0, Qt.AlignmentFlag.AlignTop)
         info.addWidget(self.lbl_warn, 1)
         rl.addLayout(info)
         lay.addWidget(g)
 
-        # apply -----------------------------------------------------------------------------------
+        # actions -----------------------------------------------------------------------------------
-        act = QHBoxLayout()
+        self.b_prev = self._btn("Previous", "prev", None, "Previous sound of the edit list (Alt+Left)")
+        self.b_next = self._btn("Next", "next", None, "Next sound of the edit list (Alt+Right)")
         self.b_apply = self._btn("Apply to game", "apply", self.apply,
                                  "Encode the replacement and install it in the game files (Ctrl+S)", "#ffffff")
         self.b_apply.setObjectName("Primary")
-        self.b_revert = self._btn("Revert to original", "revert", self.revert, "Put the game's original back")
+        self.b_revert = self._btn("Revert", "revert", self.revert, "Put the game's original back")
-        self.b_export = self._btn("Save as WAV…", "save", self.export_wav,
+        self.b_export = self._btn("Save WAV…", "save", self.export_wav,
-                                  "Save the original (or the replacement) as a WAV file")
+                                  "Save the original (or the replacement as applied) as a WAV file")
         self.busy = Spinner(16)
         self.lbl_busy = QLabel()
-        for w in (self.b_apply, self.b_revert, self.b_export, self.busy, self.lbl_busy):
+        self.action_bar = QWidget()            # the Edit tab keeps it below its scroll area (always visible)
-            act.addWidget(w)
+        act = QHBoxLayout(self.action_bar)
-        act.addStretch(1)
+        act.setContentsMargins(8, 4, 8, 6)
-        lay.addLayout(act)
+        act.setSpacing(6)
-        lay.addStretch(1)
+        act.addWidget(self.b_prev, 1)
+        act.addSpacing(10)
+        act.addWidget(self.b_apply, 3)
+        act.addWidget(self.b_revert, 1)
+        act.addWidget(self.b_export, 1)
+        act.addWidget(self.busy)
+        act.addWidget(self.lbl_busy)
+        act.addSpacing(10)
+        act.addWidget(self.b_next, 1)
+        lay.addWidget(self.action_bar)
 
         self.recorder.level.connect(lambda v: self.meter.setValue(int(min(1.0, v) * 100)))
         self.recorder.elapsed.connect(lambda t: self.lbl_rec.setText("recording %s" % fmt_s(t)))
@@ -246,12 +319,16 @@ class EditorPanel(QWidget):
             self.recorder.stop()
         self.sound, self.orig, self.installed, self.src = s, None, None, None
         enabled = s is not None
-        for w in self.findChildren(QPushButton) + [self.sp_a, self.sp_b, self.sp_gain, self.cb_norm, self.cb_match]:
+        # the action bar may live outside this widget (the Edit tab keeps it below its scroll area)
+        buttons = set(self.findChildren(QPushButton)) | set(self.action_bar.findChildren(QPushButton))
+        for w in list(buttons) + [self.sp_a, self.sp_b, self.sp_gain, self.cb_norm, self.cb_match]:
             w.setEnabled(enabled)
         self.wf_orig.set_audio(None, 1, "select a sound" if s is None else "loading…", busy=s is not None)
         self.wf_inst.set_audio(None, 1, "")
         self.wf_new.set_audio(None, 1, "load a file or record a replacement")
         self.lbl_new.setText("")
+        self.subtitle.set_sound(s)
+        self.subtitle_box.setVisible(bool(self.subtitle.keys))
         if s is None:
             self.title.setText("No sound selected")
             self.meta.setText("Search and select a sound in the list.")
@@ -260,10 +337,14 @@ class EditorPanel(QWidget):
         self.title.setText(s.name)
         who = s.character or (("mentions " + s.characters_inferred) if s.characters_inferred else "—")
         lim = self.ctx.ws.limit(s)
-        self.meta.setText("%s / %s   •   %s   •   %.2f s, %d Hz, %s   •   max size: %s   •   %s"
+        where = ""
+        if s.bank:
+            n = len(self.ctx.ws.locations(s))
+            where = "   •   bank %s (%d %s)" % (s.bank, n, "copy" if n == 1 else "copies")
+        self.meta.setText("%s / %s   •   %s   •   %.2f s, %d Hz, %s   •   max size: %s   •   %s%s"
                           % (s.group, s.category, who, s.seconds, s.rate,
                              {1: "mono", 2: "stereo"}.get(s.channels, "%d ch" % s.channels),
-                             "any" if lim is None else "%.1f KB" % (lim / 1024.0), s.key))
+                             "any" if lim is None else "%.1f KB" % (lim / 1024.0), s.key, where))
         key = s.key
         ws = self.ctx.ws
 
@@ -316,8 +397,9 @@ class EditorPanel(QWidget):
         self.ctx.runner.start(lambda job: audio.load(path), done, failed)
         return result[0] if result else True
 
-    def set_source(self, x, sr, name):
+    def set_source(self, x, sr, name, restoring=False):
         self.src, self.src_name = (np.asarray(x, np.float32), sr), name
+        self._src_gen += 1
         dur = audio.duration(x, sr)
         self.sp_a.blockSignals(True)
         self.sp_b.blockSignals(True)
@@ -328,6 +410,8 @@ class EditorPanel(QWidget):
         self.sp_a.blockSignals(False)
         self.sp_b.blockSignals(False)
         self._update_new()
+        if not restoring:
+            self.source_changed.emit()
 
     def toggle_record(self):
         if self.recorder.is_recording():
@@ -363,15 +447,65 @@ class EditorPanel(QWidget):
 
     def _changed(self, *_):
         self._reproc.start()
+        self.edited.emit()
 
-    def processed(self):
+    def set_effects(self, settings):
-        """Replacement audio after trim / gain / normalize / loudness matching -> (x, rate) or None."""
+        self.effects = settings
-        if not self.src:
+        self._changed()
-            return None
+
+    def _fx_key(self):
+        return (self._src_gen, self.sp_a.value(), self.sp_b.value(), json.dumps(self.effects, sort_keys=True),
+                self.sound.channels if self.sound else None)
+
+    def _effected(self, render=True):
+        """Trimmed replacement with the effects (cached). With render=False: None while not rendered yet."""
         x, sr = self.src
         a, b = self.sp_a.value(), self.sp_b.value()
         if b > a:
             x = audio.trim(x, sr, a, b)
+        if not fx.active(self.effects):
+            return x
+        key = self._fx_key()
+        if self._fx_cache[0] == key:
+            return self._fx_cache[1]
+        if not render:
+            return None
+        y = fx.render(x, sr, self.effects, key[-1])
+        self._fx_cache = (key, y)
+        return y
+
+    def _render_effects(self):
+        """Render the effects in the background (spinner in the replacement waveform), then show the result."""
+        key = self._fx_key()
+        if self._fx_pending == key:
+            return
+        self._fx_pending = key
+        x, sr = self.src
+        a, b = self.sp_a.value(), self.sp_b.value()
+        if b > a:
+            x = audio.trim(x, sr, a, b)
+        settings = self.effects
+        self.wf_new.set_audio(None, 1, "applying effects…", busy=True)
+
+        def done(y):
+            if self._fx_pending == key:
+                self._fx_pending = None
+                self._fx_cache = (key, y)
+                if self.src and self._fx_key() == key:
+                    self._update_new()
+
+        def failed(e, tb):
+            self._fx_pending = None
+            self._failed("The effects failed", e)
+
+        self.ctx.runner.start(lambda job: fx.render(x, sr, settings, key[-1]), done, failed)
+
+    def processed(self):
+        """Replacement audio after trim / effects / gain / normalize / loudness matching -> (x, rate) or None."""
+        if not self.src:
+            return None
+        sr = self.src[1]
+        x = self._effected()
         if self.cb_match.isChecked() and self.orig is not None:
             x = audio.match_loudness(x, self.orig[0])
         if self.cb_norm.isChecked():
@@ -381,6 +515,9 @@ class EditorPanel(QWidget):
         return x, sr
 
     def _update_new(self):
+        if self.src and self._effected(render=False) is None:
+            self._render_effects()
+            return
         p = self.processed()
         if p is None:
             self.wf_new.set_audio(None, 1, "load a file or record a replacement")
@@ -391,7 +528,9 @@ class EditorPanel(QWidget):
         self.wf_new.set_audio(x, sr)
         dur = audio.duration(x, sr)
         s = self.sound
-        self.lbl_new.setText("%s: %.2f s, peak %.1f dBFS" % (self.src_name, dur, audio.peak_db(x)))
+        on = fx.summary(self.effects)
+        self.lbl_new.setText("%s%s: %.2f s, peak %.1f dBFS" % (self.src_name, " + " + on if on else "", dur,
+                                                                audio.peak_db(x)))
         warnings = []
         if s and not s.is_pack_media and dur > s.seconds * 1.15 + 0.25:
             warnings.append("Longer than the original (%.2f s): it will be squeezed into the original's size "
@@ -413,6 +552,7 @@ class EditorPanel(QWidget):
         if not a:
             return
         self.playing_what = what
+        self.ctx.player.owner = self               # two editors share the player: only this one shows playheads
         self.ctx.player.play(a[0], a[1], start, loop=self.cb_loop.isChecked(), label=what)
 
     def toggle_play(self):
@@ -436,6 +576,8 @@ class EditorPanel(QWidget):
             w.set_playhead(None)
 
     def _pos(self, t):
+        if getattr(self.ctx.player, "owner", self) is not self:
+            return
         self.lbl_time.setText(fmt_s(t))
         wf = {"orig": self.wf_orig, "inst": self.wf_inst, "new": self.wf_new}.get(self.playing_what)
         if wf:
@@ -471,6 +613,8 @@ class EditorPanel(QWidget):
                 return
         ws = self.ctx.ws
         meta = {"source": self.src_name}
+        if fx.active(self.effects):
+            meta["effects"] = fx.summary(self.effects)
 
         def work(job):
             job.report("encoding…")

app/effects.py

@@ -0,0 +1,328 @@
+"""Effects panel of the Edit tab, built from ackit.fx (BLOCKS, PRESETS, LOOKS), sized to fit without scrolling:
+a preset list (several effects at once), the effect chain (one line per effect: tick = on, click = show its
+settings) and the settings of the selected effect (its preset buttons, one slider per parameter with a readable
+value: kHz, ms, dB, %; logarithmic for frequencies and times). Double-click a slider to reset it; moving a
+slider or pressing a preset turns its effect on."""
+import json
+import math
+
+from PySide6.QtCore import Qt, Signal
+from PySide6.QtGui import QFont
+from PySide6.QtWidgets import (QAbstractItemView, QApplication, QCheckBox, QComboBox, QGridLayout, QHBoxLayout,
+                               QLabel, QListWidget, QListWidgetItem, QPushButton, QSlider, QStackedWidget,
+                               QVBoxLayout, QWidget)
+
+from ackit import fx
+
+STEPS = 1000
+
+
+def is_log(p):
+    return p.kind != "bool" and p.lo > 0 and p.hi / p.lo >= 20
+
+
+def fmt(p, v):
+    """Value as people read it: 3.4 kHz, 160 ms, -6.0 dB, +4.0 st, 35 %."""
+    if p.kind == "bool":
+        return "on" if v else "off"
+    if p.unit == "Hz":
+        return "%.1f kHz" % (v / 1000) if v >= 1000 else ("%.2f Hz" % v if v < 10 else "%d Hz" % round(v))
+    if p.unit in ("st", "dB"):
+        return "%+.1f %s" % (v, p.unit) if v else "0 %s" % p.unit
+    if p.unit == "ms":
+        return "%.1f ms" % v if v < 10 else "%d ms" % round(v)
+    if p.unit == "s":
+        return "%.2f s" % v
+    if not p.unit and p.kind == "float" and p.hi <= 1:
+        return "%d %%" % round(v * 100)
+    if p.kind == "int":
+        return "%d%s" % (v, " " + p.unit if p.unit else "")
+    return "%.1f %s" % (v, p.unit) if p.unit else "%.2f" % v
+
+
+class Slider(QSlider):
+    """Horizontal slider that ignores the mouse wheel unless focused (scrolling the panel never changes values)
+    and resets to its default on double-click."""
+    reset = Signal()
+
+    def __init__(self):
+        super().__init__(Qt.Orientation.Horizontal)
+        self.setFocusPolicy(Qt.FocusPolicy.StrongFocus)
+
+    def wheelEvent(self, e):
+        if self.hasFocus():
+            super().wheelEvent(e)
+        else:
+            e.ignore()
+
+    def mouseDoubleClickEvent(self, e):
+        self.reset.emit()
+
+
+class ParamControl(QWidget):
+    """One parameter: name | slider | value (or a check box for on/off parameters)."""
+    changed = Signal()
+
+    def __init__(self, p, parent=None):
+        super().__init__(parent)
+        self.p = p
+        lay = QHBoxLayout(self)
+        lay.setContentsMargins(0, 0, 0, 0)
+        lay.setSpacing(8)
+        self.name = QLabel(p.label)
+        self.name.setMinimumWidth(92)
+        self.name.setToolTip(p.tip)
+        lay.addWidget(self.name)
+        if p.kind == "bool":
+            self.check = QCheckBox()
+            self.check.setToolTip(p.tip)
+            self.check.toggled.connect(lambda _: self.changed.emit())
+            lay.addWidget(self.check, 1)
+            self.slider = self.value_lbl = None
+            return
+        self.check = None
+        self.slider = Slider()
+        self.slider.setRange(0, self.steps())
+        self.slider.setToolTip("%s\nDouble-click: back to %s" % (p.tip, fmt(p, p.default)))
+        self.value_lbl = QLabel()
+        self.value_lbl.setMinimumWidth(self.value_lbl.fontMetrics().horizontalAdvance("-10.0 kHz") + 4)
+        self.value_lbl.setAlignment(Qt.AlignmentFlag.AlignRight | Qt.AlignmentFlag.AlignVCenter)
+        self.slider.valueChanged.connect(self._moved)
+        self.slider.reset.connect(lambda: (self.set_value(p.default), self.changed.emit()))
+        lay.addWidget(self.slider, 1)
+        lay.addWidget(self.value_lbl)
+
+    def steps(self):
+        p = self.p
+        return int(round((p.hi - p.lo) / p.step)) if p.kind == "int" else STEPS
+
+    def to_pos(self, v):
+        p = self.p
+        if p.kind == "int":
+            return int(round((v - p.lo) / p.step))
+        f = math.log(v / p.lo) / math.log(p.hi / p.lo) if is_log(p) else (v - p.lo) / (p.hi - p.lo)
+        return int(round(min(max(f, 0.0), 1.0) * STEPS))
+
+    def from_pos(self, i):
+        p = self.p
+        if p.kind == "int":
+            return int(p.lo + i * p.step)
+        f = i / STEPS
+        v = p.lo * (p.hi / p.lo) ** f if is_log(p) else p.lo + f * (p.hi - p.lo)
+        q = p.step or 0.01                    # keep values tidy: rounded to the parameter's step (finer when low)
+        if is_log(p) and v < 10 * q:
+            q = q / 10
+        return round(round(v / q) * q, 6)
+
+    def _moved(self, i):
+        self._value = self.from_pos(i)
+        self.value_lbl.setText(fmt(self.p, self._value))
+        self.changed.emit()
+
+    def value(self):
+        if self.check is not None:
+            return self.check.isChecked()
+        return self._value
+
+    def set_value(self, v):
+        if self.check is not None:
+            self.check.blockSignals(True)
+            self.check.setChecked(bool(v))
+            self.check.blockSignals(False)
+            return
+        self._value = v
+        self.slider.blockSignals(True)
+        self.slider.setValue(self.to_pos(v))
+        self.slider.blockSignals(False)
+        self.value_lbl.setText(fmt(self.p, v))
+
+
+class EffectsRack(QWidget):
+    changed = Signal(dict)
+
+    def __init__(self, parent=None):
+        super().__init__(parent)
+        self.controls, self.pages, self.items = {}, {}, {}
+        lay = QVBoxLayout(self)
+        lay.setContentsMargins(6, 6, 6, 6)
+        lay.setSpacing(6)
+        head = QHBoxLayout()
+        title = QLabel("EFFECTS")
+        f = title.font()
+        f.setBold(True)
+        title.setFont(f)
+        head.addWidget(title, 1)
+        self.b_reset = QPushButton("Reset")
+        self.b_reset.setToolTip("Every effect off, every value back to its default")
+        self.b_copy = QPushButton("Copy")
+        self.b_copy.setToolTip("Copy these settings (to paste them on another sound)")
+        self.b_paste = QPushButton("Paste")
+        self.b_paste.setToolTip("Use copied settings on this sound")
+        for b in (self.b_reset, self.b_copy, self.b_paste):
+            head.addWidget(b)
+        lay.addLayout(head)
+        self.cmb_preset = QComboBox()
+        self.cmb_preset.addItem("Start from a preset…")
+        for name, blocks in fx.LOOKS:
+            self.cmb_preset.addItem(name)
+            self.cmb_preset.setItemData(self.cmb_preset.count() - 1, "Turns on: " + ", ".join(
+                fx.BY_KEY[k].label for k in blocks) + " (every other effect off)", Qt.ItemDataRole.ToolTipRole)
+        self.cmb_preset.setToolTip("Sets several effects at once, like sounds heard in the game")
+        self.cmb_preset.activated.connect(self._preset_chosen)
+        lay.addWidget(self.cmb_preset)
+
+        self.chain = QListWidget()                     # the effects in their order: tick = on, select = settings
+        self.chain.setStyleSheet("QListWidget::item { padding: 3px 2px; } "     # selected: grey, so a ticked
+                                 "QListWidget::item:selected { background: #3a3a3a; color: #ffffff; }")  # box shows
+        self.chain.setSelectionMode(QAbstractItemView.SelectionMode.SingleSelection)
+        self.chain.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
+        for blk in fx.BLOCKS:
+            it = QListWidgetItem(blk.label)
+            it.setFlags(it.flags() | Qt.ItemFlag.ItemIsUserCheckable)
+            it.setCheckState(Qt.CheckState.Unchecked)
+            it.setToolTip(blk.tip)
+            it.setData(Qt.ItemDataRole.UserRole, blk.key)
+            self.chain.addItem(it)
+            self.items[blk.key] = it
+        rows = self.chain.sizeHintForRow(0) * len(fx.BLOCKS)
+        self.chain.setFixedHeight(rows + 2 * self.chain.frameWidth() + 2)
+        self.chain.itemChanged.connect(self._item_changed)
+        self.chain.currentItemChanged.connect(lambda cur, _: cur and self.show_block(
+            cur.data(Qt.ItemDataRole.UserRole)))
+        lay.addWidget(self.chain)
+        hint = QLabel("Tick an effect to use it, click its name for its settings. They run top to bottom.")
+        hint.setObjectName("Muted")
+        hint.setWordWrap(True)
+        lay.addWidget(hint)
+
+        self.stack = QStackedWidget()
+        for blk in fx.BLOCKS:
+            page = self._page(blk)
+            self.pages[blk.key] = page
+            self.stack.addWidget(page)
+        lay.addWidget(self.stack)
+        lay.addStretch(1)
+        self.b_reset.clicked.connect(lambda: (self.set_settings(fx.defaults()), self._emit()))
+        self.b_copy.clicked.connect(lambda: QApplication.clipboard().setText(json.dumps(self.settings())))
+        self.b_paste.clicked.connect(self.paste)
+        self.chain.setCurrentRow(0)
+
+    def _page(self, blk):
+        page = QWidget()
+        v = QVBoxLayout(page)
+        v.setContentsMargins(0, 4, 0, 0)
+        v.setSpacing(3)
+        name = QLabel(blk.label)
+        f = name.font()
+        f.setBold(True)
+        name.setFont(f)
+        v.addWidget(name)
+        about = QLabel(blk.tip)
+        about.setObjectName("Muted")
+        about.setWordWrap(True)
+        v.addWidget(about)
+        if fx.PRESETS.get(blk.key):
+            pg = QGridLayout()
+            pg.setSpacing(4)
+            for i, (pname, _) in enumerate(fx.PRESETS[blk.key]):
+                b = QPushButton(pname)
+                b.setToolTip("Turn %s on with the values '%s' (then adjust by ear)" % (blk.label, pname))
+                b.clicked.connect(lambda _=False, k=blk.key, n=pname: self.apply_preset(k, n))
+                pg.addWidget(b, i // 2, i % 2)
+            v.addLayout(pg)
+        for p in blk.params:
+            c = ParamControl(p)
+            c.set_value(p.default)
+            c.changed.connect(lambda k=blk.key: self._param_changed(k))
+            v.addWidget(c)
+            self.controls[(blk.key, p.key)] = c
+        v.addStretch(1)
+        return page
+
+    # --- on / off and the selected effect ----------------------------------------------------------
+    def is_on(self, key):
+        return self.items[key].checkState() == Qt.CheckState.Checked
+
+    def set_on(self, key, on):
+        self.items[key].setCheckState(Qt.CheckState.Checked if on else Qt.CheckState.Unchecked)
+
+    def show_block(self, key):
+        self.stack.setCurrentWidget(self.pages[key])
+        if self.chain.currentItem() is not self.items[key]:
+            self.chain.setCurrentItem(self.items[key])
+
+    def _style(self, key):
+        f = QFont(self.chain.font())
+        f.setBold(self.is_on(key))                 # effects that are on stand out in the chain
+        self.items[key].setFont(f)
+
+    def _item_changed(self, it):
+        key = it.data(Qt.ItemDataRole.UserRole)
+        self.chain.blockSignals(True)
+        self._style(key)
+        self.chain.blockSignals(False)
+        self._emit()
+
+    def _param_changed(self, key):
+        if not self.is_on(key):
+            self.set_on(key, True)                 # emits through _item_changed
+        else:
+            self._emit()
+
+    def set_channels(self, n):
+        layout = fx.LAYOUTS.get(n, "")
+        self.chain.blockSignals(True)
+        self.items["channels"].setText("Channels (%s)" % ("mono: no effect" if n <= 1 else "%d ch, %s" % (n, layout)))
+        self.chain.blockSignals(False)
+
+    # --- settings ----------------------------------------------------------------------------------
+    def _emit(self):
+        if not self.signalsBlocked():
+            self.changed.emit(self.settings())
+
+    def settings(self):
+        out = {}
+        for b in fx.BLOCKS:
+            d = {"on": self.is_on(b.key)}
+            for p in b.params:
+                d[p.key] = self.controls[(b.key, p.key)].value()
+            out[b.key] = d
+        return fx.normalize(out)
+
+    def set_settings(self, settings):
+        s = fx.normalize(settings)
+        self.blockSignals(True)
+        self.chain.blockSignals(True)
+        for b in fx.BLOCKS:
+            self.set_on(b.key, s[b.key]["on"])
+            self._style(b.key)
+            for p in b.params:
+                self.controls[(b.key, p.key)].set_value(s[b.key][p.key])
+        self.chain.blockSignals(False)
+        self.blockSignals(False)
+
+    def apply_preset(self, block, name):
+        self.set_settings(fx.with_preset(self.settings(), block, name))
+        self.show_block(block)
+        self._emit()
+
+    def apply_look(self, name):
+        self.set_settings(fx.look(name))
+        on = [b.key for b in fx.BLOCKS if self.is_on(b.key)]
+        if on:
+            self.show_block(on[0])
+        self._emit()
+
+    def _preset_chosen(self, i):
+        if i > 0:
+            self.apply_look(self.cmb_preset.itemText(i))
+        self.cmb_preset.setCurrentIndex(0)
+
+    def paste(self):
+        try:
+            s = json.loads(QApplication.clipboard().text())
+        except ValueError:
+            return
+        if isinstance(s, dict) and any(k in s for k in fx.BY_KEY):
+            self.set_settings(s)
+            self._emit()

app/icons.py

@@ -146,6 +146,25 @@ def _draw(p, name, c, s):
         p.drawPath(half)
         p.setBrush(Qt.BrushStyle.NoBrush)
         p.drawEllipse(rect)
+    elif name in ("prev", "next"):                    # chevron
+        d = -1 if name == "prev" else 1
+        x0 = s / 2 - d * s * 0.12
+        p.drawLine(QPointF(x0, s * 0.24), QPointF(x0 + d * s * 0.24, s / 2))
+        p.drawLine(QPointF(x0 + d * s * 0.24, s / 2), QPointF(x0, s * 0.76))
+    elif name == "add":                               # plus
+        p.drawLine(QPointF(s / 2, s * 0.22), QPointF(s / 2, s * 0.78))
+        p.drawLine(QPointF(s * 0.22, s / 2), QPointF(s * 0.78, s / 2))
+    elif name == "edit":                              # pencil
+        p.translate(s / 2, s / 2)
+        p.rotate(-45)
+        p.drawRect(QRectF(-s * 0.30, -s * 0.09, s * 0.48, s * 0.18))
+        path = QPainterPath()
+        path.moveTo(s * 0.18, -s * 0.09)
+        path.lineTo(s * 0.36, 0)
+        path.lineTo(s * 0.18, s * 0.09)
+        p.drawPath(path)
+        p.rotate(45)
+        p.translate(-s / 2, -s / 2)
     elif name == "search":
         p.drawEllipse(QRectF(s * 0.18, s * 0.18, s * 0.46, s * 0.46))
         p.drawLine(QPointF(s * 0.58, s * 0.58), QPointF(s * 0.84, s * 0.84))

app/main.py

@@ -44,12 +44,27 @@ def smoke_test(game_dir=None, report=None):
         ws = Workspace(Game(gd), store_root=os.path.join(td, "store"))
         s = next(x for x in ws.catalog.sounds if x.group == "Speech" and 1.0 < x.seconds < 4.0)
         x, sr = audio.decode_wem(ws.original_bytes(s))
-        wem, info = ws.prepare(s, x[::-1].copy(), sr)
+        from ackit import fx                  # effects (pedalboard) on the way, as the Edit tab does
+        st = fx.defaults()
+        for k in ("pitch", "eq", "reverb"):
+            st[k]["on"] = True
+        st["pitch"]["shift"] = -3.0
+        y = fx.render(x[::-1].copy(), sr, st, s.channels)
+        print("effects OK: %s (pedalboard %s), %.2f s -> %.2f s"
+              % (fx.summary(st), __import__("pedalboard").__version__, len(x) / sr, len(y) / sr))
+        wem, info = ws.prepare(s, y, sr)
         out = os.path.join(td, os.path.basename(ws.wad_path(s)))
         inject(ws.wad_path(s), {s.target: wem}, {s.target: ws.limit(s)}, out_path=out)
         assert read_target(out, s.target) == wem
         print("pipeline OK: %s (%s), %d sounds loaded, re-encoded %d bytes (limit %d), injected into a temp copy"
               % (s.name, s.archive, len(ws.catalog.sounds), len(wem), ws.limit(s)))
+        lang = ws.game.language()
+        b = next((x for x in ws.catalog.sounds if x.archive == "BANK"), None)
+        if b:
+            audio.decode_wem(ws.current_bytes(b))
+        subs = ws.texts.subtitles(lang) if lang in ws.texts.subtitle_languages() else []
+        print("texts OK: %d subtitles, %d menu texts (%s); in-memory sound decoded: %s"
+              % (len(subs), len(ws.texts.menu(lang)), lang, b.name if b else "none"))
     return 0
 
 

app/mainwindow.py

@@ -1,30 +1,48 @@
-"""Main window: filters + sound table (browser) on top, single-sound editor below."""
+"""Main window. Tabs: Sounds (browse: filters + sound table, a preview below; app/preview.py), Edit (the edit list,
+the editor and the effects: app/edit.py; the only place where sounds are replaced), Subtitles and Menu text
+(app/textpanel.py)."""
 import os
 
-from PySide6.QtCore import QByteArray, QEvent, QItemSelectionModel, QSize, Qt, QTimer
+from PySide6.QtCore import QByteArray, QItemSelectionModel, QSize, Qt, QTimer
 from PySide6.QtGui import QFont, QKeySequence, QShortcut
 from PySide6.QtWidgets import (QAbstractItemView, QCheckBox, QComboBox, QDialog, QFileDialog, QFormLayout,
                                QHBoxLayout, QHeaderView, QLabel, QLineEdit, QMainWindow, QMenu, QMessageBox,
-                               QPushButton, QScrollArea, QSplitter, QTableView, QToolBar, QVBoxLayout, QWidget)
+                               QPushButton, QSplitter, QTableView, QTabWidget, QToolBar, QVBoxLayout,
+                               QWidget)
 
 from ackit import settings as settings_mod, workspace as W
+from ackit.catalog import available_archives
 from ackit.game import Game, GameError, default_game_dir
-from ackit.modpack import export_mod, match_mod, prepare_import
+from ackit.modpack import export_mod, match_mod, match_texts, prepare_import
+from ackit.texts import TextConflictError
 from . import icons, theme
 from .audioio import Player, find_device, output_devices
 from .dialogs import ExportModDialog, ImportModDialog, SettingsDialog, pick_game_dir
-from .editor import EditorPanel
+from .edit import EditPanel
 from .models import C_FAV, C_NAME, COLS, SoundFilter, SoundModel
-from .spinner import Spinner
+from .preview import PreviewPanel
+from .spinner import BusyOverlay, Spinner
+from .textpanel import TextPanel
 from .userdata import UserData
 from .workers import Runner
 
 SIDEBAR_MIN, SIDEBAR_MAX = 220, 320
-TABLE_MIN = 160                     # the sound list never gets smaller than this when fitting the editor
 
 ANY = "Any"
 
 
+def all_languages(ws):
+    """Every language the editor offers: speech WADs + subtitle and menu text languages."""
+    return sorted(set(ws.game.languages) | set(ws.texts.menu_languages()) | set(ws.texts.subtitle_languages()))
+
+
+def editor_languages(game, language=None):
+    """(language, voices): the selected language (default: the game's) and the speech archive used for it
+    (the language itself, or INT for languages the game has no voices for)."""
+    lang = (language or game.language() or "INT").upper()
+    return lang, game.voice_archive(lang)
+
+
 class MainWindow(QMainWindow):
     def __init__(self, settings=None, runner=None, game_dir=None):
         super().__init__()
@@ -35,6 +53,7 @@ class MainWindow(QMainWindow):
         self.player.error.connect(lambda m: self.statusBar().showMessage(m, 5000))
         self.userdata = UserData()
         self.ws = None
+        self.language = None              # the editor's language (voices, subtitles, menu text)
         self.loading = False
         self._build()
         self._apply_output_device()
@@ -45,7 +64,7 @@ class MainWindow(QMainWindow):
             self.resize(1400, 900)
         gd = game_dir or default_game_dir(self.settings)
         if gd:
-            QTimer.singleShot(0, lambda: self.load_game(gd, self.settings.get("language")))
+            QTimer.singleShot(0, lambda: self.load_game(gd))     # always starts in the game's language
         else:
             self.show_no_game()
 
@@ -69,14 +88,19 @@ class MainWindow(QMainWindow):
         tb.addWidget(title)
         tb.addSeparator()
         self.cmb_lang = QComboBox()
-        self.cmb_lang.setToolTip("Speech language archive to browse")
+        self.cmb_lang.setToolTip("The one language setting of the editor: the voice lines in the Sounds tab, the "
+                                 "subtitles and the menu text.\nLanguages without their own voices (e.g. JPN, RUS) "
+                                 "use the English (INT) voice lines, like the game.")
         self.cmb_lang.activated.connect(lambda _: self.load_game(self.ws.game.root, self.cmb_lang.currentText())
                                         if self.ws else None)
         tb.addWidget(QLabel(" Language "))
         tb.addWidget(self.cmb_lang)
+        self.lbl_voices = QLabel()
+        self.lbl_voices.setObjectName("Muted")
+        tb.addWidget(self.lbl_voices)
         tb.addSeparator()
         self.act_revert_all = tb.addAction(icons.icon("revert"), "Revert all", self.revert_all)
-        self.act_revert_all.setToolTip("Put every original sound back")
+        self.act_revert_all.setToolTip("Put every original sound and text back (all languages)")
         tb.addAction(icons.icon("export"), "Export mod…", self.export_mod_dialog)
         tb.addAction(icons.icon("import"), "Import mod…", self.import_mod_dialog)
         tb.addSeparator()
@@ -157,6 +181,7 @@ class MainWindow(QMainWindow):
         self.proxy.setSourceModel(self.model)
         self.table = QTableView()
         self.table.setModel(self.proxy)
+        self.table_busy = BusyOverlay(self.table)   # while the sound list loads (start, language change)
         self.table.setSortingEnabled(True)
         self.table.setSelectionBehavior(QAbstractItemView.SelectionBehavior.SelectRows)
         self.table.setSelectionMode(QAbstractItemView.SelectionMode.ExtendedSelection)
@@ -172,40 +197,54 @@ class MainWindow(QMainWindow):
         self.table.sortByColumn(C_NAME, Qt.SortOrder.AscendingOrder)
         self.table.selectionModel().currentRowChanged.connect(self._current_changed)
         self.table.clicked.connect(self._clicked)
-        self.table.doubleClicked.connect(lambda _: self.editor.play("orig"))
+        self.table.doubleClicked.connect(lambda _: self.preview.play("orig"))
         self.table.setContextMenuPolicy(Qt.ContextMenuPolicy.CustomContextMenu)
         self.table.customContextMenuRequested.connect(self._context_menu)
 
-        # editor
+        # preview of the selected sound (no editing here)
-        self.editor = EditorPanel(self)
+        self.preview = PreviewPanel(self)
-        self.editor.applied.connect(self._after_change)
+        self.preview.edit_requested.connect(self.edit_selected)
-        self.editor.reverted.connect(self._after_change)
+        self.preview.add_requested.connect(self.add_selected_to_edit)
-        self.editor.message.connect(lambda m: self.statusBar().showMessage(m, 8000))
-        scroll = QScrollArea()
-        scroll.setWidget(self.editor)
-        scroll.setWidgetResizable(True)
 
-        self.vsplit = QSplitter(Qt.Orientation.Vertical)
+        # sidebar | sound list on top, the preview below across the whole width
-        self.vsplit.addWidget(self.table)
-        self.vsplit.addWidget(scroll)
-        self.vsplit.setStretchFactor(0, 1)       # extra height goes to the list; the editor keeps its fitted height
-        self.vsplit.setStretchFactor(1, 0)
-        self.scroll = scroll
-        self._fitted = False
-        self._user_sized = False                  # once the divider is dragged, keep the user's choice
-        self.vsplit.splitterMoved.connect(lambda *_: setattr(self, "_user_sized", True))
-        self.editor.installEventFilter(self)
         hs = QSplitter(Qt.Orientation.Horizontal)
         hs.addWidget(side)
-        hs.addWidget(self.vsplit)
+        hs.addWidget(self.table)
         hs.setStretchFactor(1, 1)
         hs.setSizes([SIDEBAR_MAX, 100000])
+        self.vsplit = QSplitter(Qt.Orientation.Vertical)
+        self.vsplit.addWidget(hs)
+        self.vsplit.addWidget(self.preview)
+        self.vsplit.setStretchFactor(0, 1)       # extra height goes to the list
+        self.vsplit.setStretchFactor(1, 0)
+        self.vsplit.setChildrenCollapsible(False)
+        # sections: sounds (browse), edit, subtitles, menu text
+        self.tabs = QTabWidget()
+        self.tabs.setDocumentMode(True)
+        self.tabs.addTab(self.vsplit, icons.icon("wave"), "Sounds")
+        self.edit = EditPanel(self)
+        self.editor = self.edit.editor            # the one sound editor
+        self.tabs.addTab(self.edit, icons.icon("edit"), "Edit")
+        self.edit.message.connect(lambda m: self.statusBar().showMessage(m, 8000))
+        self.editor.message.connect(lambda m: self.statusBar().showMessage(m, 8000))
+        self.subs = TextPanel(self, "subs")
+        self.menu = TextPanel(self, "menu")
+        self.tabs.addTab(self.subs, icons.icon("ab"), "Subtitles")
+        self.tabs.addTab(self.menu, icons.icon("settings"), "Menu text")
+        self._text_loaded = set()
+        for panel in (self.subs, self.menu):
+            panel.message.connect(lambda m: self.statusBar().showMessage(m, 8000))
+            panel.changed.connect(self._texts_changed)
+        self.subs.go_to_sound.connect(self.show_sound)
+        self.editor.subtitle.changed.connect(self._texts_changed)
+        self.editor.subtitle.open_in_list.connect(self.show_subtitle)
+        self.tabs.currentChanged.connect(self._tab_changed)
         central = QWidget()
         cl = QVBoxLayout(central)
         cl.setContentsMargins(0, 0, 0, 0)
         cl.setSpacing(0)
         cl.addWidget(self.banner)
-        cl.addWidget(hs, 1)
+        cl.addWidget(self.tabs, 1)
         self.setCentralWidget(central)
         self.spinner = Spinner(14)
         self.statusBar().addWidget(self.spinner)
@@ -214,12 +253,15 @@ class MainWindow(QMainWindow):
 
         sc = Qt.ShortcutContext.WindowShortcut
         QShortcut(QKeySequence("Ctrl+F"), self, self.search.setFocus, context=sc)
-        QShortcut(QKeySequence("Ctrl+S"), self, lambda: self.editor.apply(), context=sc)
+        edit = Qt.ShortcutContext.WidgetWithChildrenShortcut          # editing shortcuts: in the Edit tab only
-        QShortcut(QKeySequence("Ctrl+O"), self, lambda: self.editor.load_file_dialog(), context=sc)
+        QShortcut(QKeySequence("Ctrl+S"), self.edit, lambda: self.editor.apply(), context=edit)
-        QShortcut(QKeySequence("Escape"), self, self.editor.stop, context=sc)
+        QShortcut(QKeySequence("Ctrl+O"), self.edit, lambda: self.editor.load_file_dialog(), context=edit)
-        QShortcut(QKeySequence(Qt.Key.Key_Space), self.table, self.editor.toggle_play,
+        QShortcut(QKeySequence("Escape"), self, self.player.stop, context=sc)
-                  context=Qt.ShortcutContext.WidgetShortcut)
+        wsc = Qt.ShortcutContext.WidgetShortcut
-        QShortcut(QKeySequence(Qt.Key.Key_F2), self.table, self.edit_notes, context=Qt.ShortcutContext.WidgetShortcut)
+        QShortcut(QKeySequence(Qt.Key.Key_Space), self.table, self.preview.toggle_play, context=wsc)
+        QShortcut(QKeySequence(Qt.Key.Key_Return), self.table, self.edit_selected, context=wsc)
+        QShortcut(QKeySequence(Qt.Key.Key_Enter), self.table, self.edit_selected, context=wsc)
+        QShortcut(QKeySequence(Qt.Key.Key_F2), self.table, self.edit_notes, context=wsc)
 
     # --- dialogs (overridable in tests) -----------------------------------------------------------
     def confirm(self, title, text):
@@ -249,30 +291,36 @@ class MainWindow(QMainWindow):
         except GameError as e:
             self.show_no_game(str(e))
             return
-        lang = language if language in game.languages else game.language()
+        lang, voices = editor_languages(game, language)
         self.loading = True
         self.statusBar().showMessage("Loading sound list…")
         self.spinner.show()
+        self.table_busy.start("Loading sounds…")
         store = self.settings.get("store_root")
         full = self.settings.get("full_backup", True)
 
         def work(job):
-            return W.Workspace(game, store, [lang, "SFX"] if "SFX" in game.archives else [lang], full_backup=full,
+            avail = available_archives(game)
-                               progress=lambda n, t: job.report("scanning %s: %d / %d" % (lang, n, t)))
+            return W.Workspace(game, store, [a for a in (voices, "SFX", "BANK") if a in avail], full_backup=full,
+                               progress=lambda n, t: job.report("scanning game audio: %d / %d" % (n, t)))
 
         def done(ws):
             self.loading = False
             self.spinner.hide()
+            self.table_busy.stop()
             self.ws = ws
             self.settings["game_dir"] = game.root
-            self.settings["language"] = lang
             self.save_settings()
+            self.language = lang
             self.cmb_lang.blockSignals(True)
             self.cmb_lang.clear()
-            self.cmb_lang.addItems(game.languages)
+            self.cmb_lang.addItems(all_languages(ws))
             self.cmb_lang.setCurrentText(lang)
             self.cmb_lang.blockSignals(False)
+            self.lbl_voices.setText("" if voices == lang else "  voices: %s (no %s voices in the game)" % (voices, lang))
             self.populate()
+            self._text_loaded.clear()
+            self._tab_changed(self.tabs.currentIndex())
             self.banner.hide()
             self.lbl_game.setText("%s  •  store: %s" % (game.root, ws.store.root))
             self.statusBar().showMessage("Loaded %d sounds (%s)." % (len(ws.catalog.sounds), ", ".join(ws.catalog.archives)), 6000)
@@ -282,6 +330,7 @@ class MainWindow(QMainWindow):
         def failed(e, tb):
             self.loading = False
             self.spinner.hide()
+            self.table_busy.stop()
             self.show_no_game("Could not load the game audio: %s" % e)
 
         self.runner.start(work, done, failed, lambda m: self.statusBar().showMessage(m))
@@ -307,6 +356,7 @@ class MainWindow(QMainWindow):
         fill(self.cmb_arch, list(ws.catalog.archives))
         fill(self.cmb_tag, self.userdata.all_tags())
         self._filters_changed()
+        self.edit.populate(ws)
 
     def _fill_categories(self):
         g = self.cmb_group.currentText()
@@ -357,7 +407,22 @@ class MainWindow(QMainWindow):
 
     def _current_changed(self, cur, prev):
         if cur.isValid():
-            self.editor.set_sound(self.proxy.sound_at(cur))
+            self.preview.set_sound(self.proxy.sound_at(cur))
+
+    def edit_selected(self):
+        """Selected sounds onto the edit list; the Edit tab opens on the first."""
+        sel = self.selected_sounds() or ([self.preview.sound] if self.preview.sound else [])
+        if sel:
+            self.edit_sounds(sel)
+
+    def edit_sounds(self, sounds):
+        self.edit.add(sounds, open_first=True)
+        self.tabs.setCurrentWidget(self.edit)
+
+    def add_selected_to_edit(self):
+        sel = self.selected_sounds()
+        self.edit.add(sel)
+        self.statusBar().showMessage("%d sound(s) on the edit list (Edit tab)" % len(sel), 5000)
 
     def _clicked(self, index):
         if index.column() == C_FAV:
@@ -381,7 +446,10 @@ class MainWindow(QMainWindow):
         if not sel:
             return
         m = QMenu(self)
-        m.addAction("Play original", lambda: self.editor.play("orig"))
+        m.addAction(icons.icon("edit"), "Edit  (Enter)", self.edit_selected)
+        m.addAction(icons.icon("add"), "Add to edit list", self.add_selected_to_edit)
+        m.addSeparator()
+        m.addAction("Play original  (Space)", lambda: self.preview.play("orig"))
         m.addAction("Notes && tags…  (F2)", self.edit_notes)
         m.addAction("Toggle favorite", lambda: self._toggle_fav(sel))
         m.addSeparator()
@@ -432,11 +500,49 @@ class MainWindow(QMainWindow):
         self.cmb_tag.setCurrentText(cur)
         self.cmb_tag.blockSignals(False)
 
+    # --- text sections --------------------------------------------------------------------------
+    def _tab_changed(self, i):
+        panel = self.tabs.widget(i)
+        if self.ws and panel in (self.subs, self.menu) and panel.kind not in self._text_loaded:
+            self._text_loaded.add(panel.kind)
+            panel.load()
+
+    def _texts_changed(self):
+        for panel in (self.subs, self.menu):
+            if panel.kind in self._text_loaded:
+                panel.refresh()
+        self.editor.subtitle.set_sound(self.editor.sound)
+        if self.preview.sound:
+            self.preview.set_sound(self.preview.sound)
+
+    def show_sound(self, key):
+        """Sounds tab with `key` selected (from a subtitle)."""
+        s = self.ws.catalog.by_key.get(key) if self.ws else None
+        if not s:
+            self.inform("Sound", "%s is not in the sounds loaded now (choose its language at the top)." % key)
+            return
+        self.tabs.setCurrentIndex(0)
+        self.clear_filters()
+        self.select_name(s.name)
+
+    def show_subtitle(self, key):
+        """Subtitles tab with `key` selected (from the sound editor)."""
+        self.tabs.setCurrentWidget(self.subs)
+        self.subs.search.clear()
+        self.subs.cb_mod.setChecked(False)
+        for cb in self.subs.filters.values():
+            cb.setCurrentIndex(0)
+        self.subs._filter()
+        self.subs.select_key(key)
+
     # --- changes ----------------------------------------------------------------------------------
     def _after_change(self, key=None):
-        self.model.refresh_keys(modified=self.ws.modified_keys())
+        modified = self.ws.modified_keys()
-        self.proxy.refilter()
+        self.model.refresh_keys(modified=modified)     # the view refilters/re-sorts itself
         self._update_count()
+        self.edit.refresh(modified)
+        if self.preview.sound:
+            self.preview.set_sound(self.preview.sound)
 
     def revert_sounds(self, sounds):
         sounds = [s for s in sounds if self.ws.store.get(s.key)]
@@ -447,34 +553,52 @@ class MainWindow(QMainWindow):
                          "Reverted %d sound(s)." % len(sounds))
 
     def revert_all(self):
+        """Every change of this store: sounds, subtitles and menu texts (every language)."""
         if not self.ws:
             return
-        n = len(self.ws.store.entries)
+        n, t = len(self.ws.store.entries), len(self.ws.texts.records())
-        if not n:
+        if not n and not t:
-            self.inform("Revert all", "All sounds are already original.")
+            self.inform("Revert all", "All sounds and texts are already original.")
             return
-        if not self.confirm("Revert all", "Put the original back for all %d changed sound(s)?" % n):
+        if not self.confirm("Revert all", "Put the original back for all %d changed sound(s) and %d changed "
+                                          "text(s)?" % (n, t)):
             return
-        self._run_change("Reverting everything…", lambda job: self.ws.revert_all(job.report),
-                         "All sounds are original again.")
 
-    def _run_change(self, busy, fn, ok_msg):
+        def work(job):
+            self.ws.revert_all(job.report)
+            self.ws.texts.revert_all(job.report)
+        self._run_change("Reverting everything…", work, "All sounds and texts are original again.")
+
+    def _run_change(self, busy, fn, ok_msg, ask_force=False):
+        """Run a change as a background job. ask_force: fn(job, force); when files were changed outside this
+        tool, ask the user and run it again with force=True."""
         self.statusBar().showMessage(busy)
         self.spinner.show()
 
         def done(_):
             self.spinner.hide()
             self.statusBar().showMessage(ok_msg, 8000)
-            self._after_change()
+            self._refresh_after_change()
-            if self.editor.sound:
-                self.editor.set_sound(self.editor.sound)
             self.check_problems()
 
         def failed(e, tb):
             self.spinner.hide()
+            if self.ws:
+                self._refresh_after_change()          # part of the change may have been written
+            if ask_force and isinstance(e, (W.ConflictError, TextConflictError)):
+                if self.confirm("Changed outside this tool", "%s\n\nOverwrite them?" % e):
+                    self._run_change(busy, lambda job: fn(job, True), ok_msg)
+                return
             self.inform("Error", str(e))
 
-        self.runner.start(fn, done, failed, lambda m: self.statusBar().showMessage(m))
+        self.runner.start((lambda job: fn(job, False)) if ask_force else fn, done, failed,
+                          lambda m: self.statusBar().showMessage(m))
+
+    def _refresh_after_change(self):
+        self._after_change()
+        self._texts_changed()
+        if self.editor.sound:
+            self.edit.load(self.editor.sound)
 
     # --- changed underneath -----------------------------------------------------------------------
     def check_problems(self):
@@ -491,11 +615,20 @@ class MainWindow(QMainWindow):
                 st = ws.state(s)
                 if st in (W.LOST, W.CHANGED):
                     out.append((s, st))
-            return out
+            texts = []
+            for key in list(ws.texts.records()):
+                try:
+                    st = ws.texts.state(key)
+                except (GameError, OSError, ValueError):
+                    continue
+                if st in (W.LOST, W.CHANGED):
+                    texts.append((key, st))
+            return out, texts
 
-        def done(problems):
+        def done(res):
-            self.problems = problems
+            problems, texts = res
-            if not problems:
+            self.problems, self.text_problems = problems, texts
+            if not problems and not texts:
                 if self.banner.isVisible() and not self.b_choose.isVisible():
                     self.banner.hide()
                 return
@@ -508,6 +641,9 @@ class MainWindow(QMainWindow):
             if changed:
                 txt.append("%d sound(s) were changed by something else since you modified them: %s. "
                            "Revert puts the original back." % (len(changed), ", ".join(changed[:4])))
+            if texts:
+                txt.append("%d of your text changes were undone or changed outside this editor: %s"
+                           % (len(texts), ", ".join(k for k, _ in texts[:3]) + (" …" if len(texts) > 3 else "")))
             self.banner_text.setText("\n".join(txt))
             self.b_reapply.show()
             self.b_forget.show()
@@ -520,17 +656,22 @@ class MainWindow(QMainWindow):
         ws = self.ws
         items = [(s, ws.store.get_blob(ws.store.get(s.key)["applied"]), ws.store.get(s.key).get("meta", {}))
                  for s, st in getattr(self, "problems", []) if ws.store.get(s.key) and ws.store.get(s.key).get("applied")]
+        texts = {k: ws.texts.records()[k]["applied"] for k, st in getattr(self, "text_problems", [])
+                 if k in ws.texts.records() and ws.texts.records()[k].get("applied") is not None}
+
+        def work(job):
             if items:
-            self._run_change("Re-applying…", lambda job: ws.apply(items, force=True, progress=job.report),
+                ws.apply(items, force=True, progress=job.report)
-                             "Re-applied %d change(s)." % len(items))
+            if texts:
+                ws.texts.apply(texts, force=True, progress=job.report)
+        if items or texts:
+            self._run_change("Re-applying…", work, "Re-applied %d change(s)." % (len(items) + len(texts)))
 
     def forget_problems(self):
-        if not self.confirm("Forget", "Stop tracking these sounds? Their current game files are kept as they are."):
+        if not self.confirm("Forget", "Stop tracking these changes? The current game files are kept as they are."):
             return
-        for s, _ in getattr(self, "problems", []):
+        self.ws.forget([s.key for s, _ in getattr(self, "problems", [])],
-            self.ws.store.forget(s.key)
+                       [k for k, _ in getattr(self, "text_problems", [])])
-        self.ws.store.save()
-        self.ws.store.gc()
         self.banner.hide()
         self._after_change()
 
@@ -539,11 +680,13 @@ class MainWindow(QMainWindow):
         if not self.ws:
             return
         mods = self.ws.modified_keys()
-        if not mods:
+        texts = self.ws.texts.modified()
-            self.inform("Export mod", "You have not changed any sound yet.")
+        if not mods and not texts:
+            self.inform("Export mod", "You have not changed any sound or text yet.")
             return
         if keys is None:
             names = [(k, self.ws.catalog.by_key[k].name if k in self.ws.catalog.by_key else k) for k in mods]
+            names += [(k, "Text  " + k) for k in texts]
             dlg = ExportModDialog(names, self)
             if dlg.exec() != QDialog.DialogCode.Accepted:
                 return
@@ -553,8 +696,10 @@ class MainWindow(QMainWindow):
                 self.settings.get("last_dir") or "", (title or "my_mod") + ".acmod"), "Unheard City mod (*.acmod)")
             if not path:
                 return
-        self._run_change("Exporting…", lambda job: export_mod(self.ws, path, keys, title, author, description),
+        text_keys = [k for k in keys if k.startswith(("SUB:", "MENU:"))]
-                         "Saved %s" % path)
+        keys = [k for k in keys if k not in text_keys]
+        self._run_change("Exporting…", lambda job: export_mod(self.ws, path, keys, title, author, description,
+                                                               texts=text_keys), "Saved %s" % path)
 
     def import_mod_dialog(self, path=None, confirm=True):
         if not self.ws:
@@ -569,20 +714,26 @@ class MainWindow(QMainWindow):
         except Exception as e:  # noqa: BLE001
             self.inform("Import mod", "Cannot read this package: %s" % e)
             return
-        if confirm and ImportModDialog(man, items, self).exec() != QDialog.DialogCode.Accepted:
+        dlg = ImportModDialog(man, items, self, match_texts(self.ws, man)) if confirm else None
+        if dlg and dlg.exec() != QDialog.DialogCode.Accepted:
             return
         ws = self.ws
+        prepared = []
 
-        def work(job):
+        def work(job, force):
+            if not prepared:                      # encode once (a confirmed overwrite runs this again)
                 prepare_import(ws, path, items, progress=job.report)
-            changes = [(im.sound, im.wem, {"source": "mod:" + os.path.basename(path)}) for im in items if im.wem]
+                prepared.append(True)
-            try:
+            source = {"source": "mod:" + os.path.basename(path)}
-                ws.apply(changes, progress=job.report)
+            changes = [(im.sound, im.wem, source) for im in items if im.wem]
-            except W.ConflictError:
+            if changes:
-                ws.apply(changes, force=True, progress=job.report)
+                ws.apply(changes, force=force, progress=job.report)
-            return len(changes)
+            texts = {k: v for k, v, problem in match_texts(ws, man) if not problem}
+            if texts:
+                ws.texts.apply(texts, force=force, progress=job.report, meta=source)
+            return len(changes) + len(texts)
 
-        self._run_change("Installing mod…", work, "Installed %s" % os.path.basename(path))
+        self._run_change("Installing mod…", work, "Installed %s" % os.path.basename(path), ask_force=True)
 
     # --- settings ---------------------------------------------------------------------------------
     def settings_dialog(self):
@@ -590,14 +741,23 @@ class MainWindow(QMainWindow):
         if dlg.exec() != QDialog.DialogCode.Accepted:
             return
         old = dict(self.settings)
-        self.settings.update(dlg.values())
+        new = dlg.values()
+        if self.ws and (new.get("store_root") or None) != (old.get("store_root") or None) and (
+                self.ws.store.entries or self.ws.texts.records()):
+            # the changes are recorded in the current store only; another store would take the modified files
+            # as originals (and could not revert these changes)
+            self.inform("Settings", "The originals store keeps the originals of the changes you have applied. "
+                                    "Use Revert all before choosing another store folder; the store was not "
+                                    "changed.")
+            new["store_root"] = old.get("store_root")
+        self.settings.update(new)
         self.save_settings()
         self._apply_output_device()
         self.sidebar.configure(self.settings)
         if (self.settings.get("game_dir") != old.get("game_dir") or self.settings.get("store_root") != old.get("store_root")
                 or self.settings.get("full_backup") != old.get("full_backup")):
             if self.settings.get("game_dir"):
-                self.load_game(self.settings["game_dir"], self.settings.get("language"))
+                self.load_game(self.settings["game_dir"], self.language)
 
     def _apply_output_device(self):
         name = self.settings.get("output_device")
@@ -610,31 +770,8 @@ class MainWindow(QMainWindow):
         except OSError:
             pass
 
-    def fit_editor(self):
-        """Give the editor panel the height of its contents (everything visible), the list gets the rest."""
-        total = sum(self.vsplit.sizes()) or self.vsplit.height()
-        lay = self.editor.layout()
-        need = self.editor.sizeHint().height()
-        if lay.hasHeightForWidth():               # wrapped labels: height at the actual width
-            need = max(need, lay.heightForWidth(self.scroll.viewport().width()))
-        need += 2 * self.scroll.frameWidth()
-        if self.editor.minimumSizeHint().width() > self.scroll.viewport().width():   # narrow window: room for the h-bar
-            need += self.scroll.horizontalScrollBar().sizeHint().height()
-        editor_h = min(need, max(0, total - TABLE_MIN))
-        self.vsplit.setSizes([total - editor_h, editor_h])
-
-    def eventFilter(self, obj, e):
-        if obj is self.editor and e.type() == QEvent.Type.LayoutRequest and self._fitted and not self._user_sized:
-            QTimer.singleShot(0, self.fit_editor)
-        return super().eventFilter(obj, e)
-
-    def showEvent(self, e):
-        super().showEvent(e)
-        if not self._fitted:
-            self._fitted = True
-            QTimer.singleShot(0, self.fit_editor)
-
     def closeEvent(self, e):
         self.player.stop()
+        self.edit.flush()
         self.save_settings()
         super().closeEvent(e)

app/models.py

@@ -1,5 +1,5 @@
 """Table model for the sound browser + a combined filter (search text and facet filters)."""
-from PySide6.QtCore import QAbstractTableModel, QModelIndex, QSortFilterProxyModel, Qt
+from PySide6.QtCore import QAbstractProxyModel, QAbstractTableModel, QModelIndex, Qt
 from PySide6.QtGui import QFont, QPalette
 from PySide6.QtWidgets import QApplication
 
@@ -40,6 +40,44 @@ class SoundModel(QAbstractTableModel):
         if self.sounds:
             self.dataChanged.emit(self.index(0, 0), self.index(len(self.sounds) - 1, len(COLS) - 1))
 
+    def sort_key(self, row, c):
+        """Value a column sorts by (plain Python values: the view sorts all rows in one Python sort)."""
+        s, ud = self.sounds[row], self.userdata
+        if c == C_FAV:
+            return 0 if ud and ud.favorite(s.key) else 1
+        if c == C_LEN:
+            return s.seconds
+        if c == C_SIZE:
+            return 1e12 if s.is_pack_media else s.size
+        if c == C_FMT:
+            return (s.rate, s.channels)
+        if c == C_STATUS:
+            return 0 if s.key in self.modified else 1
+        return self.display(s, c).lower()
+
+    def display(self, s, c):
+        ud = self.userdata
+        if c == C_NAME:
+            return s.name
+        if c == C_CAT:
+            return s.category
+        if c == C_CHAR:
+            return s.character or (("~ " + s.characters_inferred) if s.characters_inferred else "")
+        if c == C_LEN:
+            return "%.2f s" % s.seconds
+        if c == C_ARCH:
+            return s.archive
+        if c == C_FMT:
+            return "%d Hz %s" % (s.rate, {1: "mono", 2: "stereo"}.get(s.channels, "%d ch" % s.channels))
+        if c == C_SIZE:
+            return "any" if s.is_pack_media else "%.1f KB" % (s.size / 1024.0)
+        if c == C_STATUS:
+            return "Modified" if s.key in self.modified else ""
+        if c == C_NOTES and ud:
+            t = ud.tags(s.key)
+            return " ".join("#" + x for x in t) + ("  " if t else "") + ud.note(s.key)
+        return ""
+
     def rowCount(self, parent=QModelIndex()):
         return 0 if parent.isValid() else len(self.sounds)
 
@@ -66,41 +104,9 @@ class SoundModel(QAbstractTableModel):
         if role == SOUND_ROLE:
             return s
         if role == Qt.ItemDataRole.DisplayRole:
-            if c == C_FAV:
+            return self.display(s, c)
-                return ""
-            if c == C_NAME:
-                return s.name
-            if c == C_CAT:
-                return s.category
-            if c == C_CHAR:
-                return s.character or (("~ " + s.characters_inferred) if s.characters_inferred else "")
-            if c == C_LEN:
-                return "%.2f s" % s.seconds
-            if c == C_ARCH:
-                return s.archive
-            if c == C_FMT:
-                return "%d Hz %s" % (s.rate, {1: "mono", 2: "stereo"}.get(s.channels, "%d ch" % s.channels))
-            if c == C_SIZE:
-                return "any" if s.is_pack_media else "%.1f KB" % (s.size / 1024.0)
-            if c == C_STATUS:
-                return "Modified" if s.key in self.modified else ""
-            if c == C_NOTES:
-                if not ud:
-                    return ""
-                t = ud.tags(s.key)
-                return " ".join("#" + x for x in t) + ("  " if t else "") + ud.note(s.key)
         if role == SORT_ROLE:
-            if c == C_FAV:
+            return self.sort_key(index.row(), c)
-                return 0 if ud and ud.favorite(s.key) else 1
-            if c == C_LEN:
-                return s.seconds
-            if c == C_SIZE:
-                return 1e12 if s.is_pack_media else s.size
-            if c == C_FMT:
-                return (s.rate, s.channels)
-            if c == C_STATUS:
-                return 0 if s.key in self.modified else 1
-            return (self.data(index, Qt.ItemDataRole.DisplayRole) or "").lower()
         if role == Qt.ItemDataRole.DecorationRole and c == C_FAV:
             fav = bool(ud and ud.favorite(s.key))
             return icons.icon("star" if fav else "star-off", icons.ACCENT if fav else "#4a4a4a")
@@ -122,9 +128,122 @@ class SoundModel(QAbstractTableModel):
         return None
 
 
-class SoundFilter(QSortFilterProxyModel):
+class RowView(QAbstractProxyModel):
+    """The rows a table shows: the source rows a subclass accepts, in the chosen order. Computed in Python in
+    one pass (filter, then one sort over cached keys): ~10 ms for 21,000 sounds, where QSortFilterProxyModel
+    needed up to ~1.5 s per sort or widened filter (each comparison called back into Python). Selection and the
+    current row survive every change (persistent indexes are moved, as Qt's own proxies do).
+    Subclasses: source_len(), accepts(row), sort_key(row, column)."""
+
     def __init__(self, parent=None):
         super().__init__(parent)
+        self.sort_column, self.sort_order = -1, Qt.SortOrder.AscendingOrder
+        self.rows = []                  # proxy row -> source row
+        self.pos = {}                   # source row -> proxy row
+        self._keys = {}                 # column -> [sort key per source row] (until the source changes)
+
+    def source_len(self):
+        return self.sourceModel().rowCount()
+
+    def accepts(self, row):
+        return True
+
+    def sort_key(self, row, column):
+        v = self.sourceModel().index(row, column).data()
+        return v.lower() if isinstance(v, str) else v
+
+    # --- source model --------------------------------------------------------------------------------
+    def setSourceModel(self, model):
+        """One source model for the view's lifetime."""
+        self.beginResetModel()
+        super().setSourceModel(model)
+        model.modelAboutToBeReset.connect(self.beginResetModel)
+        model.modelReset.connect(self._source_reset)
+        model.dataChanged.connect(self._source_changed)
+        self._recompute_all()
+        self.endResetModel()
+
+    def _recompute_all(self):
+        self._keys = {}
+        self.rows = self._compute()
+        self.pos = {r: i for i, r in enumerate(self.rows)}
+
+    def _source_reset(self):
+        self._recompute_all()
+        self.endResetModel()
+
+    def _source_changed(self, top_left, bottom_right, roles=()):
+        """Values changed (status, notes, favorites, texts): sort keys and filters may depend on them."""
+        self._keys = {}
+        self.refilter()
+        if self.rows:
+            self.dataChanged.emit(self.index(0, 0), self.index(len(self.rows) - 1, self.columnCount() - 1))
+
+    # --- filter and order ----------------------------------------------------------------------------
+    def refilter(self):
+        """Recompute the rows; the view keeps its selection and current row (layout change)."""
+        rows = self._compute()
+        if rows == self.rows:
+            return
+        self.layoutAboutToBeChanged.emit()
+        old = self.persistentIndexList()
+        src = [(self.rows[i.row()], i.column()) if i.isValid() and i.row() < len(self.rows) else None for i in old]
+        self.rows = rows
+        self.pos = {r: i for i, r in enumerate(rows)}
+        new = [self.index(self.pos[s[0]], s[1]) if s and s[0] in self.pos else QModelIndex() for s in src]
+        self.changePersistentIndexList(old, new)
+        self.layoutChanged.emit()
+
+    def sort(self, column, order=Qt.SortOrder.AscendingOrder):
+        self.sort_column, self.sort_order = column, order
+        self.refilter()
+
+    def _compute(self):
+        if self.sourceModel() is None:
+            return []
+        n = self.source_len()
+        rows = [r for r in range(n) if self.accepts(r)]
+        c = self.sort_column
+        if c >= 0:
+            if c not in self._keys:
+                self._keys[c] = [self.sort_key(r, c) for r in range(n)]
+            rows.sort(key=self._keys[c].__getitem__, reverse=self.sort_order == Qt.SortOrder.DescendingOrder)
+        return rows
+
+    # --- QAbstractProxyModel -------------------------------------------------------------------------
+    def index(self, row, column, parent=QModelIndex()):
+        if parent.isValid() or not (0 <= row < len(self.rows)) or not (0 <= column < self.columnCount()):
+            return QModelIndex()
+        return self.createIndex(row, column)
+
+    def parent(self, index=QModelIndex()):
+        return QModelIndex()
+
+    def rowCount(self, parent=QModelIndex()):
+        return 0 if parent.isValid() else len(self.rows)
+
+    def columnCount(self, parent=QModelIndex()):
+        m = self.sourceModel()
+        return 0 if parent.isValid() or m is None else m.columnCount()
+
+    def mapToSource(self, proxy_index):
+        if not proxy_index.isValid() or proxy_index.row() >= len(self.rows):
+            return QModelIndex()
+        return self.sourceModel().index(self.rows[proxy_index.row()], proxy_index.column())
+
+    def mapFromSource(self, source_index):
+        if not source_index.isValid() or source_index.row() not in self.pos:
+            return QModelIndex()
+        return self.index(self.pos[source_index.row()], source_index.column())
+
+    def headerData(self, section, orientation, role=Qt.ItemDataRole.DisplayRole):
+        return self.sourceModel().headerData(section, orientation, role)
+
+
+class SoundFilter(RowView):
+    """The sound browser's rows: search words and facet filters, sorted by any column."""
+
+    def __init__(self, parent=None):
         self.words = []
         self.group = None
         self.category = None
@@ -134,15 +253,7 @@ class SoundFilter(QSortFilterProxyModel):
         self.modified_only = False
         self.favorites_only = False
         self.tag = None
-        self.setSortRole(SORT_ROLE)
+        super().__init__(parent)
-        self.setDynamicSortFilter(False)
-
-    def refilter(self):
-        if hasattr(self, "beginFilterChange"):  # Qt >= 6.10
-            self.beginFilterChange()
-            self.endFilterChange(QSortFilterProxyModel.Direction.Rows)
-        else:
-            self.invalidateRowsFilter()
 
     def set(self, **kw):
         for k, v in kw.items():
@@ -153,7 +264,13 @@ class SoundFilter(QSortFilterProxyModel):
         self.words = [w for w in text.lower().replace("_", " ").split() if w]
         self.refilter()
 
-    def filterAcceptsRow(self, row, parent):
+    def source_len(self):
+        return len(self.sourceModel().sounds)
+
+    def sort_key(self, row, column):
+        return self.sourceModel().sort_key(row, column)
+
+    def accepts(self, row):
         m = self.sourceModel()
         s = m.sounds[row]
         if self.group and s.group != self.group:
@@ -181,4 +298,4 @@ class SoundFilter(QSortFilterProxyModel):
         return True
 
     def sound_at(self, proxy_index):
-        return self.sourceModel().sounds[self.mapToSource(proxy_index).row()]
+        return self.sourceModel().sounds[self.rows[proxy_index.row()]]

app/preview.py

@@ -0,0 +1,181 @@
+"""Sounds tab: preview of the selected sound (listen, read its subtitle) with the way into the Edit tab. No
+editing here: editing is the Edit tab's job."""
+from PySide6.QtCore import Qt, Signal
+from PySide6.QtWidgets import QGridLayout, QHBoxLayout, QLabel, QPushButton, QVBoxLayout, QWidget
+
+from ackit import audio, subtitles, workspace as W
+from . import icons, theme
+from .editor import STATE_TEXT
+from .waveform import Waveform
+
+
+class PreviewPanel(QWidget):
+    edit_requested = Signal()          # open the selected sounds in the Edit tab
+    add_requested = Signal()           # add them to the edit list (stay here)
+
+    def __init__(self, ctx, parent=None):
+        super().__init__(parent)
+        self.ctx = ctx                 # main window: .ws, .runner, .player, .language
+        self.sound = None
+        self.orig = self.installed = None
+        self.playing_what = None
+        self._build()
+        self.set_sound(None)
+
+    def _build(self):
+        self.setObjectName("EditorPanel")
+        lay = QHBoxLayout(self)
+        lay.setContentsMargins(8, 6, 8, 6)
+        lay.setSpacing(12)
+        info = QVBoxLayout()
+        info.setSpacing(3)
+        self.title = QLabel()
+        self.title.setObjectName("EditorTitle")
+        self.title.setTextInteractionFlags(Qt.TextInteractionFlag.TextSelectableByMouse)
+        f = self.title.font()
+        f.setPointSizeF(f.pointSizeF() * 1.2)
+        f.setBold(True)
+        self.title.setFont(f)
+        self.meta = QLabel()
+        self.meta.setObjectName("Muted")
+        self.meta.setWordWrap(True)
+        self.state = QLabel()
+        self.sub = QLabel()
+        self.sub.setWordWrap(True)
+        self.sub.setTextInteractionFlags(Qt.TextInteractionFlag.TextSelectableByMouse)
+        for w in (self.title, self.meta, self.state, self.sub):
+            info.addWidget(w)
+        info.addStretch(1)
+        lay.addLayout(info, 3)
+
+        # player: the transport buttons together, the waveforms under them (click = play from there)
+        player = QVBoxLayout()
+        player.setSpacing(4)
+        tr = QHBoxLayout()
+        tr.setSpacing(6)
+        self.b_orig = self._btn("Play original", "play", lambda: self.play("orig"), "Play the game's original (Space)",
+                                icons.STEEL)
+        self.b_inst = self._btn("Play installed", "play", lambda: self.play("inst"),
+                                "Play what the game has now (your replacement)", "#8f4a4e")
+        self.b_stop = self._btn("Stop", "stop", self.stop, "Stop (Esc)")
+        for b in (self.b_orig, self.b_inst, self.b_stop):
+            tr.addWidget(b)
+        tr.addStretch(1)
+        player.addLayout(tr)
+        waves = QGridLayout()
+        waves.setVerticalSpacing(3)
+        waves.setHorizontalSpacing(6)
+        self.wf_orig, self.wf_inst = Waveform(), Waveform()
+        for i, (w, kind, name) in enumerate(((self.wf_orig, "orig", "Original"), (self.wf_inst, "inst", "Installed"))):
+            w.accent, w.bg = theme.WAVE[kind], theme.WAVE_BG
+            w.setMinimumHeight(40)
+            lab = QLabel(name)
+            lab.setObjectName("Muted")
+            waves.addWidget(lab, i, 0)
+            waves.addWidget(w, i, 1)
+        waves.setColumnStretch(1, 1)
+        player.addLayout(waves)
+        self.wf_orig.seekRequested.connect(lambda t: self.play("orig", t))
+        self.wf_inst.seekRequested.connect(lambda t: self.play("inst", t))
+        lay.addLayout(player, 4)
+
+        # editing: into the Edit tab (on the right, apart from the player)
+        acts = QVBoxLayout()
+        acts.setSpacing(6)
+        self.b_edit = self._btn("Edit", "edit", self.edit_requested.emit,
+                                "Open the selected sound(s) in the Edit tab (Enter)", "#ffffff")
+        self.b_edit.setObjectName("Primary")
+        self.b_add = self._btn("Add to edit list", "add", self.add_requested.emit,
+                               "Put the selected sounds on the Edit tab's list and stay here")
+        for b in (self.b_edit, self.b_add):
+            acts.addWidget(b)
+        acts.addStretch(1)
+        lay.addSpacing(8)
+        lay.addLayout(acts, 1)
+        self.ctx.player.position.connect(self._pos)
+        self.ctx.player.playing.connect(lambda on: None if on else self._clear_playheads())
+
+    @staticmethod
+    def _btn(text, icon, slot, tip, color=None):
+        b = QPushButton(text)
+        b.setIcon(icons.icon(icon, color or icons.TEXT))
+        b.clicked.connect(lambda _=False: slot())
+        b.setToolTip(tip)
+        return b
+
+    def set_sound(self, s):
+        self.stop()
+        self.sound, self.orig, self.installed = s, None, None
+        for b in (self.b_orig, self.b_inst, self.b_edit, self.b_add, self.b_stop):
+            b.setEnabled(s is not None)
+        self.wf_orig.set_audio(None, 1, "select a sound" if s is None else "loading…", busy=s is not None)
+        self.wf_inst.set_audio(None, 1, "")
+        if s is None:
+            self.title.setText("No sound selected")
+            self.meta.setText("Select a sound in the list to listen to it; Edit opens it for changing.")
+            self.state.setText("")
+            self.sub.setText("")
+            return
+        self.title.setText(s.name)
+        who = s.character or (("mentions " + s.characters_inferred) if s.characters_inferred else "—")
+        self.meta.setText("%s / %s   •   %s   •   %.2f s, %d Hz, %s   •   %s"
+                          % (s.group, s.category, who, s.seconds, s.rate,
+                             {1: "mono", 2: "stereo"}.get(s.channels, "%d ch" % s.channels), s.key))
+        ws = self.ctx.ws
+        subs = ws.texts.for_sound(s, self.ctx.language) if ws else []
+        self.sub.setText("“%s”" % "  /  ".join(ws.texts.current(_key(x)) or x.text for x in subs) if subs else "")
+        key = s.key
+
+        def work(job):
+            cur = ws.current_bytes(s)
+            st = ws.state(s, cur)
+            o = audio.decode_wem(ws.original_bytes(s))
+            return key, st, o, audio.decode_wem(cur) if st != W.ORIGINAL else None
+
+        def failed(e, tb):
+            self.wf_orig.set_audio(None, 1, "could not load: %s" % e)
+
+        self.ctx.runner.start(work, self._loaded, failed)
+
+    def _loaded(self, r):
+        key, st, o, inst = r
+        if not self.sound or self.sound.key != key:
+            return
+        self.orig, self.installed = o, inst
+        self.wf_orig.set_audio(*o)
+        self.wf_inst.set_audio(*(inst or (None, 1)), placeholder="same as the original")
+        self.b_inst.setEnabled(inst is not None)
+        self.state.setText(STATE_TEXT.get(st, st))
+
+    def play(self, what, start=0.0):
+        a = {"orig": self.orig, "inst": self.installed}.get(what)
+        if not a:
+            return
+        self.playing_what = what
+        self.ctx.player.owner = self
+        self.ctx.player.play(a[0], a[1], start, label=what)
+
+    def toggle_play(self):
+        if self.ctx.player.is_playing():
+            self.stop()
+        else:
+            self.play("orig")
+
+    def stop(self):
+        self.ctx.player.stop()
+        self._clear_playheads()
+
+    def _clear_playheads(self):
+        for w in (self.wf_orig, self.wf_inst):
+            w.set_playhead(None)
+
+    def _pos(self, t):
+        if getattr(self.ctx.player, "owner", None) is not self:
+            return
+        wf = {"orig": self.wf_orig, "inst": self.wf_inst}.get(self.playing_what)
+        if wf:
+            wf.set_playhead(t)
+
+
+def _key(sub):
+    return subtitles.key(sub.lang, sub.speech, sub.line)

app/spinner.py

@@ -1,6 +1,6 @@
 """Small activity spinner: a rotating accent-coloured arc. Runs only while visible."""
-from PySide6.QtCore import QRectF, QTimer, Qt
+from PySide6.QtCore import QEvent, QRectF, QTimer, Qt
-from PySide6.QtGui import QColor, QPainter, QPen
+from PySide6.QtGui import QColor, QPainter, QPalette, QPen
 from PySide6.QtWidgets import QWidget
 
 from . import icons
@@ -51,3 +51,68 @@ class Spinner(QWidget):
     def paintEvent(self, _):
         p = QPainter(self)
         paint_arc(p, QRectF(self.rect()), self.angle)
+
+
+class BusyOverlay(QWidget):
+    """A list being (re)loaded: a light veil over the widget with the spinner and a message in its centre, the
+    same arc as the waveform's loading state. Covers the target and follows its size; blocks clicks while on."""
+
+    def __init__(self, target):
+        super().__init__(target)
+        self.target = target
+        self.message = ""
+        self.angle = 0
+        self.timer = QTimer(self)
+        self.timer.setInterval(STEP_MS)
+        self.timer.timeout.connect(self._step)
+        target.installEventFilter(self)
+        self.hide()
+
+    def start(self, message):
+        self.message = message
+        self.setGeometry(self.target.rect())
+        self.raise_()
+        self.show()
+        self.timer.start()
+        self.update()
+
+    def stop(self):
+        self.timer.stop()
+        self.hide()
+
+    @property
+    def busy(self):
+        return not self.isHidden()          # on, whether or not the window is on screen yet
+
+    def _step(self):
+        self.angle = (self.angle + STEP_DEG) % 360
+        if self.isVisible():
+            self.update()
+
+    def eventFilter(self, obj, e):
+        if obj is self.target and e.type() == QEvent.Type.Resize:
+            self.setGeometry(self.target.rect())
+        return False
+
+    def paintEvent(self, _):
+        p = QPainter(self)
+        p.setRenderHint(QPainter.RenderHint.Antialiasing)
+        pal = self.palette()
+        veil = QColor(pal.color(QPalette.ColorRole.Base))
+        veil.setAlpha(210)
+        p.fillRect(self.rect(), veil)
+        fm = p.fontMetrics()
+        d, gap, pad = 20, 12, 16
+        tw = fm.horizontalAdvance(self.message)
+        w, h = pad + d + gap + tw + pad, max(d, fm.height()) + 2 * pad - 8
+        box = QRectF((self.width() - w) / 2, (self.height() - h) / 2, w, h)
+        border = QColor(pal.color(QPalette.ColorRole.Text))
+        border.setAlpha(40)
+        p.setPen(QPen(border, 1))
+        p.setBrush(pal.color(QPalette.ColorRole.Window))
+        p.drawRoundedRect(box, 8, 8)
+        y = box.center().y()
+        paint_arc(p, QRectF(box.left() + pad, y - d / 2, d, d), self.angle)
+        p.setPen(pal.color(QPalette.ColorRole.Text))
+        p.drawText(QRectF(box.left() + pad + d + gap, y - fm.height() / 2, tw + 2, fm.height()),
+                   Qt.AlignmentFlag.AlignVCenter | Qt.AlignmentFlag.AlignLeft, self.message)

app/textpanel.py

@@ -0,0 +1,694 @@
+"""Text sections of the main window: Subtitles (linked to the speech sounds) and Menu text.
+
+One TextPanel per section: filters + table on top, the editor of the selected text below. Texts are written
+through Workspace.texts (ackit/texts.py): subtitles in place into every copy (limited to the original's
+length), menu texts into the localization files. Revert per text or all; changed-underneath warnings.
+"""
+from PySide6.QtCore import QAbstractTableModel, QModelIndex, Qt, QTimer, Signal
+from PySide6.QtWidgets import (QAbstractItemView, QCheckBox, QComboBox, QDialog, QDialogButtonBox, QFormLayout,
+                               QHBoxLayout, QHeaderView, QLabel, QLineEdit, QPlainTextEdit, QPushButton, QSplitter,
+                               QTableView, QVBoxLayout, QWidget)
+
+from ackit import subtitles
+from ackit.texts import CHANGED, FOREIGN, LOST, MODIFIED, ORIGINAL, TextConflictError
+from . import icons
+from .models import RowView
+from .spinner import BusyOverlay
+
+ANY = "Any"
+STATE_TEXT = {ORIGINAL: "", MODIFIED: "modified", LOST: "lost", CHANGED: "changed outside", FOREIGN: "not original"}
+
+
+class Row:
+    __slots__ = ("key", "cols", "text", "sound", "search")
+
+    def __init__(self, key, cols, text, sound=None):
+        self.key, self.cols, self.text, self.sound = key, cols, text, sound
+        self.search = (" ".join(str(c) for c in cols) + " " + text).lower()
+
+
+class TextModel(QAbstractTableModel):
+    def __init__(self, headers, parent=None):
+        super().__init__(parent)
+        self.headers = headers + ["Text", "State"]
+        self.rows = []
+        self.modified = {}            # key -> applied text
+
+    def reset(self, rows, modified):
+        self.beginResetModel()
+        self.rows, self.modified = rows, dict(modified)
+        self.endResetModel()
+
+    def set_modified(self, modified):
+        self.modified = dict(modified)
+        if self.rows:
+            self.dataChanged.emit(self.index(0, 0), self.index(len(self.rows) - 1, len(self.headers) - 1))
+
+    def rowCount(self, parent=QModelIndex()):
+        return 0 if parent.isValid() else len(self.rows)
+
+    def columnCount(self, parent=QModelIndex()):
+        return len(self.headers)
+
+    def headerData(self, section, orientation, role=Qt.ItemDataRole.DisplayRole):
+        if orientation == Qt.Orientation.Horizontal and role == Qt.ItemDataRole.DisplayRole:
+            return self.headers[section]
+        return None
+
+    def text_of(self, r):
+        return self.modified.get(r.key, r.text)
+
+    def data(self, index, role=Qt.ItemDataRole.DisplayRole):
+        r = self.rows[index.row()]
+        c = index.column()
+        n = len(r.cols)
+        if role in (Qt.ItemDataRole.DisplayRole, Qt.ItemDataRole.ToolTipRole):
+            if c < n:
+                return r.cols[c]
+            if c == n:
+                return self.text_of(r)
+            return "modified" if r.key in self.modified else ""
+        if role == Qt.ItemDataRole.UserRole:
+            return r
+        if role == Qt.ItemDataRole.ForegroundRole and c == n + 1 and r.key in self.modified:
+            from PySide6.QtGui import QColor
+            return QColor(icons.ACCENT)
+        return None
+
+
+class TextFilter(RowView):
+    """The text table's rows: search words, column filters, modified only; sorted case-insensitively."""
+
+    def __init__(self, parent=None):
+        self.words, self.col_filters, self.only_modified = [], {}, False
+        super().__init__(parent)
+
+    def source_len(self):
+        return len(self.sourceModel().rows)
+
+    def sort_key(self, row, column):
+        m = self.sourceModel()
+        r = m.rows[row]
+        n = len(r.cols)
+        v = r.cols[column] if column < n else m.text_of(r) if column == n else ("modified" if r.key in m.modified else "")
+        return v.lower() if isinstance(v, str) else v
+
+    def accepts(self, row):
+        m = self.sourceModel()
+        r = m.rows[row]
+        if self.only_modified and r.key not in m.modified:
+            return False
+        for c, v in self.col_filters.items():
+            if v != ANY and r.cols[c] != v:
+                return False
+        if self.words:
+            hay = r.search + " " + m.modified.get(r.key, "").lower()
+            return all(w in hay for w in self.words)
+        return True
+
+    def row_at(self, index):
+        return self.sourceModel().rows[self.rows[index.row()]]
+
+
+class FindReplaceDialog(QDialog):
+    def __init__(self, count, parent=None):
+        super().__init__(parent)
+        self.setWindowTitle("Find and replace")
+        f = QFormLayout(self)
+        self.find, self.repl = QLineEdit(), QLineEdit()
+        self.case = QCheckBox("Match case")
+        f.addRow(QLabel("In the %d text(s) shown by the current filter:" % count))
+        f.addRow("Find", self.find)
+        f.addRow("Replace with", self.repl)
+        f.addRow(self.case)
+        bb = QDialogButtonBox(QDialogButtonBox.StandardButton.Ok | QDialogButtonBox.StandardButton.Cancel)
+        bb.accepted.connect(self.accept)
+        bb.rejected.connect(self.reject)
+        f.addRow(bb)
+
+
+def replace_in(text, find, repl, case):
+    if case:
+        return text.replace(find, repl)
+    import re
+    return re.sub(re.escape(find), lambda _: repl, text, flags=re.IGNORECASE)
+
+
+class SubtitleBox(QWidget):
+    """The subtitle of the sound open in the sound editor (side by side with the audio)."""
+    changed = Signal()
+    open_in_list = Signal(str)            # SUB key
+
+    def __init__(self, win, parent=None):
+        super().__init__(parent)
+        self.win = win
+        self.keys = []
+        outer = QVBoxLayout(self)
+        outer.setContentsMargins(0, 0, 0, 0)
+        outer.setSpacing(6)
+        self.lay = QFormLayout()               # one line edit per subtitle line
+        self.lay.setContentsMargins(0, 0, 0, 0)
+        self.edits = []
+        self.lbl = QLabel()
+        self.lbl.setObjectName("Muted")
+        row = QHBoxLayout()
+        row.setSpacing(6)
+        self.b_apply = QPushButton(icons.icon("apply"), "Apply")
+        self.b_revert = QPushButton(icons.icon("revert"), "Revert")
+        self.b_open = QPushButton(icons.icon("search"), "Show in list")
+        self.b_apply.setToolTip("Write this subtitle into the game (every copy)")
+        self.b_revert.setToolTip("Put the original subtitle back")
+        self.b_open.setToolTip("Open this subtitle in the Subtitles tab")
+        self.b_apply.clicked.connect(self.apply)
+        self.b_revert.clicked.connect(self.revert)
+        self.b_open.clicked.connect(lambda: self.keys and self.open_in_list.emit(self.keys[0]))
+        for b in (self.b_apply, self.b_revert):    # two halves of the column, like the sound's action bar
+            row.addWidget(b, 1)
+        self.lbl.setWordWrap(True)
+        outer.addLayout(self.lay)
+        outer.addWidget(self.lbl)              # room counter
+        outer.addWidget(self.b_open)           # the way to the Subtitles tab, across the column
+        outer.addStretch(1)                    # actions at the bottom, level with the sound's
+        outer.addLayout(row)
+
+    def set_sound(self, s):
+        for e in self.edits:
+            self.lay.removeRow(e)
+        self.edits, self.keys = [], []
+        ws = self.win.ws
+        lines = ws.texts.for_sound(s, self.win.language) if ws and s else []
+        if not lines:
+            self.lbl.setText("No subtitle for this sound." if s else "")
+            for b in (self.b_apply, self.b_revert, self.b_open):
+                b.setEnabled(False)
+            return
+        tx = ws.texts
+        for i, ln in enumerate(lines):
+            k = subtitles.key(ln.lang, ln.speech, ln.line)
+            e = QLineEdit(tx.records().get(k, {}).get("applied") or ln.text)
+            e.setMinimumWidth(80)
+            e.setCursorPosition(0)                     # long subtitles: show their beginning
+            e.setToolTip("Original: %s\nRoom: %d characters" % (ln.text, ln.room))
+            e.textChanged.connect(self._check)
+            self.lay.insertRow(i, "Line %d" % (i + 1) if len(lines) > 1 else "Subtitle", e)
+            self.edits.append(e)
+            self.keys.append(k)
+        self.b_open.setEnabled(True)
+        self.b_revert.setEnabled(any(k in tx.records() for k in self.keys))
+        self._check()
+
+    def _check(self):
+        tx = self.win.ws.texts
+        msgs = []
+        ok = True
+        for k, e in zip(self.keys, self.edits):
+            try:
+                tx.check(k, e.text())
+            except ValueError as ex:
+                ok = False
+                msgs.append(str(ex))
+        rooms = [tx.limit(k) for k in self.keys]
+        self.lbl.setText("   •   ".join(msgs) if msgs else "Room: %s characters (%s)" % (
+            " / ".join(str(r) for r in rooms), self.keys[0].split(":")[1]))
+        self.b_apply.setEnabled(ok)
+
+    def values(self):
+        return {k: e.text() for k, e in zip(self.keys, self.edits)}
+
+    def apply(self):
+        tx = self.win.ws.texts
+        changes = {k: v for k, v in self.values().items() if v != tx.original(k) or k in tx.records()}
+        if not changes:
+            return
+        self._run(lambda job, force: tx.apply(changes, force=force, progress=job.report), "Subtitle applied.")
+
+    def revert(self):
+        tx = self.win.ws.texts
+        self._run(lambda job, force: tx.revert(self.keys, progress=job.report), "Subtitle reverted.")
+
+    def _run(self, fn, ok_msg):
+        def done(_):
+            self.win.statusBar().showMessage(ok_msg, 6000)
+            self.changed.emit()
+
+        def failed(e, tb):
+            if isinstance(e, TextConflictError) and self.win.confirm("Changed outside", "%s\n\nOverwrite anyway?" % e):
+                self.win.runner.start(lambda job: fn(job, True), done, failed)
+                return
+            self.win.inform("Error", str(e))
+
+        self.win.runner.start(lambda job: fn(job, False), done, failed)
+
+
+class SubtitleEdit(QPlainTextEdit):
+    """A subtitle line as a large wrapping text box (one line of text in the game: Enter is ignored and pasted
+    line breaks become spaces). Has the QLineEdit calls SubtitleBox uses: text(), textChanged."""
+
+    def __init__(self, text, parent=None):
+        super().__init__(parent)
+        f = self.font()
+        f.setPointSizeF(f.pointSizeF() * 1.3)
+        self.setFont(f)
+        self.setPlainText(text)
+        self.setTabChangesFocus(True)
+        self.setFixedHeight(self.fontMetrics().lineSpacing() * 2 + 14)    # 2 lines at the Edit tab's width
+
+    def text(self):
+        return self.toPlainText()
+
+    def setText(self, text):
+        self.setPlainText(text)
+
+    def keyPressEvent(self, e):
+        if e.key() in (Qt.Key.Key_Return, Qt.Key.Key_Enter):
+            e.ignore()
+            return
+        super().keyPressEvent(e)
+
+    def insertFromMimeData(self, source):
+        self.insertPlainText(" ".join(source.text().split()))
+
+
+class SubtitlePanel(SubtitleBox):
+    """The subtitle of the sound in the Edit tab: large text, the game's original under it (and the
+    English line when editing another language), room counter, Apply / Revert / Show in list."""
+
+    def __init__(self, win, parent=None):
+        super().__init__(win, parent)
+        self.lines = QVBoxLayout()
+        self.lines.setContentsMargins(0, 0, 0, 0)
+        self.lines.setSpacing(4)
+        outer = self.layout()
+        outer.insertLayout(0, self.lines)
+        outer.setSpacing(4)
+        row = outer.itemAt(outer.count() - 1).layout()  # one row: room counter | Apply | Revert | Show in list
+        for w in (self.lbl, self.b_open):
+            outer.removeWidget(w)
+        row.insertWidget(0, self.lbl, 3)
+        row.addWidget(self.b_open, 1)
+        self._parts = []
+        self._english = {}
+
+    def _english_text(self, ln):
+        if ln.lang == "INT":
+            return None
+        if not self._english:
+            try:
+                self._english = {(x.speech, x.line): x.text for x in self.win.ws.texts.subtitles("INT")}
+            except Exception:                            # noqa: BLE001 - only a reference text
+                self._english = {None: None}
+        return self._english.get((ln.speech, ln.line))
+
+    def set_sound(self, s):
+        for w in self._parts:
+            self.lines.removeWidget(w)
+            w.deleteLater()
+        self._parts, self.edits, self.keys = [], [], []
+        ws = self.win.ws
+        lines = ws.texts.for_sound(s, self.win.language) if ws and s else []
+        if not lines:
+            self.lbl.setText("No subtitle for this sound." if s else "")
+            for b in (self.b_apply, self.b_revert, self.b_open):
+                b.setEnabled(False)
+            return
+        tx = ws.texts
+        for i, ln in enumerate(lines):
+            k = subtitles.key(ln.lang, ln.speech, ln.line)
+            e = SubtitleEdit(tx.records().get(k, {}).get("applied") or ln.text)
+            e.setToolTip("Room: %d characters" % ln.room)
+            e.textChanged.connect(self._check)
+            self.lines.addWidget(e)
+            part = " (part %d of %d)" % (i + 1, len(lines)) if len(lines) > 1 else ""
+            ref = "Original%s: %s" % (part, ln.text)
+            en = self._english_text(ln)
+            if en and en != ln.text:
+                ref += "\nEnglish: " + en
+            r = QLabel(ref)
+            r.setObjectName("Muted")
+            r.setWordWrap(True)
+            r.setTextInteractionFlags(Qt.TextInteractionFlag.TextSelectableByMouse)
+            self.lines.addWidget(r)
+            self._parts += [e, r]
+            self.edits.append(e)
+            self.keys.append(k)
+        self.b_open.setEnabled(True)
+        self.b_revert.setEnabled(any(k in tx.records() for k in self.keys))
+        self._check()
+
+
+class TextPanel(QWidget):
+    """kind 'subs' (subtitles) or 'menu' (menu text)."""
+    changed = Signal()
+    message = Signal(str)
+    go_to_sound = Signal(str)            # sound key (subtitles)
+
+    def __init__(self, win, kind):
+        super().__init__(win)
+        self.win, self.kind = win, kind
+        self.lang = None
+        self.current = None              # selected Row
+        self._build()
+
+    # --- UI ---------------------------------------------------------------------------------------
+    def _build(self):
+        v = QVBoxLayout(self)
+        v.setContentsMargins(6, 6, 6, 6)
+        top = QHBoxLayout()
+        self.search = QLineEdit()
+        self.search.setClearButtonEnabled(True)
+        self.search.setPlaceholderText("Search texts, names, characters…" if self.kind == "subs" else
+                                       "Search texts, files, keys…")
+        self.search.addAction(icons.icon("search", icons.MUTED), QLineEdit.ActionPosition.LeadingPosition)
+        self._timer = QTimer(self)
+        self._timer.setSingleShot(True)
+        self._timer.setInterval(150)
+        self._timer.timeout.connect(self._filter)
+        self.search.textChanged.connect(lambda _: self._timer.start())
+        self.lbl_lang = QLabel()
+        self.lbl_lang.setObjectName("Muted")
+        self.lbl_lang.setToolTip("Change the language with the Language list at the top of the window")
+        top.addWidget(self.search, 1)
+        top.addWidget(self.lbl_lang)
+        self.filters = {}
+        names = ("Character", "Category") if self.kind == "subs" else ("File",)
+        for name in names:
+            cb = QComboBox()
+            cb.setMinimumWidth(120)
+            cb.currentTextChanged.connect(lambda _: self._filter())
+            top.addWidget(QLabel(name))
+            top.addWidget(cb)
+            self.filters[name] = cb
+        self.cb_mod = QCheckBox("Modified only")
+        self.cb_mod.toggled.connect(lambda _: self._filter())
+        top.addWidget(self.cb_mod)
+        self.lbl_count = QLabel()
+        top.addWidget(self.lbl_count)
+        v.addLayout(top)
+
+        headers = ["Speech line", "#", "Character", "Category", "Room"] if self.kind == "subs" else \
+            ["File", "Section", "Key"]
+        self.model = TextModel(headers, self)
+        self.proxy = TextFilter(self)
+        self.proxy.setSourceModel(self.model)
+        self.table = QTableView()
+        self.table.setModel(self.proxy)
+        self.busy = BusyOverlay(self.table)    # while the texts of a language load (background job)
+        self._load_gen = 0
+        self._pending_key = None
+        self.table.setSortingEnabled(True)
+        self.table.setSelectionBehavior(QAbstractItemView.SelectionBehavior.SelectRows)
+        self.table.setSelectionMode(QAbstractItemView.SelectionMode.SingleSelection)
+        self.table.verticalHeader().hide()
+        self.table.verticalHeader().setDefaultSectionSize(22)
+        self.table.setAlternatingRowColors(True)
+        self.table.setWordWrap(False)
+        h = self.table.horizontalHeader()
+        h.setSectionResizeMode(QHeaderView.ResizeMode.Interactive)
+        h.setStretchLastSection(False)
+        widths = (330, 30, 120, 130, 50) if self.kind == "subs" else (140, 150, 220)
+        for c, w in enumerate(widths):
+            self.table.setColumnWidth(c, w)
+        self.table.setColumnWidth(len(widths), 560)
+        self.table.selectionModel().currentRowChanged.connect(lambda cur, _: self._select(cur))
+
+        # editor
+        ed = QWidget()
+        f = QFormLayout(ed)
+        self.title = QLabel("No text selected")
+        self.title.setObjectName("EditorTitle")
+        self.lbl_orig = QLabel()
+        self.lbl_orig.setWordWrap(True)
+        self.lbl_orig.setTextInteractionFlags(Qt.TextInteractionFlag.TextSelectableByMouse)
+        self.edit = QLineEdit()
+        self.edit.textChanged.connect(self._edited)
+        self.edit.returnPressed.connect(self.apply_current)
+        self.lbl_info = QLabel()
+        row = QHBoxLayout()
+        self.b_apply = QPushButton(icons.icon("apply"), "Apply")
+        self.b_revert = QPushButton(icons.icon("revert"), "Revert")
+        self.b_replace = QPushButton(icons.icon("search"), "Find and replace…")
+        self.b_revert_all = QPushButton(icons.icon("revert"), "Revert all texts here")
+        self.b_apply.clicked.connect(self.apply_current)
+        self.b_revert.clicked.connect(self.revert_current)
+        self.b_replace.clicked.connect(self.find_replace)
+        self.b_revert_all.clicked.connect(self.revert_all)
+        for b in (self.b_apply, self.b_revert):
+            row.addWidget(b)
+        if self.kind == "subs":
+            self.b_sound = QPushButton(icons.icon("play"), "Go to the sound")
+            self.b_sound.clicked.connect(lambda: self.current and self.current.sound and
+                                         self.go_to_sound.emit(self.current.sound))
+            row.addWidget(self.b_sound)
+        row.addStretch(1)
+        row.addWidget(self.b_replace)
+        row.addWidget(self.b_revert_all)
+        f.addRow(self.title)
+        f.addRow("Original", self.lbl_orig)
+        f.addRow("Text", self.edit)
+        f.addRow("", self.lbl_info)
+        f.addRow(row)
+        split = QSplitter(Qt.Orientation.Vertical)
+        split.addWidget(self.table)
+        split.addWidget(ed)
+        split.setStretchFactor(0, 1)
+        split.setStretchFactor(1, 0)
+        v.addWidget(split, 1)
+        self._select(None)
+
+    # --- data ---------------------------------------------------------------------------------------
+    def ws(self):
+        return self.win.ws
+
+    def languages(self):
+        tx = self.ws().texts
+        return tx.subtitle_languages() if self.kind == "subs" else tx.menu_languages()
+
+    def load(self, lang=None):
+        """(Re)load the rows of a language (default: the window's language)."""
+        ws = self.ws()
+        if not ws:
+            return
+        lang = lang or self.win.language
+        if lang not in self.languages():
+            lang = None
+        self.lang = lang
+        self.lbl_lang.setText("Language: %s" % lang if lang else "No %s in this language" % (
+            "subtitles" if self.kind == "subs" else "menu text"))
+        self._load_gen += 1
+        gen = self._load_gen
+        if not lang:
+            self.busy.stop()
+            self.model.reset([], {})
+            return
+        self.busy.start("Loading subtitles…" if self.kind == "subs" else "Loading menu text…")
+
+        def done(rows):
+            if gen == self._load_gen:              # a newer load replaces this one
+                self.busy.stop()
+                self._show_rows(rows)
+
+        def failed(e, tb):
+            if gen == self._load_gen:
+                self.busy.stop()
+                self.message.emit("Could not load the texts: %s" % e)
+
+        self.win.runner.start(lambda job: self._rows(ws, lang), done, failed)
+
+    def _rows(self, ws, lang):
+        """The table rows of a language (runs as a background job)."""
+        tx = ws.texts
+        rows = []
+        if self.kind == "subs":
+            by_key = ws.catalog.by_key
+            for s in tx.subtitles(lang):
+                snd = by_key.get(s.sounds[0]) if s.sounds else None   # None: the line's archive is not loaded
+                who = (snd.character or snd.characters_inferred) if snd else ""
+                cat = snd.category if snd else ""
+                rows.append(Row(subtitles.key(lang, s.speech, s.line),
+                                [s.speech, s.line + 1, who, cat, s.room], s.text, s.sounds[0] if s.sounds else None))
+        else:
+            for e in tx.menu(lang):
+                rows.append(Row("MENU:" + e.key, [e.file, e.section, e.name + ("#%d" % e.occurrence if e.occurrence > 1 else "")],
+                                self._menu_original(e)))
+        return rows
+
+    def _show_rows(self, rows):
+        self.model.reset(rows, self._modified())
+        for i, cb in enumerate(self.filters.values()):
+            col = (2 + i) if self.kind == "subs" else 0
+            cur = cb.currentText()
+            cb.blockSignals(True)
+            cb.clear()
+            cb.addItems([ANY] + sorted({str(r.cols[col]) for r in rows if r.cols[col]}))
+            cb.setCurrentText(cur if cur else ANY)
+            cb.blockSignals(False)
+        self._filter()
+        if self._pending_key:                       # asked for while the rows were loading
+            key, self._pending_key = self._pending_key, None
+            self.select_key(key)
+
+    def _menu_original(self, e):
+        r = self.ws().texts.records().get("MENU:" + e.key)
+        return r["orig"] if r else e.value
+
+    def _modified(self):
+        return {k: r["applied"] for k, r in self.ws().texts.records().items()
+                if r.get("applied") is not None and k.split(":")[1] == self.lang and
+                k.startswith("SUB:" if self.kind == "subs" else "MENU:")}
+
+    def refresh(self):
+        self.model.set_modified(self._modified())                    # the view refilters/re-sorts itself
+        self._update_count()
+        if self.current:
+            self._show(self.current)
+
+    def _filter(self):
+        self.proxy.words = [w for w in self.search.text().lower().split() if w]
+        self.proxy.only_modified = self.cb_mod.isChecked()
+        cols = (2, 3) if self.kind == "subs" else (0,)
+        self.proxy.col_filters = {c: cb.currentText() for c, cb in zip(cols, self.filters.values())}
+        self.proxy.refilter()
+        self._update_count()
+
+    def _update_count(self):
+        self.lbl_count.setText("%d / %d" % (self.proxy.rowCount(), self.model.rowCount()))
+
+    def visible_rows(self):
+        return [self.proxy.row_at(self.proxy.index(i, 0)) for i in range(self.proxy.rowCount())]
+
+    def select_key(self, key):
+        if self.busy.busy:                          # rows still loading: select when they arrive
+            self._pending_key = key
+            return True
+        for i in range(self.proxy.rowCount()):
+            if self.proxy.row_at(self.proxy.index(i, 0)).key == key:
+                self.table.selectRow(i)
+                self.table.scrollTo(self.proxy.index(i, 0))
+                return True
+        return False
+
+    # --- editing ------------------------------------------------------------------------------------
+    def _select(self, index):
+        r = self.proxy.row_at(index) if index is not None and index.isValid() else None
+        self.current = r
+        self._show(r)
+
+    def _show(self, r):
+        enabled = r is not None
+        for w in (self.edit, self.b_apply, self.b_revert):
+            w.setEnabled(enabled)
+        if self.kind == "subs":
+            self.b_sound.setEnabled(bool(r and r.sound))
+        if not r:
+            self.title.setText("No text selected")
+            self.lbl_orig.setText("")
+            self.edit.setText("")
+            self.lbl_info.setText("")
+            return
+        tx = self.ws().texts
+        self.title.setText(r.key)
+        self.lbl_orig.setText(r.text)
+        try:
+            cur = tx.current(r.key)
+            st = tx.state(r.key, cur)
+        except Exception as e:  # noqa: BLE001 - unreadable file: show it, do not crash
+            cur, st = r.text, "error: %s" % e
+        self.edit.blockSignals(True)
+        self.edit.setText(cur)
+        self.edit.blockSignals(False)
+        self._state = st
+        self.b_revert.setEnabled(r.key in tx.records())
+        self._edited()
+
+    def _edited(self):
+        r = self.current
+        if not r:
+            return
+        tx = self.ws().texts
+        lim = tx.limit(r.key)
+        n = len(self.edit.text())
+        info = []
+        if lim is not None:
+            info.append("%d / %d characters" % (n, lim))
+        st = STATE_TEXT.get(self._state, self._state)
+        if st:
+            info.append("state: " + st)
+        try:
+            tx.check(r.key, self.edit.text())
+            ok = True
+        except ValueError as e:
+            info.append(str(e))
+            ok = False
+        self.lbl_info.setText("   •   ".join(info))
+        self.b_apply.setEnabled(ok)
+
+    def _run(self, busy, fn, ok_msg):
+        def done(_):
+            self.message.emit(ok_msg)
+            self.refresh()
+            self.changed.emit()
+
+        def failed(e, tb):
+            if isinstance(e, TextConflictError) and self.win.confirm(
+                    "Changed outside", "%s\n\nOverwrite anyway?" % e):
+                self.win.runner.start(lambda job: fn(job, True), done, failed)
+                return
+            self.win.inform("Error", str(e))
+
+        self.message.emit(busy)
+        self.win.runner.start(lambda job: fn(job, False), done, failed)
+
+    def apply_current(self):
+        r = self.current
+        if not r or not self.b_apply.isEnabled():
+            return
+        text = self.edit.text()
+        self._run("Writing…", lambda job, force: self.ws().texts.apply({r.key: text}, force=force, progress=job.report),
+                  "Applied %s" % r.key)
+
+    def revert_current(self):
+        r = self.current
+        if r:
+            self._run("Reverting…", lambda job, force: self.ws().texts.revert([r.key], progress=job.report),
+                      "Reverted %s" % r.key)
+
+    def revert_all(self):
+        keys = list(self._modified())
+        if not keys:
+            self.win.inform("Revert", "No %s text is modified." % self.lang)
+            return
+        if not self.win.confirm("Revert", "Put the original back for %d text(s)?" % len(keys)):
+            return
+        self._run("Reverting…", lambda job, force: self.ws().texts.revert(keys, progress=job.report),
+                  "Reverted %d text(s)." % len(keys))
+
+    def find_replace(self, find=None, repl=None, case=False):
+        rows = self.visible_rows()
+        if find is None:
+            dlg = FindReplaceDialog(len(rows), self)
+            if dlg.exec() != QDialog.DialogCode.Accepted or not dlg.find.text():
+                return
+            find, repl, case = dlg.find.text(), dlg.repl.text(), dlg.case.isChecked()
+        tx = self.ws().texts
+        changes, skipped = {}, []
+        for r in rows:
+            old = self.model.text_of(r)
+            new = replace_in(old, find, repl, case)
+            if new == old:
+                continue
+            try:
+                tx.check(r.key, new)
+                changes[r.key] = new
+            except ValueError as e:
+                skipped.append("%s: %s" % (r.key, e))
+        if not changes:
+            self.win.inform("Find and replace", "Nothing to replace." + ("\n" + "\n".join(skipped[:10]) if skipped else ""))
+            return
+        if skipped and not self.win.confirm("Find and replace", "%d text(s) cannot take the replacement and are "
+                                            "skipped:\n%s\n\nReplace in the other %d?" % (
+                                                len(skipped), "\n".join(skipped[:10]), len(changes))):
+            return
+        self._run("Replacing in %d text(s)…" % len(changes),
+                  lambda job, force: tx.apply(changes, force=force, progress=job.report),
+                  "Replaced in %d text(s)." % len(changes))
+        return len(changes)

app/theme.py

@@ -58,8 +58,9 @@ QLineEdit, QComboBox, QDoubleSpinBox, QPlainTextEdit, QListWidget, QTreeWidget {
 QLineEdit:focus, QComboBox:focus, QDoubleSpinBox:focus, QPlainTextEdit:focus { border-color: %(ACCENT)s; }
 QComboBox QAbstractItemView { background: #1b1b1b; border: 1px solid %(CONTROL)s;
                               selection-background-color: rgba(176,36,44,90); }
-QCheckBox::indicator { width: 14px; height: 14px; border: 1px solid #555555; border-radius: 3px; background: #101010; }
+QCheckBox::indicator, QGroupBox::indicator, QListView::indicator { width: 14px; height: 14px; border: 1px solid #555555; border-radius: 3px;
-QCheckBox::indicator:checked { background: %(ACCENT)s; border-color: %(ACCENT)s; }
+                                             background: #101010; }
+QCheckBox::indicator:checked, QGroupBox::indicator:checked, QListView::indicator:checked { background: %(ACCENT)s; border-color: %(ACCENT)s; }
 QProgressBar { background: #101010; border: 1px solid %(CONTROL)s; border-radius: 3px; height: 10px; }
 QProgressBar::chunk { background: %(ACCENT)s; border-radius: 2px; }
 QSlider::groove:horizontal { height: 4px; background: #333333; border-radius: 2px; }

data/catalog/README.md

@@ -2,36 +2,73 @@
 
 | file | content |
 |------|---------|
-| `<ARCHIVE>.csv` | every sound of one audio archive: `ITA`, `INT`, `FRA`, `DEU`, `ESN` (speech, one per language) and `SFX` (music, ambience, effects of the SFX WAD; sounds stored inside soundbanks are not listed, see `docs/formats/banks.md`) |
+| `<ARCHIVE>.csv` | every sound of one audio archive: `INT`, `FRA`, `ITA`, `DEU`, `ESN` (speech WADs, one per language), `SFX` (music, ambience and effects streamed from the SFX WAD) and `BANK` (in-memory sounds stored inside soundbanks: footsteps, foley, combat, weapons, UI, efforts; `docs/formats/banks.md`) |
-| `characters.csv` | `name,character`: verified speaker per sound name (language independent) |
+| `BANK_locations.csv.gz` | every stored copy of the `BANK` sounds, and the bank copies of WAD speech recordings (prefetch heads, whole copies; below) |
-| `index.json` | fingerprint of the game WAD each CSV was built from (`ackit/game.py: wad_fingerprint`) and counts |
+| `SUB_<LANG>.csv` | subtitles of the speech lines, one file per language (11 languages; `docs/formats/subtitles.md`) |
-| `source/<lang>/*.txt` | **the curated original lists** (one file per character, one wemscan-style file name per line, e.g. `Speech_TAPE_JOKER_1_4.wem`). `Files.txt` = all files, `Stats.txt` = counts, `Checkpoint.txt` = progress marker. Never edited by tools. |
+| `SUB_locations.csv.gz` | every stored copy of each subtitle string (dialogue objects in the packages) |
+| `characters.csv` | `name,character,language`: verified speaker per line name (empty `language`: every language, the same line everywhere; a language code would label a sound of that language only - none needs it now) |
+| `index.json` | per archive: fingerprint of the game files the CSV was built from (`ackit/game.py: wad_fingerprint`, `ackit/banks.py: game_fingerprint`) and counts |
+| `source/<lang>/*.txt` | **the curated original lists** (one file per character, one wemscan-style file name per line, e.g. `Speech_TAPE_JOKER_1_4.wem`). `Files.txt` = all files, `Stats.txt` = counts. Never edited by tools. |
 
 All CSVs are UTF-8 with a header row. They contain only names and metadata (no game audio).
 
 ## `<ARCHIVE>.csv` columns
 | column | meaning |
 |--------|---------|
-| `archive` | `ITA`, `INT`, ... or `SFX` |
+| `archive` | `INT`, `FRA`, ..., `SFX` or `BANK` |
-| `target` | location inside the archive: `HASH` (entry with one WEM), `HASH:i` (i-th variation, 0-based), `PACK/MEDIA` (music stream pack) |
+| `target` | location: `HASH` (WAD entry with one WEM), `HASH:i` (i-th variation, 0-based), `PACK/MEDIA` (music stream pack), `MEDIA` (BANK: Wwise media ID) |
-| `name` | display name (`NAME`, `NAME_1..n` for variations, see `ackit/catalog.py`) |
+| `name` | display name. Speech: the line's name, the same in every language (`docs/decisions.md` 5): its speech event (e.g. `Speech_TAPE_JOKER_1_04`, `NAME#2` = second version an event plays) or, for lines no named event plays, the INT recording's position name. A recording stored twice keeps its position name in the second place. SFX: `NAME`, `NAME_1..n`. BANK: the Wwise event that plays it (`Play_..._i`) or `<Bank>_<n>` (`<Bank>_<MEDIA>` where that name is taken, e.g. by the SFX WAD's position names) |
-| `aliases` | other accepted names, `\|`-separated (`NAME_1` for single entries, `NAME#i` for variations) |
+| `aliases` | other accepted names, `\|`-separated: `NAME#i` = i-th WEM of the entry in this language's WAD (positions differ between languages), the entry name for single entries, `<Bank>/<MEDIA>` for BANK |
 | `group` | Speech, Music, Ambience, SFX, Unknown (`docs/categories.md`) |
 | `category` | finer category (`docs/categories.md`) |
+| `bank` | BANK: the soundbank holding the sound (first by name if several do); empty for WAD sounds |
+| `language` | BANK: language(s) of a localized speech line, `;`-separated (`INT`, or `ESN;FRA;INT;ITA` for one recording shared by those); empty = the same sound in every language. The editor shows the sounds of the selected language; names are unique per language |
+| `line` | speech (WAD and BANK): the line's identity, the same in every language (smallest Wwise Sound object ID of the line; `ackit/lines.py`). Two rows of one archive with the same `line` are one recording stored twice (written together). Empty for non-speech sounds |
 | `character` | verified speaker(s) from `characters.csv`, `; `-separated, empty if unknown |
-| `characters_inferred` | characters/roles mentioned by the name (hint only) |
+| `characters_inferred` | characters/roles mentioned by the name (and bank) (hint only) |
 | `seconds`, `rate`, `channels` | of the original sound |
 | `size` | size of the original WEM in bytes = maximum size of a replacement (except music packs: any size) |
-| `sha1` | first 16 hex digits of the original WEM's SHA-1 (detects files changed outside the tool) |
+| `sha1` | first 16 hex digits of the original WEM's SHA-1 (detects files changed outside the tool; empty when the copies of a BANK sound differ) |
 | `notes` | free text (empty; for future curation, e.g. transcripts) |
 
-The editor uses these files directly when the installed WAD matches `index.json`; for another game build it
+## `BANK_locations.csv.gz` columns (gzip-compressed CSV)
-scans the WAD itself and applies the same rules (no checksums then).
+| column | meaning |
+|--------|---------|
+| `media` | Wwise media ID (hex) |
+| `file` | file under `BmGame/CookedPCConsole`, `/`-separated (`SFX/xxxxxxxx.bnk` or `Name.upk`) |
+| `offset` | position of the WEM in the *uncompressed* file (hex; `docs/formats/upk.md`) |
+| `size` | bytes reserved for it there |
+| `kind` | `m` = copy of a BANK sound, `p` = copy of (the start of) a WAD speech recording: a prefetch head, or a whole copy with the same media ID |
+| `sound` | `p` rows: key of that speech recording (e.g. `INT:2B151272:0`); each language has its own recordings and heads |
+
+A replacement is written into every `m` copy of a BANK sound; replacing a speech line also rewrites its `p`
+copies (`docs/formats/banks.md`).
+
+## `SUB_<LANG>.csv` columns
+| column | meaning |
+|--------|---------|
+| `speech` | the speech event of the line (e.g. `Speech_CHURCH_B1_Ch2_POST_Explosion_BM_Cash_04`) |
+| `line` | 0-based line of a multi-line subtitle |
+| `text` | original subtitle text |
+| `room` | maximum length of a replacement in characters (it is written in place) |
+| `enc` | `a` = cp1252 string, `w` = UTF-16 string |
+| `sound` | key(s) of the linked voice recording(s), `;`-separated when the event plays two versions: this language's WAD line or in-memory `BANK:` line (the INT ones for languages without voices); empty when no event reaches a sound |
+
+`SUB_locations.csv.gz`: `lang, speech, line, file, pos` (hex, uncompressed offset of the FString), `size`, `enc`.
+Menu text has no catalog: the localization files are read directly (`docs/formats/text.md`).
+
+The editor uses these files directly when the installed game matches `index.json`; for another game build it
+scans the game itself and applies the same rules (no checksums then; the bank scan takes a few minutes).
+`ackit verify --catalog` cross-checks the catalog in use against the game files (`--quick` samples packages).
 
 ## Rebuilding / adding a language
 1. Make sure the game files are original (`ackit status`; `ackit revert --all`).
-2. Optional: add listening-based lists for another language in `source/<lang>/<Character>.txt` (same format;
+2. Optional: add listening-based lists for another language in `source/<lang>/<Character>.txt` (same format:
-   names are the same in every language, so the Italian lists already label all languages).
+   wemscan names of THAT language's WAD). Each list is resolved against its own language's positions and moved
-3. Run `python tools/build_catalog.py` - it scans every archive of the installed game (all languages present)
+   to the lines, so a label then applies to the same recording in every language.
-   and rewrites the CSVs, `characters.csv` and `index.json`.
+3. Run `python tools/build_catalog.py` - it scans every archive and all soundbanks of the installed game (all
+   languages present; ~5 minutes) and rewrites the CSVs, the location indexes, `characters.csv` and `index.json`.
+   It stops before writing anything if a sound or a subtitle differs from the current catalog (a modified game); after a game
+   update run it with `--accept-changes`.
+4. Run `ackit verify --catalog` (no issues expected).
 New character labels need a display name in `ackit/classify.py: USER_LABELS` (otherwise the file name is used).

data/catalog/index.json

@@ -61,5 +61,79 @@
   },
   "with_character": 0
  },
+ "BANK": {
+  "fingerprint": "3f9766af425b8eef",
+  "sounds": 15355,
+  "groups": {
+   "Ambience": 1015,
+   "SFX": 6912,
+   "Speech": 7422,
+   "Music": 2,
+   "Unknown": 4
+  },
+  "named_by_event": 13501,
+  "named_by_bank": 1854,
+  "copies": 93702,
+  "files": 1498,
+  "prefetch_heads": {
+   "recordings": 445,
+   "copies": 2185,
+   "unmapped": 0
+  },
+  "issues": []
+ },
+ "SUB": {
+  "fingerprint": "3f9766af425b8eef",
+  "lines": {
+   "DEU": 9667,
+   "ESM": 9504,
+   "ESN": 9495,
+   "FRA": 9570,
+   "INT": 9404,
+   "ITA": 9377,
+   "JPN": 9035,
+   "KOR": 9252,
+   "POL": 9379,
+   "POR": 9478,
+   "RUS": 9497
+  },
+  "speech_lines": 9035,
+  "linked": {
+   "DEU": 9632,
+   "ESM": 9470,
+   "ESN": 9462,
+   "FRA": 9534,
+   "INT": 9373,
+   "ITA": 9346,
+   "JPN": 9011,
+   "KOR": 9223,
+   "POL": 9346,
+   "POR": 9444,
+   "RUS": 9463
+  },
+  "copies": 168688,
+  "dialogue_objects": 14744,
+  "dialogue_lines": 9032,
+  "scripted_objects": 41,
+  "issues": []
+ },
+ "LINES": {
+  "languages": [
+   "DEU",
+   "ESN",
+   "FRA",
+   "INT",
+   "ITA"
+  ],
+  "recordings": 61485,
+  "lines": 12434,
+  "wad_lines": 10811,
+  "wad_lines_named_by_event": 10174,
+  "wad_lines_named_after_int_recording": 637,
+  "lines_in_every_language": 12434,
+  "lines_missing_a_language": {},
+  "without_line": [],
+  "issues": []
+ },
  "_info": "unmodified game files"
 }

data/catalog/source/ita/Checkpoint.txt

@@ -1 +0,0 @@
-10816

docs/README.md

@@ -3,13 +3,16 @@
 | document | content |
 |----------|---------|
 | [user-guide.md](user-guide.md) | installing and using Unheard City (screenshots) |
-| [decisions.md](decisions.md) | stack, dependencies/licenses, originals store, size rule, trade-offs |
+| [decisions.md](decisions.md) | stack, dependencies/licenses, originals store, size rule, in-memory sounds, texts, language, verification |
 | [categories.md](categories.md) | group/category scheme and character fields |
 | [building.md](building.md) | building the editor and its installer (Windows, Linux) |
 | [formats/wad.md](formats/wad.md) | WWAD archives, grouped entries, BKPK stream packs |
-| [formats/banks.md](formats/banks.md) | soundbanks and the in-memory sounds not covered yet (footsteps, foley, effects) |
+| [formats/banks.md](formats/banks.md) | soundbanks and the in-memory sounds (footsteps, foley, effects, localized speech); how each voice line is linked across languages |
+| [formats/subtitles.md](formats/subtitles.md) | subtitles: dialogue objects in the packages, link to the speech sounds |
+| [formats/text.md](formats/text.md) | menu and UI text: the localization files |
+| [formats/upk.md](formats/upk.md) | UE3 packages: summary, LZO chunks, how ackit reads and patches them |
 | [formats/wem.md](formats/wem.md) | WEM layout the game accepts, setup stripping, hash and decode-alloc formulas |
 | [reverse-engineering/decoder.md](reverse-engineering/decoder.md) | BatmanAC.exe decoder functions and addresses |
 | [reverse-engineering/findings.md](reverse-engineering/findings.md) | root cause, findings, in-game experiment log (incl. failed hypotheses) |
 | [../data/catalog/README.md](../data/catalog/README.md) | catalog CSV schema, adding languages |
-| [history/](history/) | the two mission briefs this project was built from |
+| [history/](history/) | the mission briefs this project was built from |

docs/building.md

@@ -1,7 +1,8 @@
 # Building Unheard City
 
 The build is a PyInstaller **one-folder** bundle: Python, Qt (incl. Qt Multimedia), libsndfile (libvorbis,
-FLAC, opus, mpg123) and soxr are all inside; nothing else is installed or needed at runtime. Build on each
+FLAC, opus, mpg123), soxr and pedalboard (the effects) are all inside; nothing else is installed or needed at
+runtime. Build on each
 target OS (no cross-compiling).
 
 ## Windows (x86-64)
@@ -15,8 +16,8 @@ That one command does everything, offline. If `app/assets/` (committed icon and
 made from `bac_logo_full.png` and `wallpaper.jpg` in `Downloads\Arkham City` (or `--art FOLDER`); without those
 images the build still works, with a plain icon and no wallpaper.
 Output: `dist\UnheardCity\UnheardCity.exe` and `dist\UnheardCity-<ver>-windows-amd64.zip`
-(~78 MB zip, ~190 MB unpacked). The build also writes `THIRD_PARTY_LICENSES.txt` (license texts come from the
+(~86 MB zip, ~205 MB unpacked, catalogs included). The build also writes `THIRD_PARTY_LICENSES.txt` (license
-wheels and from `tools/licenses/`).
+texts come from the wheels and from `tools/licenses/`).
 
 **Installer.** On Windows the folder (and zip) also contains `Install.cmd`, `install.ps1` and `uninstall.ps1`
 (sources in `tools/installer/`, version filled in by the build). It is a plain PowerShell installer, so no
@@ -42,6 +43,9 @@ QT_QPA_PLATFORM=offscreen pytest -q
 python tools/build_app.py          # -> dist/UnheardCity-<ver>-linux-x86_64.tar.gz
 QT_QPA_PLATFORM=offscreen dist/UnheardCity/UnheardCity --smoke-test --game /path/to/game
 ```
+First built with version 3.4.0 (Debian 13 in a container, Python 3.14.8 built from source because Debian has no
+3.14 package): ~124 MB tar.gz, ~330 MB unpacked; the smoke test passes with `PATH=/usr/bin:/bin` and the GUI
+starts.
 Notes:
 - The `soundfile` and `soxr` wheels for manylinux bundle libsndfile/libsoxr; the PySide6 wheels bundle Qt and
   Qt Multimedia's FFmpeg backend (PulseAudio/PipeWire output). The target desktop only needs the usual X11/

docs/categories.md

@@ -4,8 +4,9 @@ Implemented in `ackit/classify.py`; used both when building the shipped catalog
 and when a game build is not in the catalog (runtime scan), so every language/build gets the same scheme.
 
 ## Groups and categories
-Every sound has a **group** and a finer **category**, derived from its Wwise name (from the community hash
+Every sound has a **group** and a finer **category**, derived from its name (taken from the game files: the
-list; unnamed IDs show as hex). First matching rule wins:
+soundbank names in `CookedBanks.txt` and the Wwise event names; the 3 SFX entries no name reaches show as hex).
+First matching rule wins:
 
 | name starts with | group | category |
 |------------------|-------|----------|
@@ -25,40 +26,63 @@ list; unnamed IDs show as hex). First matching rule wins:
 | `Speech_CHAR_` | Speech | Character lines |
 | `Speech_CH_`, `Speech_SEQUENCE_`, `Speech_STEEL_`, `_MUSEUM_`, `_CHURCH_`, `_COURT_`, `_UNDER_`, `_GCPD_`, `_ZP1_` | Speech | Story missions |
 | `Speech_GAME_`, `Speech_MAP_`, `Speech_AUGMENTED_`, `Speech_FORENSICS_`, `Speech_TEMP_` | Speech | Gameplay & hints |
-| `NPC_` | Ambience | Crowds |
+| `NPC_...Crowd...` | Ambience | Crowds |
 | other `Speech_` | Speech | Other speech |
-| media of the `Music` / `Music_PDLC` stream packs, `Music*` | Music | Music / Music (DLC) |
+| media of the `Music` / `Music_PDLC` stream packs, `Music*`, `MUS_` | Music | Music / Music (DLC) |
 | `AMB_`, `PSA_` | Ambience | Ambience |
+| `PSO_` (flies, drips, fans, lights) | Ambience | Ambient emitters |
 | `WND_` | Ambience | Wind |
+| `WTR_` | Ambience | Weather |
 | `VOL_`, `PVOL_` | Ambience | Room tone |
-| `SEQ_` | SFX | Scripted sequences |
+| anything containing `Footstep` | SFX | Footsteps |
+| `CHR_`/`NPC_` containing `Combat`, `BoneBreak`, `Bodyfall` | SFX | Combat |
+| `SEQ_`, `MTN_` | SFX | Scripted sequences |
 | `OBJ_` | SFX | Objects |
+| `PX_`, `PHY_` | SFX | Physics |
 | `WPN_` | SFX | Weapons |
-| `CHR_` (in SFX) | SFX | Character foley |
+| `DVC_` | SFX | Gadgets |
+| `GUI_`, `SBJ_` | SFX | Interface |
+| `VFX_` | SFX | Visual effects |
+| other `NPC_` | SFX | Enemies & NPCs |
+| `CHR_`, `GEN_` | SFX | Character foley |
 | `Global` | SFX | Global |
 | anything else (e.g. unnamed hex IDs) | Unknown | Unknown |
 
-Counts in the ITA catalog: Story missions 1993, Predator chatter 1796, Character lines 1537, Open-world chatter
+BANK sounds (in-memory, `docs/formats/banks.md`) are classified by the name of their **soundbank** (e.g.
-1295, Side missions 808, Combat barks 801, Efforts & emotes 616, Interview tapes 514, Riddler 360, Radio & TV 226,
+`CHR_CatwomanFoleySuit`, `Speech_EMOTE_Batman`), which is more systematic than their event names.
-Gameplay & hints 199, DLC 167, Boss fights 150, Cinematics 150, Reactions 108, Interrogations 95. SFX archive:
+
-Music 389, Ambience 225, SFX 31, Unknown 6.
+Counts per voice language (identical in all 5): Story missions 1996, Predator chatter 1796, Character lines 1537,
+Open-world chatter 1295, Side missions 808, Combat barks 801, Efforts & emotes 616, Interview tapes 514,
+Riddler 360, Radio & TV 223, Gameplay & hints 199, DLC 167, Boss fights 150, Cinematics 150, Reactions 108,
+Interrogations 95. SFX archive: Music 389, Ambience 225, SFX 31, Unknown 6. BANK (all languages): Combat barks
+4360, Character foley 3051, Other speech 2590, Footsteps 1256, Ambient emitters 705, Weapons 494, Combat 412,
+Objects 401, ...
 
 ## Characters
 Two separate fields, because Wwise names describe a *scene*, not the speaker of each variation (a conversation
 alternates speakers between `NAME_1`, `NAME_2`, ...):
 
 - **character** - the verified speaker, only from the curated lists in `data/catalog/source/<lang>/*.txt`
-  (one file per character, listening-based). Mapped by sound name, so labels made on the Italian files apply to
+  (one file per character, listening-based; the lists so far were made on the Italian voices, `source/ita/`).
-  every language (the five language WADs have identical names and variation counts). Display names:
+  A list names sounds by their position in its language's WAD (wemscan names); positions differ between
+  languages, so each listed recording is found at its position in that language and the label goes to its line
+  (`characters.csv`, keyed by the line's name), i.e. to the same recording in every language. A label applies to exactly the listed recordings (not to other versions of the same event).
+  Checked by `ackit verify --catalog` against the speaker field of the game's dialogue objects: 2,766 labelled
+  lines agree (spelling aside: Mister Freeze, Bruce Wayne, TwoFace, Mysterious_stranger = Azrael), 25 listed
+  as Unknown are named there, 19 differ (to listen to: `Batman_sword_raised_01-04` game: Ra's al Ghul,
+  `ENCRYPTION_CARD2_02` game: Hugo Strange, `thug1_heckle_01_3` game: Thug 01, `SIDE_CAT_TF_intro1_03` game:
+  Thug 06; the 12 Wonder City lines say "Announcement").
+  Display names:
   Calendar -> Calendar Man, Croc -> Killer Croc, Freeze -> Mr. Freeze, Ivy -> Poison Ivy, Madhatter -> Mad
   Hatter, Quinn -> Harley Quinn, Ras -> Ra's al Ghul, Strange -> Hugo Strange, Talia -> Talia al Ghul,
   Twoface -> Two-Face, Vicky -> Vicki Vale, Zsazz -> Victor Zsasz, Unknown -> Unknown (speaker checked but not
   identified). A line in two lists gets both ("A; B").
 - **characters_inferred** - characters and roles *mentioned by the name* (token rules in `classify._TOKENS`,
   e.g. `BM`/`BATMAN` -> Batman, `PENG` -> Penguin, `TF` -> Two-Face, `THUG*`/`GENTHUG*` -> Thug, `TYGER` ->
-  TYGER guard; interview tapes add Hugo Strange). A hint for the ~7,500 lines without a verified speaker,
+  TYGER guard; interview tapes add Hugo Strange). A hint for the ~7,400 lines without a verified speaker,
   never shown as the speaker. In the editor the character filter matches verified speakers, and optionally
   inferred ones.
 
-Coverage (every language): 3,366 sounds with a verified speaker, 5,930 more with inferred characters, 1,520 with
+Coverage of the 10,811 WAD voice lines (every voice language, the same lines): 3,366 with a verified speaker,
-neither (mostly predator/efforts from unnamed thugs).
+6,258 more with inferred characters, 1,187 with neither (combat barks 407, story missions 267, open-world
+chatter 228, radio & TV 122, ...).

docs/decisions.md

@@ -3,7 +3,7 @@
 ## 1. Stack: Python + Qt (PySide6), packaged with PyInstaller
 - **Reuse:** the toolkit (`ackit`) is Python and proven against the real game; the editor imports it unchanged.
 - **Cross-platform GUI:** Qt Widgets via PySide6 (LGPL-3.0, official Qt for Python) looks native on
-  Windows and Linux, has a fast model/view table for 11,000+ rows, and runs offscreen for automated GUI tests.
+  Windows and Linux, has a fast model/view table for 20,000+ rows, and runs offscreen for automated GUI tests.
 - **Audio I/O:** Qt Multimedia (`QMediaDevices`, `QAudioSink`, `QAudioSource`) for playback, recording and
   device selection. It ships inside the PySide6 wheels with its own backends (WASAPI on Windows,
   PulseAudio/PipeWire on Linux), so unlike PortAudio/`sounddevice` nothing has to be installed on Linux.
@@ -22,6 +22,13 @@ The first toolkit used ffmpeg (decoding inputs and libvorbis encoding) and vgmst
   ffmpeg: an ffmpeg found on the system is used if present, otherwise the user gets a clear "convert to WAV/MP3"
   message. Bundling ffmpeg (~80-100 MB, GPL/LGPL build care) for rare formats was judged not worth it.
 - **Resampling:** `soxr` (LGPL-2.1).
+- **Effects for replacements (any sound):** `pedalboard` (Spotify, GPL-3.0, compatible with this project's GPL-3.0; it
+  statically contains JUCE and the Rubber Band library): filters, distortion, codec, delay and pitch/formant
+  shifting (`ackit/fx.py`). The reverb is not pedalboard's (Freeverb sounded metallic): a generated room response
+  (early reflections + a 4-band noise tail, the highs decaying first) convolved with numpy's FFT. Chosen over
+  writing the DSP ourselves (user: "just use some libraries") and over scipy (no ready pitch shifter). Its
+  high/low-pass filters are first order only, so the voice-band cuts are Butterworth magnitudes applied with
+  numpy's FFT. Wheels for Python 3.14 on Windows and Linux (0.9.26); it needs only numpy.
 So there are **no vendor binaries to fetch** (no `tools/fetch_vendor` script was needed): every native
 dependency comes from version-pinned PyPI wheels (which embed libsndfile, libvorbis, libogg, FLAC, opus, mpg123,
 Qt and Qt Multimedia's FFmpeg backend) that PyInstaller bundles, resolved relative to the app (never `PATH`).
@@ -30,7 +37,7 @@ plus official texts kept in `tools/licenses/` (sources in SOURCES.txt; no networ
 its decode/encode/inject smoke test with `PATH` reduced to `C:\Windows` (no ffmpeg, vgmstream or Python).
 **Licenses:** PySide6/Qt LGPL-3.0 (dynamic linking, unmodified Qt shared libraries in the one-folder build,
 source at https://code.qt.io), libsndfile LGPL-2.1, mpg123 LGPL-2.1, soxr LGPL-2.1, libvorbis/libogg/FLAC/opus
-BSD, numpy BSD, Python PSF. One-folder (not one-file) builds keep the LGPL libraries replaceable.
+BSD, numpy BSD, pedalboard GPL-3.0, Python PSF. One-folder (not one-file) builds keep the LGPL libraries replaceable.
 The only Windows-only part is the in-game test harness (`ackit verify --ingame`, uses WinDbg/cdb and WASAPI
 loopback); it is a developer tool and not part of the editor.
 
@@ -38,30 +45,114 @@ loopback); it is a developer tool and not part of the editor.
 - **Originals store** per game folder: `<user data>/UnheardCity/stores/<id>/` (Windows `%APPDATA%`, Linux
   `~/.local/share/unheardcity`), id = hash of the game path; configurable (`--store`, editor settings).
   Before a sound is first changed, its original WEM bytes (SHA-256-named blob) and metadata are saved;
-  the applied replacement is saved too. **Revert** re-injects the stored original; **Revert all** does this for
+  the applied replacement is saved too. **Revert** re-injects the stored original; **Revert all** (editor toolbar,
-  every record. Stream-pack padding bytes are stored so pack reverts are byte-identical.
+  `ackit revert --all`) does this for every record of the store, sounds and texts, in every language. Stream-pack
+  padding bytes are stored so pack reverts are byte-identical. The uninstaller counts sound and text records and
+  does not offer to delete the store while any is left.
+- **One store per game at a time:** the changes are recorded only in the store that made them. The editor
+  refuses to switch to another store folder while changes are applied (Revert all first). A store takes the
+  full copy of a soundbank or package only from a file that matches the shipped catalog (every in-memory
+  sound's checksum, every bank copy of a voice recording, every subtitle; `Workspace.unoriginal_files`), so a
+  second store (`--store`) or another tool's change is a conflict, never a new "original". (Menu files and other
+  game builds have no catalog checksums to check.)
 - **Full-file fallback:** before the first change to a WAD (and its pack index), one verified full copy is kept
-  in the store (`full/`), default on, can be disabled. Used by "Restore from full backup" if the store is damaged.
+  in the store (`full/`), default on, can be disabled. If the store is damaged, `ackit restore-backup --yes`
+  copies the WADs back (their sounds are forgotten) and rebuilds the soundbanks, packages and localization files
+  from their copies, keeping the sounds of WADs without a full copy recorded and revertible.
+  Soundbank files, packages and localization files always get this copy (they are rebuilt from it, 4b/4c).
 - **Why the user data folder:** it survives app updates and reinstalls, needs no write access next to the game,
   and works the same on Linux/Proton. Steam never touches it.
 - **Safe writes:** every WAD is rebuilt into a temp file and atomically renamed over the original, pack indexes
   first. Several sounds of one WAD are written in one pass. A locked WAD (game running) gives a clear message.
+  Writes are serialized (`Workspace.lock`): the editor runs them as background jobs, and two rebuilds of one
+  package at once (e.g. a sound and its subtitle applied side by side) would otherwise lose one edit.
 - **Changed underneath:** each sound's state is computed from its bytes: original / modified (ours) / lost
   (ours was replaced by the original, e.g. Steam "verify") / changed (neither: update or another tool) /
   foreign (never touched by us, but not the catalog's original checksum). Writing over changed/foreign sounds
-  requires explicit confirmation; the catalog is used only when the WAD fingerprint matches.
+  or texts requires explicit confirmation (editor: a question, also when installing a mod; CLI: `--force`); the
+  catalog is used only when the game's fingerprint matches.
 
 ## 4. Size rule
 Speech/ambience variations are addressed by offset+size from BKPK tables inside compressed `.upk` packages,
-which ackit does not rewrite. Replacements are written in place and must not exceed the original WEM's size;
+which ackit leaves unchanged (it patches packages only in place). Replacements are written in place and must not
+exceed the original WEM's size;
 the encoder lowers quality, then sample rate, to fit, and refuses otherwise. Music stream packs have a loose
-index (`SFX\*.bnk`) that ackit rewrites, so music can be any length. (Rewriting `.upk` packages would lift the
+index (`SFX\*.bnk`) that ackit rewrites, so music can be any length. In-memory sounds follow the same rule
-limit but means LZO recompression of large engine packages: out of scope, high risk.)
+(below). Lifting the limit would mean re-laying out the uncompressed packages (export offsets, BKPK/DIDX
+tables): high risk for little gain, not done.
+
+## 4b. In-memory sounds (soundbanks in loose `.bnk` files and `.upk` packages)
+- **Model:** one catalog row per Wwise media ID (archive `BANK`, target = media ID), not per bank: the same
+  media is stored in up to dozens of bank copies (banks are cooked into every package that uses them), and the
+  game may load any of them. A replacement is written into **every** copy (`BANK_locations.csv.gz`).
+- **In place:** each copy keeps its size; DIDX/HIRC offsets and the package layout never change, so nothing
+  else in the engine data has to be rewritten. Unlike WAD speech, an in-memory copy is never zero-padded:
+  the replacement fills its slot exactly with a label list chunk (the game plays a padded in-memory WEM as
+  silence; `docs/formats/banks.md`). Shrinking the declared sizes (DIDX and every Sound object's in-memory
+  size) would also work, but needs every HIRC source of the media in every bank copy located and rewritten.
+- **Packages:** ackit reads the LZO packages itself (pure Python, ~47 MB/s, block-level access) and writes
+  only the changed 128 KiB blocks, stored as a single LZO literal run (valid LZO1X without matches), then
+  fixes the chunk table. No compressor and no external tool are needed; untouched blocks stay byte-identical.
+- **Revert:** before the first change to a bank file a verified full copy goes to the store (mandatory, not
+  optional like the WAD copies: the file is rebuilt from it). The file content is always "original copy +
+  replacements applied now", so reverting the last one restores the original bytes exactly, and an interrupted
+  operation is repaired by the next one. Conflicts: the whole-file SHA-1 must be the original's or the last one
+  ackit wrote, and before the copy is taken the file must match the catalog (section 3).
+- **Prefetch heads and whole copies:** 396 voice recordings (81 lines, 78-80 recordings per language) have
+  their first bytes copied into banks, and 49 more are stored whole in banks (same media ID, identical bytes);
+  a replacement of such a recording rewrites those copies too, otherwise the game could play the old line.
+  They are not listed as sounds of their own.
+- **Names:** no list names these media. Names come from Wwise events whose names are in the
+  packages' name and export tables (FNV-1 of the name = event ID), following event -> action -> containers ->
+  sounds (13,501 of 15,355, voice lines included); the other 1,854 are numbered per bank. Categories use the bank name. A localized speech line
+  has the same name in every language (its line, section 5).
+- **Localized speech:** speech banks hold one copy per voice language (4d); every media is labelled with the
+  language(s) whose banks hold it (a few recordings are shared by several languages) and the catalog shows
+  only the selected language's lines.
+
+## 4c. Subtitles and menu text
+- **Subtitles** are written in place into every copy of the dialogue object, NUL-padded inside the original
+  string (the game stops at the first NUL), so nothing in the package moves; the limit is the original
+  string's length ("room"). Moving the object to the end of the package (new export offset/size and an extra
+  chunk) would lift the limit, but no dialogue is loaded on the title screen, so the result could not be
+  checked automatically in the game: not done.
+- **Links** subtitle -> speech sound come from the soundbanks (event -> ... -> per-language media -> WAD line
+  or localized in-memory line), not from names: the `_NN` suffixes do not follow the WEM order inside WAD
+  entries. Scripted subtitles (`RSubtitle`) are linked through the game's own lookup table (FNV-1 of the line
+  name without `Speech_`). The subtitle objects' names also find events that no package name table lists, and
+  an event that plays two versions of a line links its subtitle to both.
+- **Menu text** files are plain UTF-16 text: no size limit; only the edited values change.
+- Both use the originals store like sounds: texts are recorded as strings (`texts` in store.json) and every
+  file is rebuilt from its original copy + the applied edits (sound and text edits combine in one package).
+- Keys (`SUB:<LANG>:<speech>:<line>`, `MENU:<LANG>:<File>:<Section>:<Key>`) are stable and used in mod packages.
+
+## 4d. One language setting
+The editor has a single *Language* list (top bar) for voices, subtitles and menu text, starting at the language
+the game is set to on every launch (not remembered: the game's setting is what the player hears). The game
+has voices in 5 languages and texts in 11; for the other 6 it plays the INT voices, so the editor loads the INT
+voice lines with that language's texts and says so next to the list. Localized in-memory speech (one bank
+copy per voice language) is filtered the same way, so the sound list never mixes languages.
 
 ## 5. Catalog
 - Shipped CSVs per archive (names, categories, characters, original size + checksum) make loading instant and
   enable change detection; any other build is scanned at runtime (header reads only, cached).
+- Names come from the game files only: a WAD entry or stream pack is named after its soundbank in the game's
+  `CookedBanks.txt` (entry ID = FNV-1 of the bank name; this gives exactly the names of the community hash list
+  the toolkit used to ship, which was removed), speech lines after their events (below).
 - Verified speakers come only from the user's lists; name-based character inference is kept in a separate field.
+- Voice lines are identified by the game's own data, not by names or positions (`docs/formats/banks.md`,
+  *Lines*): the recordings that are sources of the same Wwise Sound objects are one line, with exactly one
+  recording per language (12,434 lines, all 5 languages). The catalog stores that identity (`line`), and
+  speaker labels, mod packages (another language: the same `line`) and subtitles follow it. Positions are
+  useless across languages (WEMs are stored in media-ID order; 81.2-81.6 % of the WEMs in multi-variation entries
+  are at another position than in INT).
+- A line has one name in every language: the most specific speech event that plays it (10,174 of the 10,811
+  WAD voice lines), numbered `NAME#2` in the event's own order when it plays two versions; the other 637 keep
+  the name of their English (INT) recording's position. The position in each language stays an alias (`NAME#i`).
+- A recording stored twice in a WAD (5 per language) is written and reverted in both places.
+- The catalog records the ORIGINAL bytes and texts: `tools/build_catalog.py` stops before writing anything if a
+  sound or a subtitle differs from the current catalog (a modified game), unless `--accept-changes` (a game
+  update).
 - Category scheme: `docs/categories.md`.
 
 ## 6. Repository cleanup choices
@@ -78,8 +169,22 @@ limit but means LZO recompression of large engine packages: out of scope, high r
   store does not need it.
 
 ## 7. Editor UX choices
-- One window: filters (left), sound table (top), editor of the selected sound (bottom). The primary flow
+- One window with four tabs; browsing and editing are separate (user, mission 4: "the browser for browsing, the
-  (search -> select -> Space to listen -> Load file/Record -> Apply) needs no menus; power features (tags,
+  edit tab for editing, not both for both"). *Sounds*: filters (left), sound table, and below it a compact
+  preview (details, subtitle as text, original/installed waveforms with play buttons, **Edit** / **Add to edit
+  list**); no editing tools: details | player (Play original, Play installed, Stop together above the
+  waveforms) | Edit, Add to edit list. *Edit*: the edit list (sounds sent from the browser, kept in the user
+  data folder with each sound's replacement audio, trim/level values and effects; `app/edit.py`) | the editor
+  (`app/editor.py`, top to bottom: SUBTITLE large, with the original and the English line, room counter and
+  buttons on one row; LISTEN = original, installed and replacement, one row each with its play button, the
+  waveforms take the free height, transport row; REPLACEMENT = Record / input / level / Load file, trim and
+  level in an aligned grid) with its action bar (Previous | Apply to game | Revert | Save WAV | Next) pinned
+  below | the effects (`app/effects.py`: a preset list, the effect chain = one line per effect, tick = on, click
+  = its settings shown below: preset buttons and sliders with readable values, logarithmic for frequencies and
+  times). No scrolling at 1400 x 900 and 1280 x 800 (tested). Wording is generic (replacement, effects), not
+  dubbing-specific: the tab edits every kind of sound. *Subtitles* and *Menu text*:
+  search, filters, table, editor. The primary flow
+  (search -> select -> Space to listen -> Edit -> Record/Load file -> Apply -> Next) needs no menus; power features (tags,
   favorites, multi-select revert, mod packages, settings) are in the toolbar and context menu.
 - Long work (loading, encoding, writing 0.5 GB WADs) runs in background jobs with status messages; the
   window stays responsive. Errors are shown as plain sentences (game running/locked file, wrong folder,
@@ -95,6 +200,30 @@ limit but means LZO recompression of large engine packages: out of scope, high r
 - Background: the wallpaper sits only behind the filter panel, scaled to the panel height so its left part fills
   the panel, under a light veil. Settings -> Background toggles it and sets its strength (default 10 %).
 - Loading feedback stays where the work is, never modal: a spinner inside the waveform being loaded (sound or
-  replacement file, decoded off the GUI thread), next to the Apply/Revert buttons while encoding/writing, and in
+  replacement file, decoded off the GUI thread), next to the Apply/Revert buttons while encoding/writing, over
-  the status bar for game loading and bulk changes.
+  a list while it loads (the sound list at start and on a language change, the subtitles / menu text of a
+  language, read off the GUI thread), and in the status bar for game loading and bulk changes.
+- Lists filter and sort in Python, not with Qt's QSortFilterProxyModel (`app/models.py: RowView`): the view
+  computes the visible rows in one pass and one sort over cached keys (~10-20 ms for 21,000 sounds; the Qt
+  proxy called back into Python for every comparison: up to 1.5 s per sort or widened filter). Rows a filter
+  shows again keep their sorted place; selection and the current row survive every change.
   All filter selectors (including Character) are plain, non-editable drop-downs so they look and behave alike.
+
+## 8. Verification
+- `pytest`: unit tests, a synthetic game (WADs, packs, packages with banks, localized banks, dialogue and subtitle
+  objects, localization files), a two-language game built like the real one (per-language BKPK tables, a media
+  ID shared by both languages, a recording stored twice, one also stored whole in a bank), offscreen GUI
+  tests, and read-only real-game tests when the game is installed.
+- `ackit verify` (WEMs rebuilt byte-for-byte, encode/decode), `ackit verify --catalog` (catalog vs game files,
+  and every relation re-derived from the game files with independent checks; it prints them as facts).
+- Real-game write round trips without launching the game: `tools/e2e_features.py [--lang all]` (every copy of
+  an in-memory sound, localized line, prefetch heads and whole bank copies of a line, subtitles incl. RSubtitle, menu text, sound + subtitle in
+  one package, a mod moved to another voice language; `all` = each of the 11 languages) and
+  `tools/e2e_editor.py` (through the editor's code path).
+- Documentation: `tools/check_docs.py [--scan] [--wems]` recomputes every figure the documents state (from the
+  catalog, the build's `index.json`, the menu files; with `--scan` from a fresh package scan and the catalog
+  check's facts; with `--wems` from rebuilding every WAD WEM) and fails on any difference, so figures are never
+  copied by hand.
+- In the running game: `ackit verify --ingame` (music heard), `--banks` (patched packages loaded, probe data in
+  memory), `--texts` (menu texts and Startup subtitles found in memory). Every tool reverts in a `finally` block
+  and checks that every touched file is byte-identical afterwards.

docs/formats/banks.md

@@ -2,32 +2,39 @@
 
 Most short sound effects (footsteps, foley, combat, weapons, objects, UI, grunts) are **not in the WADs**: they
 are WEMs stored inside Wwise soundbanks, either loose in `SFX\*.bnk` or embedded in the UE3 packages (`.upk`).
-The catalog and the editor only cover the WADs (`wad.md`), so none of these sounds are listed or editable yet.
+ackit lists them as the catalog archive `BANK` and writes them (see the end of this page).
-Survey of the installed game (buildid 3718300) on 2026-10-09, read-only.
+Survey of the installed game (buildid 3718300), 2026-10-09.
 
 ## Inventory
 
-| where | banks | unique in-memory WEMs | in the catalog |
+| where | banks | unique in-memory WEMs |
-|-------|-------|-----------------------|----------------|
+|-------|-------|-----------------------|
-| loose `SFX\*.bnk` (Batman/Catwoman footsteps, 32 surfaces each, + `TST_Prep_Media`) | 65 | 1,117 | 0 |
+| loose `SFX\*.bnk` (Batman/Catwoman footsteps, 32 surfaces each, + `TST_Prep_Media`) | 65 | 1,117 |
-| banks embedded in `.upk` packages (in 1,432 of the 2,650 packages) | 3,340 | 15,641 (~276 MB) | 0 |
+| banks embedded in `.upk` packages (in 1,431 of the 2,648 packages) | 3,340 | 14,802 |
 
-The two sets share 119 media IDs: about **15,800 unique sounds** in total. The WADs, by comparison, are fully
+All of them are in the catalog (archive `BANK`).
-covered: the BKPK tables in the packages reference 2,543 WAD entries, all of them in the catalog.
 
-Embedded WEMs by bank name prefix: `Speech` 7,872, `CHR` (character foley, combat, bosses) 3,896, `PSO` 781,
+The two sets share 119 media IDs: 15,800 unique media IDs in 95,887 places (bank copies). 396 of them are
-`WPN` 664, `NPC` 475, `OBJ` 439, `SEQ` 265, `GUI` 208, `PSA` 203, `MTN` 196, `PX` 195, `DVC` 100, `GEN` 80,
+**prefetch heads** of streamed WAD sounds and 49 are **whole copies** of WAD recordings (below), so there are
-`PHY` 73, `AMB` 65, `SBJ` 50, others < 30. The `Speech_` banks (e.g. `Speech_EMOTE_Batman`, `..._GRUNT`) are
+**15,355 in-memory sounds**. All copies of a
-probably efforts/grunts rather than dialogue (there are no per-language packages); not verified by listening.
+media ID are byte-identical except `1C898731` (two variants). The WADs, by comparison, are fully covered: the
-All WEMs seen are Vorbis (format tag 0xFFFF), the same codec as the WADs: the footsteps are mono (4 stereo),
+BKPK tables in the packages reference 2,543 WAD entries, all of them in the catalog.
-the embedded ones mono or stereo (~22 % stereo in a 2,533-WEM sample).
+
+Embedded WEMs by bank name prefix (a media held by banks of two prefixes counts for both): `Speech` 7,827, `CHR`
+(character foley, combat, bosses) 3,624, `PSO` 737, `WPN` 577, `NPC` 471, `OBJ` 411, `SEQ` 253, `PSA` 203,
+`PX` 195, `MTN` 191, `GUI` 155, `DVC` 100, `GEN` 80, `PHY` 73, `AMB` 61, `SBJ` 50, others < 30. The `Speech_` banks (e.g. `Speech_EMOTE_Batman`, `..._GRUNT`) hold
+speech: efforts, grunts and a few other lines stored once for every language (119 sounds) and **localized
+lines** (about 1,450 per language, see *Localized banks*).
+All WEMs seen are Vorbis (format tag 0xFFFF), the same codec as the WADs: the loose footsteps are mono (4
+stereo); of all 15,355 in-memory sounds 13,982 are mono, 1,196 stereo, 177 have 3-6 channels.
 
 ## Names
-- The community hash list names none of these media IDs; only the bank names are known.
+- Media IDs have no names of their own: a sound is named after the Wwise event that plays it (event names
+  are in the packages' name and export tables, event ID = FNV-1 of the name), else after its bank.
 - `CookedPCConsole\CookedBanks.txt` lists all 3,345 cooked bank names. Bank ID = FNV-1 32 of the lowercased
-  name (the same hash Wwise uses for every name). 3,338 of them are embedded in packages, 4 are loose `.bnk`
+  name (the same hash Wwise uses for every name). 3,338 of them are embedded in packages (`Music`, `Music_PDLC`
-  files (`Init`, `Global`, `Music`, `Music_PDLC`); `AMB_OW_A6`, `AMB_OW_C1`, `AMB_OW_E1`,
+  also as loose `.bnk` files), 2 are only loose `.bnk` files (`Global`, `Init`); `AMB_OW_A6`, `AMB_OW_C1`,
-  `Speech_EMOTE_Thug_Muffled_Stop` and `Speech_GFxUI_ArkhamCityStories` were found nowhere.
+  `AMB_OW_E1`, `Speech_EMOTE_Thug_Muffled_Stop` and `Speech_GFxUI_ArkhamCityStories` were found nowhere.
 - Loose footstep banks are named in their STID chunk (`CHR_BatmanFootsteps_Concrete_Media`, ...).
 
 ## Bank layout (version 56, Wwise 2011.1)
@@ -40,26 +47,86 @@ HIRC Sound object (type 2) payload: u32 object id, u32 plugin id (`0x00040001` =
 (0 = in memory, 1 = streamed, 2 = prefetch), u32 media id, u32 bank id of the media; for in-memory sources
 then u32 offset, u32 size, where offset = `DATA` payload position (from the start of the bank) + DIDX offset
 and size = DIDX size (checked on every in-memory source of the footstep banks and of `BmGame.upk`).
-A few sources use media of another bank. Every streamed media ID found is in a BKPK table, i.e. a WAD sound
+Source kinds over all bank copies: 115,737 in-memory sources of the bank's own media; 2,233 in-memory sources
-(`wad.md`).
+with bank id 0 whose media is not in the same bank (loaded by another bank); 133,075 streamed sources;
+2,101 **prefetch** sources. Every streamed or prefetched media ID is in a BKPK table, i.e. a WAD sound
+(`wad.md`). A prefetch source's media is also in the bank's DIDX: that in-memory copy is the **head** of the
+streamed WEM (its size = the source's size), which the game plays while the rest is read from the WAD. 396
+WAD sounds have such heads (2,101 copies); replacing one of them must rewrite the heads too.
+49 more WAD recordings (9-11 per language: game-over taunts, an encryption-card line, a cutscene line, scripted
+story lines) are
+also stored **whole** in banks: an in-memory source with the WAD recording's own media ID, byte-identical to
+the WAD WEM (84 copies). They are handled like heads: part of the WAD line, written and reverted with it, not
+sounds of their own (`catalog.whole_recordings`). Heads and whole copies together: 445 recordings, 2,185
+copies (`BANK_locations.csv.gz` kind `p`).
+
+## Localized banks
+Speech banks are localized inside the (language-neutral) packages: one `AkBank` export starts with a language
+list (FStrings `INT FRA ITA DEU ESN`) and then holds one block per language, in that order: the BKPK table(s)
+of the speech WAD entry for that language (`wad.md`) and a complete bank, with the same bank ID in every
+language. ackit gives each bank and each BKPK table the language of its position in that list (an export
+whose block count differs from its language count would be reported; none does). Each in-memory media gets
+the languages of the banks that hold it (`BANK.csv` column `language`): 8,092 sounds are language-neutral,
+1,450-1,456 belong to one voice language, 6 to several (5 shared by INT/FRA/ITA/ESN while DEU has its own, 1
+by INT/FRA/ITA); a media stored in the banks of every language is listed as `''` (the same recording
+everywhere; 167 of these are voice lines, e.g. efforts and radio chatter).
+A catalog shows the neutral sounds plus the loaded language's lines; names are unique per language.
+
+## Lines: one recording per language
+Implemented in `ackit/lines.py`, checked by `ackit verify --catalog`. A voice line exists once per language,
+and the game ties the recordings together itself:
+- Every WEM of a speech WAD has exactly one row in its own language's BKPK table (media ID, offset, size):
+  10,816 of 10,816 in each of the 5 languages, no row without a WEM. 657 media IDs are used by several
+  languages (cinematics, emotes, Clayface), each at another position in each language's WAD, so the tables
+  are kept per language and never merged.
+- Every recording, streamed, prefetched or in memory, is the source of a Wwise Sound object; Sound objects,
+  events, actions and containers have the same IDs in every language, only the media differ. A *line* = the
+  recordings that are sources of the same Sound objects: 12,434 lines (WAD and in-memory), each with exactly
+  one recording in each of the 5 voice languages (61,485 recordings), no Sound object with two recordings in
+  one language. Line identity (`line` column) = the smallest Sound object ID.
+- Prefetched sources count too: the 81 lines with a prefetch head (396 recordings over the 5 languages; mostly
+  the game-over taunts) are linked only through them. A WAD recording that a bank also stores whole (49) is
+  the same recording (same media ID), so its line is unchanged and the bank copies are written with it.
+- 5 recordings per language are stored twice in that language's WAD (same media ID in two entries, byte-
+  identical, e.g. `Speech_STEEL_A1_CH2_Thug_behinddoormuffled` and `DIA_SteelA1Muffled`): the second place
+  is a copy of the line (same `line`, keeps its position name) and is written and reverted with it, because
+  the game plays whichever place the loaded package's table names.
+- Not every line has its own recording in every language; the game shares some itself: in each voice WAD
+  616-651 lines are byte-identical to the INT recording (ITA 651: efforts and emotes 509, cinematics 117,
+  boss fights 10, story 8, radio/TV 4, character lines 2), 167 in-memory voice lines are stored once for every
+  language (efforts, radio chatter, crowds) and 6 DLC intro lines are one recording shared by INT/FRA/ITA(/ESN).
+  So some lines play the English recording in every language: that is the game's own data, which the catalog
+  shows as it is.
+- Independent checks (no BKPK involved): every WAD recording is named as a source by a Sound object of a bank
+  of its own language (54,055 of 54,055); every stored head or whole copy equals the start of the recording
+  its line gives it (2,185 of 2,185); a line's recordings have similar lengths across languages (11,761 of 12,434
+  within a factor 1.5; the others are different edits, e.g. a 0.55 s ESN line vs 5.46 s elsewhere).
 
 ## Banks inside packages
-- The packages are fully LZO-compressed (package version 805, licensee 101). For the survey they were unpacked
+- The packages are LZO-compressed UE3 packages (`upk.md`); ackit decompresses them itself and searches the
-  with Gildor's `decompress.exe` (gildor.org; used from the user's old modding folder, not part of this
+  uncompressed data for `BKHD` (banks) and `BKPK` (WAD indexes). (The first survey used Gildor's
-  repository) and searched for `BKHD` (banks) and `BKPK` (WAD indexes). `UnrealX.upk` (512 bytes) is not a
+  decompress.exe; `ackit/upk.py` gives identical data.)
-  readable package (wrong tag, probably encrypted).
+- Bank blobs in packages: 41,797 for 3,340 bank IDs. Many banks are copied into several packages (e.g.
-- Bank blobs in packages: 41,798 for 3,340 bank IDs. Many banks are copied into several packages (e.g.
+  `CHR_CatwomanFoleySuit` 31 times, `CHR_BatmanFoleyBelt` 25). 302 bank IDs occur as several blobs with
-  `CHR_CatwomanFoleySuit` 31 times, `CHR_BatmanFoleyBelt` 26). 302 bank IDs occur as several blobs with
   *different* media subsets (often one WEM each, several per package), so a bank's media set is the union
   of all its blobs.
 - Largest: `CHR_RobinFoleyCape` 300 WEMs, `CHR_BatmanFoleyCape` 282, `CHR_Clayface` 263,
   `CHR_NightwingFoley` 242, `CHR_CatwomanFoleySuit` 225, `CHR_MrFreeze` 176, `CHR_Combat` 166.
 
-## Consequences for editing (not implemented)
+## How ackit edits them
-- **Loose banks**: a replacement means rebuilding `DATA`/`DIDX` and the in-memory offsets/sizes in the HIRC
+- Catalog archive `BANK` (one row per media ID) + `data/catalog/BANK_locations.csv.gz` (every copy, and the
-  Sound objects of that bank (and in any other bank whose sources use its media). Whether the game accepts a
+  heads and whole copies of WAD speech recordings); built by `tools/build_catalog.py`, checked by `ackit verify
-  bank that changed size is untested.
+  --catalog`.
-- **Embedded banks**: the package must be rewritten (decompress, patch every blob of the bank, recompress
+- A replacement is written **in place into every copy** (no larger than the smallest copy), so DIDX/HIRC
-  LZO, fix the package's chunk table and export offsets/sizes). A sound must be replaced in **every** package
+  offsets and sizes and the package layout never change. Loose banks are patched directly; in packages only
-  that carries a copy, otherwise the game may load the original from another package.
+  the touched blocks are re-encoded (`upk.md`). A sound left in one package only would let the game load the
-- Either way: unverified in game so far; the same WEM rules apply (`wem.md`).
+  original from another one.
+- **Each copy is filled exactly**, never zero-padded: every in-memory WEM the game ships ends exactly at its
+  slot's end, and the game plays a replacement followed by zero padding there as **silence** (findings 24:
+  footsteps replaced that way were silent in game; filled, they play). A smaller replacement is grown to the
+  slot size with a label list chunk before `data` (`LIST{adtl, labl}`, the form the game's own WEMs use for
+  markers; `ackit/wem.py: fill`), so the encoder leaves at least 26 spare bytes, an even number.
+- Replacing a speech line that has prefetch heads or whole copies rewrites them: a head gets the first bytes
+  of the new, zero-padded line (the start of the streamed WEM as the WAD stores it); a whole copy is filled
+  like an in-memory sound.
+- Design and revert rules: `docs/decisions.md` (4b). The same WEM rules apply (`wem.md`).

docs/formats/subtitles.md

@@ -0,0 +1,64 @@
+# Subtitles (`RDialogueEvent` objects in the packages)
+
+There are no subtitle files: every speech line has a dialogue object (class `RDialogueEvent`, Rocksteady) inside
+the UE3 packages (`upk.md`), holding the subtitle text of all 11 languages. ackit reads them with
+`ackit/subtitles.py` and writes them through `Workspace.texts` (`ackit/texts.py`).
+
+## Inventory (buildid 3718300)
+- 14,744 dialogue objects in the packages for 9,032 speech lines (a line's object is copied into every package
+  that uses it; all copies are identical), plus 41 scripted subtitles (below): ~9,400 subtitle strings and
+  168,688 stored copies.
+- 11 languages: INT, FRA, ITA, DEU, ESN, ESM, POL, POR, RUS, JPN, KOR (the speech WADs exist only for INT, FRA,
+  ITA, DEU, ESN; the other languages play the INT voices with their own subtitles).
+- 9,035-9,667 subtitle strings per language: 94-98 % of the subtitles have one line (JPN: all), the rest up
+  to 5 lines.
+- Strings are stored as cp1252 (positive FString length) or UTF-16 (negative length): almost all RUS, JPN and
+  KOR strings and 85 % of POL are UTF-16; the other languages have 17-56 UTF-16 strings each.
+  4 cp1252 strings hold UTF-8 bytes instead (the same line in KOR, POL, POR, RUS: `Speech_TAPE_J_Answer_1_00`;
+  the KOR one with a damaged byte; the game's own data). ackit reads cp1252 as Windows maps it (every byte, also
+  the 5 that Python's cp1252 leaves undefined, which the RUS and KOR ones contain), so they are shown as stored
+  and every string is written back byte-for-byte.
+
+## Scripted subtitles (`RSubtitle`)
+41 more subtitles (radio and encryption-card lines) are `RSubtitle` objects in `Startup.upk`: the same layout,
+with the per-language array named `ArraySubtitles` and a float (duration) at `0x2C` instead of a speech
+reference. `RSubtitleLookup` (`Startup.upk`, `Lookups`) is an array of `{int key @0, object @4}`, key =
+FNV-1 of the speech line name without `Speech_` (e.g. `CH_ZERO_CH0_ENCRYPTION_CARD2_01`), which links each
+`RSubtitle` exactly to its line. ackit reads and writes both kinds the same way.
+
+## Object layout (compact tagged properties)
+The export data is a list of properties `u16 type, u16 field offset, value`, ended by `u16 0`. Types seen:
+`0x10` object reference (i32 export/import index), `0x0D` FString, `0x09` struct/array (name i32+i32 = struct
+or property name from the name table, i32 byte size, i32 array index, then `size` bytes), `0x04` 4-byte value,
+`0x03` bool, `0x01` enum. Relevant properties:
+
+| field | content |
+|-------|---------|
+| `0x10` @ `0x2C` | the speech event this line plays, e.g. export `Speech_CHURCH_B1_Ch2_POST_Explosion_BM_Cash_04` |
+| `0x0D` @ `0x34` | speaker, e.g. `Batman` (`ackit verify --catalog` compares it with the curated speaker labels) |
+| `0x09 LocalizedSubtitles` | u32 count, then per language: `0D 00 00 00` FString language code, `09 00 0C 00 Subtitles` struct, `00 00` |
+| `Subtitles` | u32 line count, then per line: `0D 00 00 00` FString text, `04 00 0C 00` 4-byte value, `00 00` |
+
+The sizes add up exactly in all 14,744 objects (checked by the parser: `SubtitleError` otherwise).
+
+## Link to the speech sound
+The speech event name is hashed (FNV-1) to the Wwise event ID and followed through the soundbanks (event ->
+action -> containers -> Sound objects, the same in every language) to that language's recording of the line
+(`banks.md`, *Lines*): a WAD variation or a localized in-memory sound. This is exact; the `_NN` suffix of the
+event name does **not** match the variation order inside the WAD entry (WEMs are stored in media-ID order).
+The names come from the packages' name and export tables (numbered names like `Speech_X_04` exist only as
+export names) and from the subtitle objects themselves: some events are listed in no name table, but their
+dialogue / RSubtitle object names them (Vicki Vale's radio reports, the Hush tape reveal).
+An event that plays two versions of a line (105 subtitled events, e.g. Mr. Freeze's lines) links its subtitle
+to both (`sound` = `key1;key2`). Linked: 103,304 of 103,658 strings in the 11 languages (INT 9,373 / 9,404),
+each to the same line in every language. The rest (24 speech names, 24-36 strings per language: Riddler trophy
+texts, Arkham City Stories, radio reports and tapes the game does not ship audio events for) have no event in
+any bank and stay editable without a sound link.
+
+## How ackit writes them
+- In place, into **every copy** (`data/catalog/SUB_locations.csv.gz`): the new text replaces the FString's
+  bytes and is padded with NUL characters, keeping the length field. The game stops at the first NUL, so no
+  size field, export or package layout changes. Limit: the new text must fit the original string's bytes
+  (cp1252: one byte per character; UTF-16: two) - shown as "room" in the CLI and the editor.
+- A cp1252 string can only hold cp1252 characters; UTF-16 strings can hold any character.
+- Packages are rebuilt from the store's original copy + all applied sound and text edits (`docs/decisions.md`).

docs/formats/text.md

@@ -0,0 +1,19 @@
+# Menu and UI text (`BmGame\Localization\<LANG>\*.<lang>`)
+
+The menus, HUD, character bios, riddles, credits and other UI strings are UE3 localization files, one folder
+per language (INT, ITA, FRA, DEU, ESN, ESM, POL, POR, RUS, JPN, KOR), ~37 files and ~7,000 strings each
+(`FRONTEND`, `GFxUI`, `Bios`, `Credits`, `PC`, `RIDDLER`, ...). ackit reads and writes them with
+`ackit/loctext.py` through `Workspace.texts`.
+
+## Format
+- UTF-16LE with a BOM (`FF FE`), CRLF line ends; the first line is empty. (`INT\Version.int` and
+  `VersionData.int` are plain ASCII build IDs, not UI text: ackit ignores them.)
+- `[Section]` lines, `Key=Value` lines, `;` comments. A few values are in double quotes (`Quit="Quit game"`).
+- File extension = language code (`FRONTEND.int`, `FRONTEND.FRA`); case varies, lookups are case-insensitive.
+- Markers inside values (e.g. `\n`, button tags) are the game's own; ackit keeps them as plain text.
+
+## Keys and writing
+- Key: `MENU:<LANG>:<File>:<Section>:<Key>` (`#n` for a repeated key; none in the shipped files).
+- Writing replaces only the value part of the edited lines (quotes kept); BOM, encoding, other lines and line
+  ends stay byte-identical. Values cannot contain line breaks; characters must fit the file's encoding.
+- The files are rebuilt from the store's original copy + the applied edits, so reverting restores them exactly.

docs/formats/upk.md

@@ -0,0 +1,48 @@
+# UE3 packages (`*.upk`)
+
+Arkham City's 2,648 packages in `BmGame\CookedPCConsole` are Unreal Engine 3 packages, version 805,
+licensee 101. 2,647 are fully LZO-compressed, 2 are stored uncompressed, and `UnrealX.upk` (512 bytes)
+is not a readable package. ackit reads and patches them with `ackit/upk.py` and `ackit/lzo.py` (no external
+tools). Soundbanks inside them: `banks.md`.
+
+## Summary (file start, little-endian)
+
+| field | notes |
+|-------|-------|
+| u32 tag `0x9E2A83C1`, u16 version 805, u16 licensee 101, i32 TotalHeaderSize | |
+| FString FolderName | i32 length incl. NUL, then bytes (`None`) |
+| 12 x i32 | PackageFlags, NameCount, NameOffset, ExportCount, ExportOffset, ImportCount, ImportOffset, DependsOffset, ImportExportGuidsOffset, ImportGuidsCount, ExportGuidsCount, ThumbnailTableOffset |
+| 16 bytes Guid | |
+| i32 N + N x {ExportCount, NameCount, NetObjectCount} | generations (N = 1) |
+| u32 EngineVersion 7748, u32 CookerVersion, u32 CompressionFlags | 2 = LZO, 0 = none |
+| i32 C + C x {i32 UncompressedOffset, UncompressedSize, CompressedOffset, CompressedSize} | chunk table |
+| u32 PackageSource, ... | not used by ackit |
+
+All offsets in the summary (names, exports, ...) refer to the **uncompressed** package. The uncompressed
+package = the summary without the chunk table, followed by the chunks' data at their UncompressedOffset
+(the first chunk starts right after the summary, at NameOffset).
+
+## Name, import and export tables (uncompressed package, at NameOffset / ImportOffset / ExportOffset)
+- Name: FString (positive length: ANSI incl. NUL; negative: UTF-16) + u64 flags.
+- Import (28 bytes): ClassPackage (name index, number), ClassName (index, number), i32 OuterIndex,
+  ObjectName (index, number).
+- Export: i32 ClassIndex, SuperIndex, OuterIndex, ObjectName (index, number), ArchetypeIndex, **one extra u32
+  (licensee 101)**, u64 ObjectFlags, i32 SerialSize, SerialOffset, u32 ExportFlags, i32 count + count x i32
+  (net objects), 16-byte guid, u32 package flags: 72 bytes when the net-object list is empty. Checked on every
+  export of the game. A name with number n is shown as `NAME_<n-1>` (e.g. the speech event `..._04`): such full
+  names exist only in the export table. `Package.tables()` returns names, imports and exports.
+
+## Compressed chunks
+Each chunk (at CompressedOffset): `u32 tag, u32 block size (0x20000), u32 compressed sum, u32 uncompressed
+sum`, then one `{u32 compressed size, u32 uncompressed size}` per block, then the blocks. Every block is an
+independent LZO1X stream. In all game packages the chunks follow each other without gaps, start right
+after the summary and end at the end of the file (checked for every package).
+
+## How ackit reads and writes them
+- Positions of sounds are offsets in the uncompressed package, the same as in a decompressed copy
+  (e.g. Gildor's decompress.exe output). Reading decompresses only the blocks that are needed.
+- Writing (`Package.patched`): every changed block is stored again as a single LZO literal run (a valid
+  LZO1X stream without matches, `lzo.compress_stored`), the chunk headers of the changed chunks and the
+  CompressedOffset/CompressedSize of all following chunks are updated; every other byte is copied, so a
+  package without changes is rebuilt byte-for-byte. Replacements never change the uncompressed layout
+  (sounds are replaced in place), so names, exports and their offsets stay valid.

docs/formats/wad.md

@@ -21,19 +21,27 @@ Rules:
 - Each payload starts at the next **0x1000-aligned** offset after the previous one.
 - `size` = payload length rounded up to **0x800**; padding is zero.
 - File ends 0x1000-aligned; header area after the table is zero.
-- `hash` = Wwise media/file ID (names: `hashlist.txt`, e.g. `CHR_RadioScanner`, `AMB_Steel_B3`, `Music`).
+- `hash` = Wwise ID of the soundbank whose streamed media the entry holds: FNV-1 of a bank name listed in
+  `CookedPCConsole\CookedBanks.txt` (e.g. `CHR_RadioScanner`, `AMB_Steel_B3`, `Music`); this names every
+  entry the game uses (2,544 of 2,547; ackit takes the names from there).
 
 Payload kinds:
-- **LOC WADs**: each entry is a group of WEMs, the variations of one line (ITA: 10816 WEMs in
+- **LOC WADs**: each entry is a group of WEMs, the variations of one line (every language: 10,816 WEMs in
-  2464 entries; e.g. `CHR_RadioScanner` holds 91). Each WEM starts at the next 0x800 boundary after
+  2,464 entries; e.g. `CHR_RadioScanner` holds 91). Each WEM starts at the next 0x800 boundary after
-  the previous one (zero padding); 3 entries pack two WEMs back to back. Addressed `HASH:i`.
+  the previous one (zero padding); 3 entries pack two WEMs back to back. Addressed `HASH:i`. The WEMs of an
+  entry are stored in media-ID order and the languages have different media IDs, so variation `i` is
+  usually a different line in each language (81.2-81.6 % of the WEMs in multi-variation entries, compared with
+  INT; e.g. `Speech_TAPE_JOKER_1_04` is variation 1 in INT and ITA, 8 in FRA, 6 in DEU, 5 in ESN). Which recording is which line comes from the banks
+  (`banks.md`, *Lines*), not from positions.
 - **SFX WAD**: 79 grouped ambience/effect entries (260 WEMs) plus 4 **stream packs** (below).
 
 Where the game gets the inner offsets: the entry IDs appear next to `BKPK` headers inside the
-LZO-compressed UE3 packages (e.g. `BmGame.upk`, `OW.upk`, `Church_B1_*.upk`). Those embedded BKPK
+LZO-compressed UE3 packages (e.g. `BmGame.upk`, `OW.upk`, `Church_B1_*.upk`). A speech entry has one table
-tables give each WEM's offset **and size**. ackit does not rewrite `.upk` files, so these WEMs are
+per language, inside a localized `AkBank` export (its k-th table belongs to the k-th language the export lists;
-replaced **in place, no larger than the original** (zero-filled to the original size), and the WAD
+the SFX WAD's 79 tables are outside such exports). Those embedded BKPK tables give each WEM's offset **and
-layout never changes. The user's earlier quickbms-reimport mod did the same and was loaded by the game.
+size**; in every language each WEM has exactly one row (`banks.md`, *Lines*). ackit never changes them,
+so these WEMs are replaced **in place, no larger than the original** (zero-filled to the original size), and
+the WAD layout never changes. The user's earlier quickbms-reimport mod did the same and was loaded by the game.
 Only the 4 SFX stream packs, whose BKPK index is a loose `.bnk`, can change size.
 
 ## BKPK stream packs (`SFX\<ID>.bnk` + SFX WAD entry `<ID>`)
@@ -61,9 +69,9 @@ end rounded up to 0x800. Padding is mostly zero (a few non-zero bytes in `EDF036
 original bytes where nothing moved).
 
 Replacing a pack media (`ackit` target `EDF036D6/393BC53A`) rewrites the pack table (`.bnk`), the
-WAD entry payload, and the WAD table/offsets. Verified in-game (NOTES.md, E2).
+WAD entry payload, and the WAD table/offsets. Verified in-game (findings E2, E14).
 
 ## Regular soundbanks
 
 The other `SFX\*.bnk` files and the banks embedded in `.upk` packages hold their WEMs in memory (not in a
-WAD); see `banks.md`. The toolkit does not read or modify them yet.
+WAD); ackit lists and writes them as the catalog archive `BANK` (`banks.md`).

docs/formats/wem.md

@@ -1,10 +1,16 @@
 # WEM format accepted by Arkham City (Wwise 2011.1, Vorbis)
 
 Derived from the game's WEMs; `ackit verify` / `tests/test_realgame.py` rebuild them byte-for-byte from these rules
-alone (all 10816 ITA WEMs; in SFX only the 4 looping music tracks with loop points differ).
+alone. Checked on every WEM of the 5 speech WADs: 54,079 of 54,080 are identical; the exception (INT
+`FE9963E8:74`) differs only in the informational avg-bytes/s field (one lower than the formula below). In SFX
+626 of 651 are identical; the others are 14 music tracks whose seek table follows another placement (one entry per
+1,024-1,103 samples), 5 music tracks whose sample count exceeds what their packets hold, and 6 4-channel
+ambience WEMs with a `smpl` (loop) chunk. These rules are not needed to write sounds the game plays (replaced
+music, built with the rules here, is heard in game: `ackit verify --ingame`), only to reproduce those originals.
 
 ## RIFF layout
-`RIFF` size `WAVE`, then `fmt ` (0x42 bytes), optional `cue `/`LIST` (37 files), `data`.
+`RIFF` size `WAVE`, then `fmt ` (0x42 bytes), optional `LIST` + `cue ` (659 WEMs), `LIST` + `cue ` + `smpl` (1 WEM),
+`cue ` (18 WEMs), `cue ` + `smpl` (3 WEMs), `smpl` (41 WEMs), `data`.
 Odd-sized `data` chunks are **not** padded.
 
 ### `fmt ` (0x42)

docs/history/mission3_audio_text_prompt.md

@@ -0,0 +1,35 @@
+# CONTINUE: Finish full-game audio editing, then add subtitle and menu-text editing
+
+Same autonomous rules as before: I won't answer questions, so decide, log assumptions in `NOTES.md`, and continue. Start by reading `NOTES.md`, `CLAUDE.md`, `docs/` and `docs/decisions.md` to restore context, and apply the new findings from the last session. Keep the Steam, backup and byte-identical-originals rules.
+
+## 1. Complete audio editing for every sound in the game
+- Make editing work for every audio file in the game (all WADs and languages), using the new findings. Handle edge cases explicitly (different codecs/variants, streamed vs. in-memory, sizes, alignment) in `ackit`, not in app-side workarounds.
+- Make sure every single audio file is cataloged and the catalog is correct: add a verification command and test that cross-checks the catalog CSVs against the real WADs (missing, extra, duplicate, wrong character, wrong category, wrong language). Fix the data, and report remaining unknowns in `docs/`.
+
+## 2. Subtitles and menu text editing
+- Reverse engineer where and how the game stores subtitles and menu/UI text (they are text files in the game folders). Document the formats and encodings in `docs/`, and implement parsing and writing in `ackit` (with safe backup/revert, as for audio).
+- In the editor:
+  - **Subtitles section**, linked to audio. Each sound shows its subtitle, and each subtitle links back to its sound. Editing both together should be convenient: search by text, character and category, show both side by side in the single-file editor, and bulk edit/find-and-replace.
+  - **Separate "Menu text" section**, kept apart from audio and subtitles, with its own search, filters and per-string editing.
+- Support revert (per entry and all), mod export/import for text, and detect when game files changed underneath.
+
+## Quality rules
+- Do it properly: no hacks. If the editor UI or `ackit` need major restructuring to fit this cleanly, do it, and keep the repository organised as it is now (clear modules, no duplicated code, no leftover scripts).
+- Add the necessary tests: unit, headless app-logic, GUI smoke tests, and a real end-to-end test against the game for audio and for text (apply, check the game starts and shows the change, revert, byte-identical).
+- Document everything and update the existing documents (`README`, `docs/`, `decisions.md`, user guide, `NOTES.md`, `CLAUDE.md`), including the new catalog schema and the text file formats.
+- Update the relevant docs and README in the same commit as each change, not at the end. A stage is not finished until its docs, tests and `NOTES.md` status are updated.
+- Rebuild the packaged distribution (with bundled tools and licenses) and re-verify it.
+
+## Saving quota (I'm on a limited Claude Pro quota; this is a hard requirement)
+- Plan before acting: write a short ordered plan in `NOTES.md`, work in small stages, and commit after each. Finish and verify one stage before starting the next, so a usage pause never leaves broken work.
+- Keep context small: never print whole files, big logs, WAD listings or catalogs. Use grep, head/tail, counts and summaries, and read only the line ranges you need. Write long outputs to files and inspect them selectively.
+- Don't re-read what you already know: rely on `NOTES.md` and `docs/` instead of re-deriving or re-reading large sources. Update them as you go so a context reset is cheap.
+- Use cheaper subagents (Sonnet or Haiku) for mechanical and bulk work: catalog cross-checks, CSV fixes, log triage, repetitive edits, test runs, and doc updates. Keep the main Opus context for reverse engineering, design decisions and tricky debugging.
+- Run the cheapest check that proves the point: run the specific failing test first, not the whole suite after every edit. Run the full suite only at stage boundaries.
+- Run the expensive real-game end-to-end tests (launching the game) sparingly: once per stage after the headless tests pass, plus the full pass at the end. Batch many changes into one game run where possible.
+- Avoid GUI screenshots and image reads unless a layout problem needs them. Prefer headless assertions.
+- Don't rebuild the distribution until the end, except for a single early build to check the pipeline.
+- Avoid repeated failed attempts: after two failures on the same problem, write down the hypotheses tried, then change approach instead of retrying.
+- Order of priority if the quota runs short: (1) correct, complete audio editing and catalog, (2) subtitle editing linked to audio, (3) menu text section, (4) tests and docs for each as you go, (5) packaging. Always leave `NOTES.md` with a clear "Current status / Next steps" so work can resume cleanly after a pause.
+
+Finish with a short summary and a `REPORT.md` of anything unresolved.

docs/history/mission4_dubbing_plan.md

@@ -0,0 +1,284 @@
+# Plan: dubbing voice lines, with the game's voice effects
+
+Status (2026-10-27): stage 0 done (section 9); the scope changed after it (section 10: manual effect controls,
+no automatic profiles); stages 1-2 of section 10 done (effects engine, Dubbing tab). The user's answers are in section 8. Sections 1-7 are the original proposal, written
+from the code, the docs and the shipped catalog CSVs; the steps marked **(game)** read the game files.
+The game is also installed on the Linux machine (Steam Flatpak, mounted into the podman container at
+`/home/gora/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Batman Arkham City GOTY`;
+the container's HOME differs, so it is passed with `--game`), so the read-only stages can run there; in-game
+checks still need the Windows PC.
+Sections 3-7 name modules and files that were proposed but not built after the scope change (`ackit/fxdetect.py`,
+`ackit/dub.py`, `data/catalog/FX.csv`, `fx_profiles.json`, `docs/formats/effects.md`); what was built is in
+sections 10-11 and `NOTES.md`: `ackit/fx.py`, `app/edit.py`, `app/editor.py`, `app/effects.py`, `app/preview.py`.
+
+## 1. The problem
+A dub replaces a character's lines with new recordings. Some original lines are not dry: they sound like a
+telephone or radio (band-limited EQ), a PA announcement over the city (echo, several speakers), a room (reverb),
+a suit or a machine (Mr. Freeze). A dry dub next to them sounds wrong. Two questions:
+
+1. **Where is the effect applied?** This decides everything:
+   - **Baked into the WEM** (applied in the studio, or by Wwise when the bank was built, *rendered* effect):
+     the dry dub loses it, so the dub must get the effect before it is written.
+   - **Applied by the game at run time** (a Wwise effect on the Sound, on a parent container or on a bus, an
+     environmental/reverb zone, attenuation, obstruction): the game also applies it to the replacement. The
+     dub must stay **dry**, or it gets the effect twice.
+
+   The user heard effects "in game and here": the ones heard *here* (the editor plays the WEM) are baked; the ones
+   heard only in game are run-time and need nothing.
+2. **Which effects exist?** Unknown, so they have to be found in the game data (section 3).
+
+## 2. What the catalog already shows (INT, measured on the shipped `data/catalog/INT.csv`)
+Cheap signals that are already in the catalog (`rate`, `channels`) and point to baked effects:
+
+| signal | lines | groups (examples) | likely effect |
+|--------|-------|-------------------|---------------|
+| 3 or more channels | 858 WAD speech rows (mostly 5 ch, then 4 and 3 ch), plus 86 in-memory 5 ch | Predator chatter 251, story 211, Riddler 90: `PRED_RIDDLER`, `PRED_J`, `PRED_STRANGE`, `PRED_MAYOR`, `RIDDLER_R`, `GAME_OVER`, `COURT_B`, `UNDER_S` | PA / hacked-speaker broadcasts: one voice spread over several channels, probably with delays (echo) between them |
+| 32,000 Hz | 110 (102 are `CHAR_FREEZE`) | Mr. Freeze | suit voice filter |
+| 11,025 / 12,000 Hz | 91 | `CHR_RadioScanner_*` | police radio (band-limited) |
+| 6,000 Hz | 25 | `PSO_CarRadio_Chatter_*` | car radio |
+| category *Radio & TV* (`Speech_RADIO`, `Speech_TV`, `Speech_CAMERAS`, `DIA_`) | see `categories.md` | Vicki Vale reports, TV, cameras | radio/TV EQ, maybe noise |
+
+These are hypotheses until someone listens and the metadata scan (3a) confirms them. Two things follow already:
+- A mono dub of a 5-channel line is copied to every channel today (`audio.to_channels`): the spatial echo between
+  the channels is lost. Multichannel lines need a per-channel layout (gain + delay per channel) in their effect.
+- Low-rate lines limit the band themselves (6 kHz = nothing above 3 kHz), so part of the "radio" sound is just
+  the encode rate; the rest (low cut, mid boost, distortion, noise) has to be added.
+
+## 3. Finding every effect the game uses
+Three independent sources, from exact to fuzzy. Every result goes into the catalog, so the editor can say
+for each line: *baked: Radio EQ (from metadata)*, *baked?: echo, 3 channels (analysis, 0.8)*, *game adds: room
+reverb (run time, record dry)*.
+
+### 3a. Wwise metadata (exact, where it exists) **(game)**
+The banks (version 56, Wwise 2011.1) describe effects in HIRC; `ackit/bnk.py` reads only the Sound sources today.
+To add (`ackit/bnk.py` + a survey tool, like the bank survey of mission 3):
+- Parse the **node base parameters** of Sounds, containers, actor-mixers and buses: the effect list (slot,
+  effect object ID, and a **"rendered"** flag in this bank version, per wwiser's v56 layout, to confirm on the
+  game's banks), the parent ID (effects are inherited unless the child overrides them), the output bus.
+- Parse the **effect objects** (HIRC effect / share-set type; the type number for v56 to confirm): plugin ID
+  (Audiokinetic Parametric EQ, Delay, RoomVerb, Matrix Reverb, Peak Limiter, Compressor, Expander, Pitch
+  Shifter, Flanger, Guitar Distortion, Tremolo, ...; plugin ID = company + type + number) and its **parameters**
+  (EQ band frequencies/gains/Q, delay time and feedback, reverb time ...). The `FXPR` chunk the parser already
+  recognizes and `ENVS` in `Init.bnk` (environments, i.e. run-time reverb zones) belong here too.
+- Walk each line's Sound object up the parent chain to the master bus: list of `(effect, plugin, params,
+  where: sound/parent/bus, rendered or run time)`.
+- Line identity helps: Sound objects are the same in every language (`banks.md` *Lines*), so a line's chain is
+  the same in all 5 languages.
+
+Expected result: run-time effects (bus reverbs, environments) are fully known, with parameters. Rendered
+effects are known **with their exact parameters**, which is the best case for a dub. Effects applied in the
+studio before the files reached Wwise leave **no metadata at all**: likely many of the radio/PA lines. Those need 3b.
+
+### 3b. Signal analysis of the WEMs (no metadata needed) **(game, ~minutes over all speech)**
+`ackit/fxdetect.py` (numpy only), per recording, a small feature vector:
+- **Band limits**: frequency below which / above which 1 % of the energy lies (telephone ~300-3,400 Hz), steepness.
+- **Effect EQ curve**: long-term average spectrum (LTAS) of the line divided by the LTAS of the **same speaker's
+  dry lines** (speaker from `characters.csv` / the line's speaker). The voice cancels out and what is left is the
+  effect's frequency response: directly usable as the dub's EQ.
+- **Reverb**: energy decay after speech offsets (Schroeder-style decay fit -> RT60 estimate), direct-to-
+  reverberant ratio, tail length after the last word.
+- **Echo / delay**: peaks in the autocorrelation of the onset envelope, or in the cepstrum, at lags 30-800 ms;
+  for multichannel lines the cross-correlation **between channels** (delay and gain per channel).
+- **Distortion / noise**: crest factor, clipping, spectral flatness of the gaps between words (static, hiss),
+  harmonics added to a narrow band.
+- **Modulation / pitch**: envelope modulation 2-20 Hz (tremolo), pitch and formant shift against the speaker's
+  dry lines (masks, Freeze, Clayface, synthesized voices).
+
+Then **cluster** the vectors (e.g. k-means on normalized features, or simple thresholds first) and **listen to a
+few lines per cluster** (the user names it: "Radio scanner", "Freeze suit", "Strange PA", "Riddler hacked
+speakers", "Interview tape", ...). The names and cluster labels go into a profile table (3d). Cross-language
+check: the 5 recordings of one line must land in the same cluster; disagreements are lines to listen to.
+
+### 3c. Comparing with the game's output (optional, run-time effects only) **(game)**
+The in-game harness can already play sounds and record the loopback (`ackit/verify/ingame.py`, `loopback.py`).
+For a sample of lines with a run-time effect, the same features computed on the game's output vs on the WEM
+show what the game adds. **Never save raw captures** (CLAUDE.md: they can contain voice chat): compute the
+features in memory and keep only the numbers. Useful to confirm, not needed for the dub itself.
+
+### 3d. What gets stored
+- `data/catalog/FX.csv`: per line (`line` identity, language-independent where the metadata says so): `profile`,
+  `baked` (yes / no / unknown), `source` (metadata / analysis / manual), `confidence`, `runtime` (what the game
+  adds, for the "record dry" hint). Built by `tools/build_catalog.py` from the original files only, like the rest.
+- `data/catalog/fx_profiles.json`: the profiles (section 4), with where each parameter came from.
+- `docs/formats/effects.md` (format of the HIRC effect data, survey figures) and claims in `tools/check_docs.py`
+  for every figure; `ackit verify --catalog` re-derives the metadata part.
+- Manual labels of the user live in user data (like notes/tags), never in `source/`.
+
+## 4. Re-creating the effects on a dub
+An **effect profile** = a chain of simple, deterministic processors with parameters (data, not code), rendered by
+`ackit/fx.py`:
+
+| block | parameters | source of the numbers |
+|-------|------------|-----------------------|
+| high-pass / low-pass / peaking / shelf (biquads) | freq, gain, Q | metadata EQ, or fitted to the 3b EQ curve |
+| matched EQ (FIR) | the 3b curve itself | analysis (generic fallback for any baked EQ) |
+| delay / echo | time, feedback, mix, per-channel delays/gains | metadata, or 3b autocorrelation / inter-channel lags |
+| reverb | RT60, pre-delay, damping, mix | metadata (RoomVerb), or 3b decay fit; convolution with a generated IR (filtered, exponentially decaying noise) keeps it deterministic |
+| distortion / bit-crush / noise bed | drive, bits, noise level and colour | 3b, tuned by ear |
+| compressor / limiter | threshold, ratio | metadata, or by ear |
+| channel layout | per-channel gain + delay (replaces the plain mono copy for 3-5 channel lines) | 3b inter-channel analysis |
+
+The rendered dub then goes through the existing chain: loudness matched to the original, encoded at the
+original's rate/channels, fitted to the size limit (`Workspace.prepare`). A reverb or echo tail makes the audio
+longer: the editor's length warning and size fitting already cover that, but a dub UI should show the room.
+
+**DSP dependency** (user: "just use some libraries", no own DSP, no VST hosting):
+- Effects: **pedalboard** (Spotify, GPL-3.0, compatible with this project's GPL-3.0): ready filters (high/low-
+  pass, peak, shelf), reverb, delay, chorus, phaser, distortion, bit-crush, GSM compression (telephone),
+  compressor/limiter, pitch shift. A profile is a list of pedalboard plugins with parameters; the per-channel
+  layout (gain + delay per channel) is a few numpy lines. To check before stage 1: wheels for the project's Python
+  (3.14) on Windows and Linux, and that PyInstaller bundles it.
+- Analysis (3b): numpy (FFT, correlations), scipy only if a step really needs it.
+- Decision goes into `docs/decisions.md` like the other dependencies, licenses into `THIRD_PARTY_LICENSES.txt`.
+
+## 5. Where to build it: new app or the existing editor?
+
+| | Separate dubbing app on `ackit` | Inside Unheard City (new **Dubbing** tab) |
+|-|-------------------------------|------------------------------------------|
+| Write safety | `Workspace.lock` is a `threading.RLock`: it serializes writes **inside one process** only. Two apps open on the same game and store can write one package at the same time (the lost-edit bug of 2026-10-13, across processes). Needs a cross-process file lock + "changed outside" handling in both apps. | Same process, same lock, same store, same changed-outside banner: nothing new. |
+| Reuse | Re-implements catalog loading (background jobs, spinners), language setting, line identity, subtitles, waveform, player/recorder, encode/fit, apply/revert, mod export/import, settings, packaging. | All of that exists (`app/editor.py`, `audioio.py`, `waveform.py`, `textpanel.py`, `modpack.py`). |
+| Users | Two programs, two installs, two settings files, two originals stores to keep apart. | One program; a dub is just another kind of change (and a mod package). |
+| Complexity | The editor stays as it is. | `mainwindow.py` (774 lines) and `editor.py` (590) must not grow into one blob: the tab is its own package. |
+| UX | Free to design a session-based workflow. | Same freedom inside a tab; the Sounds tab stays for one-off replacements. |
+
+**Recommendation: build it into Unheard City, with the engine in `ackit`.**
+- `ackit/fx.py` (profiles + rendering), `ackit/fxdetect.py` (features, clustering), `ackit/dub.py` (dub
+  sessions/takes: files, states; no Qt), CLI `ackit fx survey | scan | render` and `ackit dub ...`: all headless,
+  testable, usable from scripts.
+- `app/dubbing/` (own package): the **Dubbing** tab.
+- The **Sounds** editor gets one small row: *Effect: [profile ▾] (detected: Radio EQ, metadata)* and a *game
+  adds ... record dry* hint, so single replacements benefit too.
+- A separate program makes sense only later, if voice actors record **without the game** (remote dubbing): a
+  small recorder that reads a script export and writes a take pack (WAVs + JSON), which Unheard City imports.
+  It never touches game files, so the lock problem does not arise.
+
+## 6. The Dubbing tab (workflow)
+1. **Session**: language + character (or a filter: category, a list of lines). Lists the character's lines via the
+   line identity; shows the script (subtitle of the loaded language, the INT text as reference), the original's
+   length, the effect profile and baked/run-time status, and a state per line: *to do / recorded / approved /
+   applied*. Sessions and takes are stored in user data (`%APPDATA%\UnheardCity\dubs\...`), never in the game.
+2. **Record**: play the original as a cue, count-in, record, auto-trim silence, several takes per line, pick one;
+   length vs the original (and optional time-stretch to fit, if pedalboard or a WSOLA stretch is added).
+3. **Process**: clean-up (high-pass, gate), loudness match to the original (exists), the effect profile
+   (suggested from FX.csv, editable, A/B against the original with and without the effect).
+4. **Apply** the approved lines in one `Workspace.apply` call (it takes a list); **export** the session as a mod
+   package (`modpack.export_mod`, with edited subtitles if the dub's wording changed, within their room).
+5. Non-destructive: raw takes + parameters are kept, so a changed profile re-renders every line.
+
+Warnings the tab must show (from the existing data):
+- Lines the game shares between languages (167 in-memory voice lines stored once for every language; 616-651 WAD
+  lines per language identical to INT, `banks.md` *Lines*): dubbing one changes it for every language that uses it.
+- Size limit (speech in place, never larger than the original WEM): long takes plus reverb tails get squeezed or
+  refused; show the room.
+- Subtitle room (NUL-padded in place).
+- Lines without a speaker label or with a disputed one (19 listed in `categories.md`).
+
+## 7. Stages (one commit each: code + tests + docs + NOTES status)
+0. **Evidence (game)**: HIRC effect survey (3a) and a first feature scan (3b) over all speech; listen to samples
+   per cluster with the user; write `docs/formats/effects.md` with figures (+ `check_docs` claims). Outcome: how
+   many lines are baked vs run time, and the list of profiles. **This decides the size of everything after it.**
+1. `ackit/fx.py` on pedalboard: profiles JSON, render; tests with synthetic signals (an EQ's
+   measured response, a delay's lag, an RT60 measured back from the rendered reverb).
+2. `ackit/fxdetect.py` + `FX.csv` / `fx_profiles.json` in `build_catalog.py`; `verify --catalog` checks; CLI.
+   Test: render a profile onto a dry synthetic voice, detect it back.
+3. Sounds editor: effect row + run-time hint; GUI tests.
+4. `ackit/dub.py` + Dubbing tab (sessions, takes, script, batch apply, mod export); GUI tests.
+5. Real-game round trips (`tools/e2e_features.py` / `e2e_editor.py` cases for a dubbed line with an effect,
+   reverted in `finally`), the user listens in game, packaging (`build_app.py`, licenses), version 3.4.0 or 4.0.0.
+
+## 8. Decisions (user, 2026-10-27)
+- **All characters** can be dubbed, from the existing catalogs (no per-character priority): every effect group
+  found in stage 0 needs a profile.
+- **Approximate effects** are enough; profiles are tuned later by ear (the Riddler lines' effect is complex).
+- **Libraries, not own DSP**: pedalboard for effects (section 4); no VST hosting, no time stretching for now.
+- **Everyone has the game**: no separate recorder or take packs; everything lives in Unheard City (section 5).
+- Still open, out of scope unless asked: audio inside video files (pre-rendered movies carry their own sound
+  track, not in the WADs or banks; to check whether the game has any with speech).
+
+## 9. Stage 0 results (2026-10-27, INT, read-only)
+Measured with scratch scripts (HIRC survey of every bank, signal features of 12,430 INT speech recordings),
+then the user listened to 77 candidate groups (`external/fx_samples/INDEX.md`, git-ignored). Figures here are a
+record of that day, not re-checked by `tools/check_docs.py`.
+
+**Metadata (Wwise, v56).** Layouts from wwiser: node base params = u8 override-parent-fx, u8 count, [u8 bypass,
+count x {u8 slot, u32 fx id, u8 share-set, u8 rendered}], u32 output bus, u32 parent; a Sound has them after its
+source (+ u8 source bits); a bus (v56) has u8 is-environmental at payload offset 33, ducks of 17 bytes, then its
+effects. Effect objects are HIRC types 0x12 (share-set) / 0x13 (custom): u32 plugin, u32 size, parameters.
+- Run time (the game adds them; the files are clean, so a dub is recorded **dry**): McDSP FutzBox share-set
+  `4272216A` (in 1,125 speech banks, on 464 actor-mixer slots, RTPC-driven) -> 2,858 speech rows; Parametric EQ
+  on a bus (30 rows) or on the sound (3); the ML1 limiter on the top bus (everything); 84 RoomVerb + 6 Matrix
+  Reverb share-sets in `Init` = the environments the game switches per area.
+- Rendered (baked by Wwise): 33 speech rows (car radio 25, radio scanner 3, museum thug-to-Penguin 4, Steel HQ
+  1); the slot's fx id is zeroed, so the effect type is lost.
+- Everything else that is baked was made in the studio and has no metadata.
+
+**Channels.** Speech WEMs with 2-5 channels carry the WAVE channel mask: 2 ch `0x3` (L R), 3 ch `0x7` (L R C),
+4 ch `0x33` (L R SL SR), 5 ch `0x37` (L R C SL SR); ackit decodes and encodes them in that order. PA lines put
+the dry voice in C and an echo 160-173 ms later in the surrounds (Harley/HQ 121 ms; Freeze's quad rears 73 ms);
+Riddler, Ivy and Ra's 5-channel lines are processed differently per channel.
+
+**What the user heard** (per group; "no effects" groups omitted: thug/Ninja/Catwoman/JR efforts, combat barks,
+Riddler `RIDDLER_R`, DLC `PDLC_PDLC`/`PDLC_EMOTES`/Robin/Harley, Outcast, Penguin `MUSEUM_B`, Steel/Church Harley,
+Bully hobo, Talia `UNDER_B`, `CH_ZERO` Strange, Strange/Black Mask game-over):
+
+| effect heard | groups |
+|--------------|--------|
+| radio EQ | radio scanner, Alfred/Oracle/Penguin/Strange/Ra's comms, copter, cameras, Gordon radio, car radio |
+| phone EQ | Penguin PRED, Zsasz calls, Two-Face court, Joker (PRED, theatre), Strange/Mayor/Harley PA, Riddler (Enigma, cameos), Vicki Vale radio scan, Penguin exhibits |
+| phone-mic EQ / TV-speaker EQ | interview tapes (Vicki Vale, Joker), Strange on TV, Joker church message, `UNDER_C` TV |
+| radio noise | copter, car radio, radio scanner (Vicki), Vicki Vale radio scan, map radio |
+| distortion (slight, speaker) | Penguin exhibits, radio scanner (Vicki), `UNDER_C` TV |
+| echo (PA delay) | Strange/Mayor/Harley/Joker/Two-Face PA and court lines, Catwoman side mission (Strange) |
+| very short delay (width) | Riddler cameos, radio scanner (Riddler) |
+| reverb (small/short room, slight wide room) | Riddler PRED, Harley 5 ch, Strange TV, Joker church, Talia sequence, Harley DLC, game-over lines (barely) |
+| reverse reverb (occasional) | Riddler (PRED, cameos, radio scanner), Poison Ivy |
+| glitches (parts repeat at another pitch), small repeats | Riddler (PRED, cameos, Enigma, radio scanner) |
+| pitch shift, fixed, down | Clayface |
+| pitch shift over time + robot + pan over time | Mr. Freeze |
+| octaves overlaid (down), formant shift, stereo width | Ra's al Ghul (giant, trials), Poison Ivy |
+| formant shift, unintelligible voice | car radio |
+
+## 10. Scope after stage 0 (user, 2026-10-27)
+Matching effects automatically per line is too complex: the Dubbing section gets **manual controls for every
+effect found** (section 9), which the user sets by ear. Dropped: automatic profiles, `FX.csv` /
+`fx_profiles.json`, detection in the catalog build (stage 2 of section 7). Kept: pedalboard, the Dubbing tab in
+Unheard City, everything else of sections 5-6.
+
+Effects rack (`ackit/fx.py`, one table of blocks and parameters that the UI is generated from; every block off
+by default; fixed order; seeded randomness, so a setting always renders the same audio):
+
+| # | block | controls | built on |
+|---|-------|----------|----------|
+| 1 | Pitch | shift (semitones), formant shift, drift depth + rate (pitch over time) | `pedalboard.time_stretch` (pitch curve, formants) |
+| 2 | Octave layers | two layers: interval + level | `time_stretch` pitch, mixed |
+| 3 | Robot | ring modulator frequency + mix, metallic comb (delay, feedback) | numpy |
+| 4 | Glitch | events per 10 s, length, repeats, pitch of the repeats, seed | numpy + `time_stretch` |
+| 5 | Voice band (EQ) | low cut, high cut, slope, mid boost (freq, gain, Q) | `HighpassFilter`, `LowpassFilter`, `PeakFilter` |
+| 6 | Distortion | drive, clip, bit depth, phone codec (GSM) | `Distortion`, `Clipping`, `Bitcrush`, `GSMFullRateCompressor` |
+| 7 | Noise | level, colour (band), crackle | numpy |
+| 8 | Echo | time, feedback, mix | `Delay` |
+| 9 | Reverb | room size, damping, wet, width; reverse reverb amount + how often | `Reverb` (reversed for the reverse swell) |
+| 10 | Channels (2-5 ch lines) | width (L/R delay), auto-pan rate + depth, surround echo delay + level, dry voice in C | numpy, channel order of section 9 |
+
+Then the existing chain: loudness match / normalize / gain, encode within the size limit.
+
+Stages from here (one commit each; 1 and 2 done on 2026-10-27, see NOTES.md):
+1. `ackit/fx.py` + tests (synthetic signals: each block measured back) + pedalboard dependency
+   (`pyproject.toml`, `docs/decisions.md`, licenses).
+2. Dubbing tab (`app/dubbing/`): line list (character/category/search filters, state per line), the existing
+   sound editor embedded (record/load, trim, A/B, subtitle, apply/revert), the effects rack between the take and
+   the loudness step, settings saved per line in user data; GUI tests; user guide.
+3. Real-game round trip of a dubbed line with effects, packaging (PyInstaller + pedalboard), version bump.
+
+## 11. UI revision (user, 2026-10-27, after trying stages 1-2)
+- The subtitle box was too small for dubbing, and the Sounds tab and the Dubbing tab both had an editor:
+  browsing and editing are now separate. **Sounds** only browses (list + a compact preview with **Edit** / **Add
+  to edit list**); **Dubbing** became **Edit**, for every kind of sound, with an edit list instead of a second
+  browser, the subtitle as a large **script**, and Previous / Next.
+- The effects panel was "raw": sliders with readable values (log scale for frequencies and times), **looks**
+  for the voices heard in the game (section 9) and **preset buttons** per effect that set the sliders in place.
+- The reverb (pedalboard's Freeverb) sounded poor: replaced by a generated room response (decay, pre-delay,
+  reflections, damping, width) with presets (small room to church, tiled cell, outdoors).
+- Second round: the browser's player keeps Play/Stop together (editing actions apart on the right); the Edit tab
+  fits the window without scrolling (effects as a chain with one settings area); the app and docs use generic
+  terms (replacement, effects, subtitle), since the Edit tab is for every kind of sound, not only dubbing.

docs/reverse-engineering/findings.md

@@ -70,8 +70,70 @@ Conclusion: approach A works end-to-end in-game. Use >= 30 s probes for the intr
     expects WAVEFORMATEX order (FL FR FC LFE BL BR) is unverified (identical-channel probes were used).
 16. (2026-10-09) The WADs hold only streamed audio. About 15,800 more sounds (footsteps, foley, combat, weapons,
     UI, grunts) are in-memory WEMs inside soundbanks: 65 loose `SFX\*.bnk` (1,117) and 3,340 banks embedded in
-    the `.upk` packages (15,641; many banks copied into several packages). None are in the catalog; the WAD
+    the `.upk` packages (14,802; many banks copied into several packages). None were in the catalog then (they
-    side is complete. Inventory, layout and editing consequences: `docs/formats/banks.md`.
+    are now: archive BANK); the WAD side was complete. Inventory and layout: `docs/formats/banks.md`.
+17. (2026-10-09) Speech lines have **per-language media IDs**: every package carries one BKPK table per language
+    for the same WAD entry. 396 lines (78-80 per language) have a prefetch head (first N bytes) in banks; each
+    head equals the start of exactly one language's WEM.
+18. (2026-10-09) In-game test of in-memory sounds (`ackit verify --ingame --banks`): 77 probes written into the
+    78 sounds stored in the Frontend*/Startup packages (54 files: patched LZO packages + loose banks). The game
+    starts and runs, all 68 probe byte patterns are found in its memory (it loaded the patched packages), revert
+    byte-identical: PASS. Not shown: a probe being *heard*. Only 7 Vorbis streams start on the title screen
+    (setups 4f708975, 6ded1c65, 92f94a3f, d93b5944, d4b2e824: shared by hundreds of sounds, music included) and
+    none is a probe. Tried: Enter/arrow keys posted to the game window (WM_KEYDOWN/WM_KEYUP; that helper was removed later) to reach
+    the main menu: no UI sound followed (the game probably reads input another way; global SendInput is not
+    used on purpose); a breakpoint on the setup parser entry: no extra events. Hypotheses: no in-memory sound
+    plays without input; the decode path is the same as for WAD sounds (same WEM rules, 300/300 in-memory WEMs
+    rebuild byte-identically), so playback is expected to work like E4/E10.
+19. (2026-10-09) In-game text test (`ackit verify --ingame --texts`, ITA): 12 menu texts of FRONTEND/GFxUI
+    prefixed with markers and 12 subtitles patched in place (21 files). All 12 markers are found as UTF-16 in the
+    game's memory after ~45 s (the localization files are read at startup), no crash, revert byte-identical:
+    PASS. Subtitles are not loaded on the title screen (no dialogue object in Frontend*/Startup/BmGame).
+20. (2026-10-10) **Variation positions differ between languages.** A WAD entry stores its WEMs in media-ID order
+    and each language has its own media IDs, so for 7,010 of 9,366 lines (75 %) the same game line is at a
+    different position in the ITA and INT entries (e.g. Speech_CH_ZERO_CH0_fight_intro_01 = ITA :0, INT :2).
+    Position names (NAME_i, wemscan) therefore mean different lines per language. Speech sounds are now named
+    after their speech event (same in every language); the curated ITA lists are mapped through it. Independent
+    check: the speaker field of the dialogue objects agrees with the mapped labels for 955 / 955 lines.
+    (Superseded by 22: the identity of a line now comes from the Sound objects, for every recording; measured
+    that way, 81.2-81.6 % of the WEMs in multi-variation entries are at another position than in INT.)
+21. (2026-10-10) In-memory speech banks are localized (language list + one bank per language in one AkBank
+    export, checked on 215 banks against the WAD languages of their streamed lines); 41 more subtitles are
+    RSubtitle objects linked through RSubtitleLookup (key = FNV-1 of the line name without `Speech_`). Subtitle
+    coverage by category is uneven in the game itself: efforts 0 %, combat barks 11 %, cinematics 10 % (cutscene
+    lines are mostly not reached by speech events and have no dialogue object), story/predator/open world ~90 %+.
+    (With every version of a line linked, item 22: combat barks 9 %, cinematics 23 %; REPORT.md.)
+
+22. (2026-10-10) **One recording per language, from the game data alone** (`docs/formats/banks.md`, *Lines*).
+    The BKPK tables of a speech entry sit inside the localized AkBank export, one per declared language; they
+    used to be merged by media ID, but 657 media IDs are used by several languages at different positions, so
+    the last language read (ESN) overwrote the others (120-157 WEMs per language looked unreferenced). Kept per
+    language: 10,816 / 10,816 WEMs matched in every language. Prefetch sources were left out of the Sound-object
+    map, which hid the 396 prefetched lines (game-over taunts). Five recordings per language are stored twice
+    (same media ID in two entries, identical bytes). Subtitle objects name events no package lists (FNV-1 of the
+    name = event ID: Vicki Vale's radio reports, Hush tape reveal). Result: 12,434 lines with exactly one
+    recording in each of the 5 languages, no conflict; independent checks: 54,055 / 54,055 WAD recordings are
+    sources in a bank of their own language, 2,101 / 2,101 prefetch heads equal the start of their recording,
+    every linked subtitle is on the same line in all 11 languages.
+23. (2026-10-10, final check) **Whole WAD recordings in banks.** 49 in-memory media (9-11 per language) have the
+    media ID of a WAD recording of their bank's language and are byte-identical to it (game-over taunts,
+    `Speech_CH_ZERO_CH0_ENCRYPTION_CARD3_01`, `Speech_SEQUENCE_CH2_RESCUE_DOC_01`, ...): the line's Sound
+    object names that media, and some bank also stores the whole WEM in memory. They were listed as separate BANK sounds (with the
+    line's name: the CLI called the name ambiguous), so a replacement of the line left the in-memory copies
+    old. Now they are bank copies of the WAD recording like prefetch heads (84 copies; 445 recordings with
+    heads or whole copies, 2,185 copies), written and reverted with the line. Also: 8 in-memory sounds per
+    catalog had the same `<Bank>_<n>` name as an SFX WAD sound of the same bank (`CHR_RasBossFight_1`,
+    `SEQ_DemonTrials_1`); they now take `<Bank>_<MEDIA>`, and a line's own name wins over another recording's
+    position alias (`CHR_RadioScanner#1` in FRA). Every name of every catalog resolves to its own sound.
+24. (2026-10-10, gameplay test by the user) **Padded in-memory replacements play as silence.** All 568 Batman
+    footsteps were replaced; in game the original steps were heard and the replaced cape-open sound was silent.
+    Memory scan while walking: no original footstep WEM loaded, 137 replacements loaded; debugger: the
+    replacements' Vorbis setups are created and parsed OK (ret 0). Every untouched in-memory WEM fills its slot
+    exactly (RIFF size = DIDX size, 500 / 500 sampled), while the replacements were zero-padded. Filled with a
+    `LIST{adtl, labl}` chunk before `data` (55 in-memory originals carry cue+LIST chunks), every step plays the
+    replacement. ackit now fills every in-memory copy and every whole bank copy of a WAD recording
+    (`wem.fill`); WAD entries and prefetch heads keep zero padding (streamed playback is unaffected). The
+    remaining sound after each step is Batman's gear and cape foley (`CHR_BatmanFoleyBelt`, `..Cape`).
 
 ## Old path -> new path
 `scripts/harness.py` -> `ackit verify --ingame` (ackit/verify/ingame.py); `scripts/validate_builder.py` ->

docs/user-guide.md

@@ -1,7 +1,7 @@
 # Unheard City - user guide
 
-Replace voice lines, music and ambience in **Batman: Arkham City (PC, Steam)**. Everything you change can be put
+Replace voice lines, music, ambience and sound effects, and edit subtitles and menu text in **Batman: Arkham
-back with one click.
+City (PC, Steam)**. Everything you change can be put back with one click.
 
 ## Install and start
 1. **Windows:** unzip the download and double-click **Install.cmd**. It installs Unheard City for your user (no
@@ -16,68 +16,180 @@ back with one click.
 
 ![Main window](img/main.png)
 
+## Language
+There is **one language setting**: the **Language** list at the top of the window. It starts at the language the
+game is set to (Steam > game properties > Language) and applies to everything you edit:
+- **Sounds**: the voice lines of that language, plus the sounds every language shares (music, ambience,
+  effects). The game has voices in English (INT), French, Italian, German and Spanish; for the other languages
+  (Latin American Spanish, Polish, Portuguese, Russian, Japanese, Korean) it plays the English voices, so the
+  editor shows those and says *voices: INT* next to the list.
+- **Edit**: the sounds of the edit list in that language (sounds of another language stay on the list, hidden).
+- **Subtitles** and **Menu text**: the texts of that language.
+Changing it reloads the lists; the next start is again in the game's language.
+Some voice lines sound English in every language: the game itself ships the English recording for most
+efforts and grunts, many cinematics and a few other lines, and some in-memory lines exist only once for all
+languages. The editor shows what each language's game files contain.
+
+## Names
+Every voice line has one name in every language, taken from the game itself (e.g. `Speech_TAPE_JOKER_1_04`, the
+same as in the Subtitles tab): the line `Speech_X` in English is the same line `Speech_X` in French, Italian,
+German and Spanish. Inside a WAD entry the game stores the variations of a line in a different order in each
+language, so the position-based name is kept only as a second name (`Speech_TAPE_JOKER_1#4` = the 4th WEM of
+that entry in this language; search finds both). When the game plays one of two versions of a line, the
+second is shown as `NAME#2`. About 6 % of the lines are named by no game event; they take the name of their
+English recording in every language. A few recordings are stored twice in the game files: both rows belong to
+the same line and are changed and reverted together.
+
 ## Find a sound
 - Type in the **search box** (Ctrl+F): words can be parts of the name, a category, a character, your own
   notes or tags, or an ID like `b49cdf48`. All words must match.
 - Narrow down with **Group** (Speech, Music, Ambience, SFX), **Category** (e.g. *Interview tapes*,
-  *Combat barks*, *Cinematics*), **Character**, **Archive** (speech language or SFX), **Tag**, **Modified
+  *Combat barks*, *Cinematics*), **Character**, **Archive** (speech language, SFX or BANK), **Tag**, **Modified
   only** and **Favorites only**. Combine them freely; results update instantly.
 - **Character** is the verified speaker from the curated catalog. Grey `~` names are only *mentioned* in the
   sound's name (useful hints for the many unlabelled lines); tick *Include characters mentioned in names*
   to filter on them too.
 - Click a column header to sort (length, size, status, ...). Ctrl/Shift-click selects several rows.
-- Most short effects (footsteps, foley, weapons, menu sounds) are stored inside soundbanks and are not listed
+- Archive **BANK** holds the short sounds stored inside soundbanks (footsteps, foley, combat,
-  yet; the list covers voice lines, music, ambience and the effects of the SFX archive.
+  weapons, menu sounds, grunts). Many are named after the game event that plays them (`Play_...`), the
-- **Language** (top left) switches the speech language. Speech is read from the language the game is set to.
+  rest after their bank (`CHR_BatclawGrapple_3`). The Edit tab shows the bank and how many copies the game keeps
+  of the sound; a replacement is written into every copy.
 
 ![Browsing by category](img/browse.png)
 
 ## Listen
-Select a row and press **Space** (or double-click) to play the original. In the editor below:
+The **Sounds** tab is for finding and listening; it does not change anything. Below the list, the selected sound
-- **Original** / **Installed** play the game's original and what is currently in the game. Click on a
+is shown with its details and subtitle (left), a player (middle) and the way into the Edit tab (right).
-  waveform to play from that point. **Loop**, **Volume** and **Stop** (Esc) work for everything.
+**Play original** (or Space, or double-click the row), **Play installed** (what the game has now, if you
-- **A/B** switches between the original and your replacement at the same position, to compare.
+replaced it) and **Stop** (Esc) sit together above the waveforms; click on a waveform to play from that point.
 
-## Replace
+To change sounds, select them (Ctrl/Shift-click for several) and press **Edit** (or Enter, or right-click >
-![Editor](img/editor.png)
+**Edit**): they go onto the **edit list** and the **Edit** tab opens on the first. **Add to edit list** puts them
-1. **Load file…** (Ctrl+O): WAV, FLAC, OGG, OPUS, MP3 or AIFF. (M4A/WMA work only if ffmpeg is installed.)
+there without leaving the list, e.g. to collect all the lines of a character or all the sounds of a weapon
-   Or choose a microphone in the list and press **Record**, then **Stop recording**. Listen with
+first.
-   **Replacement**; record again as often as you like.
-2. Adjust: **Keep from / to** trims (or right-drag on the replacement waveform), **Trim silence**, **Gain**,
-   **Normalize**, **Match original loudness** (recommended for voice lines).
-3. **Apply to game** (Ctrl+S). The audio is converted to the original's format and written into the game files.
-   The row turns bold and shows *Modified*.
 
-Size rule: voice lines and effects must fit into the original's size (the *Max size* column), because the
+## Edit
-game reads exactly that many bytes. Longer audio is encoded at lower quality to fit; if it still does not fit
+![Edit tab](img/editor.png)
-you are asked to shorten it. Music (`Music_…`) can be any length. A warning appears when your audio is longer
+The **Edit** tab is where sounds are replaced: voice lines, music, ambience, sound effects. Everything fits in
-than the original, since the game may cut it off.
+the window: left the **edit list**, middle the sound you are working on, right its effects.
+- **Edit list**: filter it, **Remove** sounds you are done with, or **Remove applied**. Columns: a dot = a
+  replacement is saved, FX = number of effects on, a tick = applied to the game. **Previous** / **Next**
+  (Alt+Left / Alt+Right) walk the list.
+1. **Subtitle** (voice lines only): shown large, with the game's original (and the English line when you edit
+   another language) under it. You can rewrite it there and **Apply** it (see *Subtitles*).
+2. **Listen**: **Original**, **Installed** and **Replacement** each have a play button and a waveform; **A/B**
+   switches between the original and the replacement at the same position; **Loop**, **Volume**, **Stop**
+   (Esc).
+3. **Replacement**: **Record** (choose the input in the list next to it; press again to stop) or **Load file…**
+   (Ctrl+O: WAV, FLAC, OGG, OPUS, MP3 or AIFF; M4A/WMA only if ffmpeg is installed), as often as you like.
+   **Keep … to** trims (or right-drag on the replacement's waveform), **Trim silence**, **Gain**,
+   **Normalize**, **Match loudness** (to the original; on by default for voice lines). The replacement is saved
+   at once in your user data folder (`edits`), with its trim, level and effect settings, so you can come back
+   to it any time; nothing is written to the game until you apply it.
+4. **Apply to game** (Ctrl+S). The audio (with its effects) is converted to the original's format and written
+   into the game files; the list shows a tick and the Sounds tab *Modified*.
+
+Size rule: voice lines and sound effects must fit into the original's size (the *Max size* column), because
+the game reads exactly that many bytes. Longer audio is encoded at lower quality to fit; if it still does not
+fit you are asked to shorten it. Music (`Music_…`) can be any length. A warning appears when your audio is
+longer than the original, since the game may cut it off.
+
+### Effects
+Many sounds are not dry in the game (radios, phones, announcements, speakers, Mr. Freeze's suit...); the
+replacement gets the effects you turn on, and **Replacement** plays it with them.
+- **Start from a preset…** sets several effects at once: Radio, Phone, Tape / phone mic, TV in a room, PA
+  announcement, Mr. Freeze, Clayface, Riddler broadcast, Giant (Ra's al Ghul), Car radio.
+- The **effect chain** lists every effect in the order it runs: tick an effect to turn it on, click its name to
+  see its settings below. Each effect has preset buttons that set its sliders; then adjust by ear. Moving a
+  slider turns its effect on; double-click a slider to put it back.
+
+| effect | in the game, for example |
+|--------|--------------------------|
+| **Pitch** (shift, formant, drift = pitch over time) | Clayface (shift down), Mr. Freeze (drift) |
+| **Octave layers** | Ra's al Ghul, Poison Ivy (octaves below the voice) |
+| **Robot** (ring modulator, metallic comb) | Mr. Freeze's suit |
+| **Glitch** (pieces repeated, at another pitch) | the Riddler's hacked broadcasts |
+| **Band-pass EQ** (low/high cut, mid boost) | radio, phone, phone microphone, TV and small speakers |
+| **Distortion** (drive, clip, bits, phone codec) | speakers, radios, the car radio |
+| **Noise** (level, colour, crackle) | radio static |
+| **Echo** | Hugo Strange, the Mayor, Joker and Harley over the city speakers (the *PA announcement* preset: 160 ms) |
+| **Reverb** (decay, mix, pre-delay, reflections, damping; reverse swells before phrases) | rooms, TV, halls; reverse: the Riddler, Poison Ivy |
+| **Channels** (sounds with 2-5 channels) | stereo width, pan over time, the PA layout: dry sound in the centre, echo in the surrounds |
+
+**Copy** / **Paste** move settings between sounds; **Use these effects for the selected** gives every selected
+sound of the edit list the same settings (e.g. all of Mr. Freeze's lines), keeping their replacements.
+**Reset** turns everything off.
+
+Some effects are added by the game itself while it plays, not stored in the sound (the radio sound of many
+thugs and street conversations, the reverb of rooms and areas): if the original sounds clean here but not in
+the game, keep your replacement clean too, or the effect is applied twice. Glitch, echo and reverb make a sound
+longer, which counts for the size rule.
 
 ## Undo
-- **Revert to original** in the editor puts that sound back exactly.
+- **Revert** in the Edit tab puts that sound back exactly (the replacement stays saved for later).
-- Right-click rows → **Revert selected**; toolbar **Revert all** puts every original back.
+- Right-click rows → **Revert selected**; toolbar **Revert all** puts every original back: all sounds,
+  subtitles and menu texts you changed, in every language.
 - Originals are kept in your user data folder (`%APPDATA%\UnheardCity` on Windows,
   `~/.local/share/unheardcity` on Linux), so reverting works after updating or reinstalling the app too.
-  Uninstalling keeps this folder; use **Revert all** before uninstalling if you want the game back to original. Before the first
+  Uninstalling keeps this folder; use **Revert all** before uninstalling if you want the game back to original.
-  change to an audio archive, one full copy of it is also kept there as a fallback (Settings).
+  Before the first change to a game file, a full copy of it is kept there too: optional for the WAD archives
+  (Settings), always for soundbanks, packages and text files (they are rebuilt from that copy). Only files the
+  game ships unchanged are copied; a file another tool (or another originals store) has changed is reported as
+  *changed outside this tool* first.
+- The originals of your changes are only in the store that made them, so the store folder (Settings) cannot be
+  changed while changes are applied: use **Revert all** first.
 - If Steam "Verify integrity of game files" or an update undoes your changes, a bar offers **Re-apply my
   changes** or **Forget them**. Do not use "verify" to undo mods; use Revert all.
+- If the store folder itself is damaged, the full copies can still be put back with the command-line toolkit
+  that Unheard City is built on (`ackit restore-backup`, see the project README; not part of the editor
+  download).
+
+## Subtitles
+The **Subtitles** tab lists the subtitle of every speech line in the selected language (see *Language*).
+- Search by text or line name, filter by **Character** and **Category** (taken from the linked speech sound),
+  or show **Modified only**. Select a row to edit its text below and press **Apply** (or Enter).
+- Each subtitle replaces the original *in place*, so it can be at most as long as the original (the counter
+  shows *used / room* characters). Most subtitles are stored in the Western European character set
+  (Windows-1252: accents are fine, other scripts are not); those stored as Unicode (almost all Russian,
+  Japanese and Korean ones, most Polish ones) accept any character. The editor says so when a character does
+  not fit.
+- **Go to the sound** selects the speech line in the Sounds tab (its subtitle shows in the preview). In the
+  Edit tab, a voice line's **Subtitle** is shown large above the audio: edit it there, **Apply** / **Revert**
+  it, or **Show in list** to jump back to the Subtitles tab.
+- **Find and replace…** works on the rows the filters show (texts that would not fit are listed and skipped).
+- **Revert** / **Revert all texts here** put the original text back (every copy in every game file).
+- About 30 subtitles per language have no sound in the game (texts such as the Arkham City Stories, unused
+  radio reports); they have no linked sound. A subtitle of a line the game plays in two versions belongs to
+  both. The game itself has no subtitles for efforts and grunts and for most combat barks and cutscene lines.
+
+## Menu text
+The **Menu text** tab lists every menu, HUD, bio, riddle and credits string of the selected language (about
+7,000), grouped by **File** (e.g. `FRONTEND`, `GFxUI`, `Bios`). Search, edit, apply and revert work like
+subtitles; menu texts can be of any length but cannot contain line breaks (keep the game's own markers such
+as `\n` as they are).
 
 ## Notes, tags, favorites
 Right-click → **Notes & tags…** (F2): free text and comma-separated tags, searchable and filterable.
 Click the **★** column to mark favorites. These are stored in your user data folder, not in the game.
 
 ## Share your changes (mods)
-- **Export mod…**: choose which changed sounds to include and save a `.acmod` file (the exact game data plus
+- **Export mod…**: choose which changed sounds and texts to include and save a `.acmod` file (the exact game
-  the audio, so it also works for another language whose original has a different size).
+  data plus the audio, so it also works for another language whose original has a different size).
 - **Import mod…**: shows which sounds match this game, then installs them. You can revert them like any change.
+  If a sound or text the mod replaces was changed outside the editor, you are asked before it is overwritten.
+- Changed subtitles and menu texts are exported and imported with the sounds (listed as *Text*); a text stays
+  in its language (a French subtitle edit changes the French subtitle).
+- In another language, voice lines are matched by the game's own identity of the line, so the same line is
+  replaced even where its position differs. Voice lines from packages made with versions before 3.1 carry no
+  such identity: the import lists them instead of guessing (export the mod again with this version).
 
 ## Settings
 ![Settings](img/settings.png)
-Game folder, where originals are kept, the full-backup option and the playback device. The recording device
+Game folder, where originals are kept (changeable only while nothing is modified), the full-backup option and
+the playback device. The recording device
 is chosen next to the Record button. **Background**: show or hide the Arkham City wallpaper behind the filter
 panel; **Background strength**: how visible it is (10 % by default).
-The editor panel opens tall enough to show all its controls; drag the divider above it to change that.
+Drag the dividers to resize: the preview under the sound list, and the edit list, editor and effects columns of the Edit tab.
 
 ## Troubleshooting
 | message | what to do |
@@ -85,6 +197,7 @@ The editor panel opens tall enough to show all its controls; drag the divider ab
 | "Cannot write … close the game" | quit Arkham City (and anything else using the file) and try again |
 | "not a Batman: Arkham City folder" | pick the folder that contains `BmGame` and `Binaries` |
 | "audio is too long for this sound" | shorten the audio (see the original's length) |
+| "the text is … characters long but this subtitle has room for …" | shorten the subtitle (the counter shows the room) |
 | "Cannot read … Supported: WAV, FLAC, …" | convert the file to WAV/MP3/FLAC, or install ffmpeg |
 | "changed outside this tool" | the game files were changed by an update or another tool; revert or confirm to overwrite |
 | no sound when playing | choose another playback device in Settings |

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ackit"
-version = "2.0.0"
+version = "3.4.0"
 description = "Batman: Arkham City (PC) audio toolkit and editor"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -12,6 +12,7 @@ dependencies = [
   "numpy>=2.0",
   "soundfile==0.14.0",
   "soxr==1.1.0",
+  "pedalboard==0.9.26",
 ]
 
 [project.optional-dependencies]

tests/conftest.py

@@ -11,8 +11,14 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 
 @pytest.fixture(autouse=True)
 def isolated_home(tmp_path, monkeypatch):
-    """User data/config/cache go to a temp folder (never touch the real settings or stores)."""
+    """User data/config/cache go to a temp folder (never touch the real settings or stores), and the language
+    the game is set to on this machine (Documents\\WB Games\\...\\UserEngine.ini) does not leak into tests."""
+    from ackit import game as G
     monkeypatch.setenv("ACKIT_HOME", str(tmp_path / "home"))
+    real = G.Game.config_dirs
+    mine = os.path.normcase(os.path.join(os.path.expanduser("~"), "Documents", "WB Games"))
+    monkeypatch.setattr(G.Game, "config_dirs",
+                        lambda self: [d for d in real(self) if not os.path.normcase(d).startswith(mine)])
     yield
 
 

tests/synth.py

@@ -25,6 +25,16 @@ def wem(seconds, freq, rate=24000, ch=1):
     return encoder.encode(x, sr, rate, ch)[0]
 
 
+def fitting_wem(ws, s, seconds, freq):
+    """A tone WEM that `ws` accepts for `s` (Workspace.fits: in-memory slots are filled exactly, which needs
+    an even number of spare bytes): the tone is shortened a little until the size allows it."""
+    for k in range(40):
+        w = wem(seconds * (1 - 0.01 * k), freq)
+        if ws.fits(s, w):
+            return w
+    raise AssertionError("no fitting WEM for %s" % s.key)
+
+
 def _a(x, n):
     return (x + n - 1) // n * n
 
@@ -89,6 +99,153 @@ def pack(media):
     return bytes(hdr) + bank, bytes(payload)
 
 
+def bank(bank_id, media, name=None, extra_sources=(), prefetch=(), events=(), mem_events=()):
+    """A v56 soundbank holding `media` {id: wem} in memory (DIDX/DATA, 16-byte aligned) with one HIRC
+    Sound object per media; extra_sources: [(media id, bank id, off, size)] for sources of other banks;
+    media ids in `prefetch` get a prefetch source (their DIDX data is the head of a streamed WEM);
+    events: [(event name, streamed media id)] -> event -> action -> streamed Sound object;
+    mem_events: [(event name, media id of this bank)] -> event -> action -> that media's in-memory Sound."""
+    data = bytearray()
+    didx = b""
+    for m in sorted(media):
+        data += bytes(_a(len(data), 16) - len(data))
+        didx += struct.pack("<III", m, len(data), len(media[m]))
+        data += media[m]
+    bkhd = struct.pack("<II", 56, bank_id) + bytes(8)
+    head = b"BKHD" + struct.pack("<I", len(bkhd)) + bkhd + b"DIDX" + struct.pack("<I", len(didx)) + didx
+    data_pos = len(head) + 8                       # DATA payload, from the start of the bank
+    objs = []
+    for i, m in enumerate(sorted(media)):
+        off = struct.unpack_from("<I", didx, 12 * i + 4)[0]
+        objs.append((m, bank_id if m not in prefetch else 0x1234, data_pos + off, len(media[m]),
+                     2 if m in prefetch else 0))
+    objs += [x + (0,) for x in extra_sources]
+    from ackit.bnk import fnv1
+    obj = (bank_id & 0xFFFFF000) + 0x100         # HIRC object ids: unique per bank, like real Wwise ids
+    hirc_objs = []
+    for i, (m, b, off, size, stream) in enumerate(objs):
+        hirc_objs.append((2, struct.pack("<7I", obj + i, 0x00040001, stream, m, b, off, size) + bytes(6)))
+    for i, (ev, m) in enumerate(events):
+        sid, aid = 0x300 + i, 0x400 + i
+        hirc_objs.append((2, struct.pack("<7I", sid, 0x00040001, 1, m, 0, 0, 0) + bytes(6)))
+        hirc_objs.append((3, struct.pack("<III", aid, 0x4011, sid) + bytes(8)))
+        hirc_objs.append((4, struct.pack("<III", fnv1(ev), 1, aid)))
+    for i, (ev, m) in enumerate(mem_events):
+        aid = 0x500 + i
+        hirc_objs.append((3, struct.pack("<III", aid, 0x4011, obj + sorted(media).index(m)) + bytes(8)))
+        hirc_objs.append((4, struct.pack("<III", fnv1(ev), 1, aid)))
+    hirc = struct.pack("<I", len(hirc_objs))
+    for t, body in hirc_objs:
+        hirc += bytes([t]) + struct.pack("<I", len(body)) + body
+    out = head + b"DATA" + struct.pack("<I", len(data)) + bytes(data) + b"HIRC" + struct.pack("<I", len(hirc)) + hirc
+    if name:
+        nm = name.encode()
+        stid = struct.pack("<II", 1, 1) + struct.pack("<IB", bank_id, len(nm)) + nm
+        out += b"STID" + struct.pack("<I", len(stid)) + stid
+    return out
+
+
+def package(path, body, block_size=0x1000, chunk_blocks=3, tables=(0, 0, 0, 0, 0, 0)):
+    """A fully LZO-compressed UE3 package (docs/formats/upk.md) whose uncompressed data area is `body`.
+    Small blocks so that tests cross block and chunk boundaries. `tables`: (NameCount, NameOffset,
+    ExportCount, ExportOffset, ImportCount, ImportOffset) as absolute uncompressed offsets."""
+    from ackit import lzo
+    folder = b"None\x00"
+    nblocks = max(1, _a(len(body), block_size) // block_size)
+    nchunks = _a(nblocks, chunk_blocks) // chunk_blocks
+
+    def summary(chunks):
+        s = struct.pack("<IHHi", 0x9E2A83C1, 805, 101, 0) + struct.pack("<i", len(folder)) + folder
+        s += struct.pack("<12i", 0, *tables, 0, 0, 0, 0, 0) + b"GUID-synthetic!!"
+        s += struct.pack("<i3i", 1, 0, 0, 0) + struct.pack("<3I", 7748, 0, 2)
+        s += struct.pack("<i", len(chunks)) + b"".join(struct.pack("<4i", *c) for c in chunks)
+        return s + struct.pack("<Ii", 0x12345678, 0)
+    header_u = len(summary([])) + 16                       # uncompressed header (some spare bytes)
+    chunks, blobs = [], []
+    coff = len(summary([(0, 0, 0, 0)] * nchunks)) + 16
+    u = header_u
+    for c in range(nchunks):
+        part = body[(u - header_u):(u - header_u) + block_size * chunk_blocks]
+        blocks = [part[i:i + block_size] for i in range(0, len(part), block_size)] or [b""]
+        comp = [lzo.compress_stored(b) for b in blocks]
+        hdr = struct.pack("<4I", 0x9E2A83C1, block_size, sum(map(len, comp)), len(part))
+        hdr += b"".join(struct.pack("<2I", len(cb), len(b)) for cb, b in zip(comp, blocks))
+        blob = hdr + b"".join(comp)
+        chunks.append((u, len(part), coff, len(blob)))
+        blobs.append(blob)
+        coff += len(blob)
+        u += len(part)
+    s = summary(chunks)
+    out = s + bytes(16) + b"".join(blobs)
+    assert len(s) + 16 == chunks[0][2]
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    open(path, "wb").write(out)
+    return header_u
+
+
+HEADER_U = None   # uncompressed summary size of synthetic packages (fixed; computed on first use)
+
+
+def _header_u():
+    global HEADER_U
+    if HEADER_U is None:
+        import tempfile
+        with tempfile.TemporaryDirectory() as d:
+            HEADER_U = package(os.path.join(d, "x.upk"), b"\0" * 16)
+    return HEADER_U
+
+
+def ue_package(path, objects, prefix=b""):
+    """A package with name/import/export tables. objects: [(class name, object name, bytes or callable)]:
+    a callable gets {name: index} and the export index map and returns the object's bytes.
+    Returns {object name: (absolute offset, size)}."""
+    classes = sorted({c for c, _, _ in objects})
+    names = ["None", "Core", "Class", "LocalizedSubtitles", "Subtitles"] + classes + [n for _, n, _ in objects]
+    idx = {n: i for i, n in enumerate(names)}
+    exp_index = {n: i + 1 for i, (_, n, _) in enumerate(objects)}
+    blobs = [o(idx, exp_index) if callable(o) else o for _, _, o in objects]
+    nt = b"".join(struct.pack("<i", len(n) + 1) + n.encode() + b"\0" + bytes(8) for n in names)
+    it = b"".join(struct.pack("<7i", idx["Core"], 0, idx["Class"], 0, 0, idx[c], 0) for c in classes)
+    base = _header_u() + len(prefix)
+    name_off, imp_off = base, base + len(nt)
+    exp_off = imp_off + len(it)
+    data_off = exp_off + 72 * len(objects)
+    et, pos, offs = b"", data_off, {}
+    for (c, n, _), b in zip(objects, blobs):
+        et += struct.pack("<7i", -(classes.index(c) + 1), 0, 0, idx[n], 0, 0, 0) + bytes(8)
+        et += struct.pack("<ii", len(b), pos) + bytes(4 + 4 + 16 + 4)
+        offs[n] = (pos, len(b))
+        pos += len(b)
+    body = prefix + nt + it + et + b"".join(blobs) + bytes(64)
+    package(path, body, tables=(len(names), name_off, len(objects), exp_off, len(classes), imp_off))
+    return offs
+
+
+def _fstr(text, wide=False):
+    if wide:
+        return struct.pack("<i", -(len(text) + 1)) + text.encode("utf-16-le") + b"\0\0"
+    return struct.pack("<i", len(text) + 1) + text.encode("cp1252") + b"\0"
+
+
+def dialogue(speech_object, texts, wide=()):
+    """An RDialogueEvent export (compact tagged properties, docs/formats/subtitles.md) referencing the
+    export `speech_object`; texts: {lang: [line, ...]}; languages in `wide` are stored as UTF-16."""
+    def build(idx, exp_index):
+        content = struct.pack("<I", len(texts))
+        for lang, lines in texts.items():
+            sub = struct.pack("<I", len(lines))
+            for ln in lines:
+                sub += struct.pack("<HH", 0x0D, 0) + _fstr(ln, lang in wide) + struct.pack("<HHf", 4, 0x0C, 1.5)
+                sub += struct.pack("<H", 0)
+            content += struct.pack("<HH", 0x0D, 0) + _fstr(lang)
+            content += struct.pack("<HHiiii", 9, 0x0C, idx["Subtitles"], 0, len(sub), 0) + sub + struct.pack("<H", 0)
+        b = bytes(4) + struct.pack("<HHi", 0x10, 0x2C, exp_index[speech_object])
+        b += struct.pack("<HH", 0x0D, 0x34) + _fstr("Batman")
+        b += struct.pack("<HHiiii", 9, 0x9C, idx["LocalizedSubtitles"], 0, len(content), 0) + content
+        return b + struct.pack("<H", 0)
+    return build
+
+
 def make_game(root):
     """Create the folder; returns the dict of original WEMs by target."""
     cooked = os.path.join(root, "BmGame", "CookedPCConsole")
@@ -106,4 +263,97 @@ def make_game(root):
     originals = {"%08X:%d" % (SPEECH_ENTRY, i): w for i, w in enumerate(v)}
     originals["%08X" % SINGLE_ENTRY] = single
     originals.update({"%08X/%08X" % (PACK_ID, m): w for m, w in music.items()})
+    originals.update(make_banks(cooked, v))
+    make_localization(root)
     return originals
+
+
+# soundbanks (docs/formats/banks.md): media ids of the in-memory sounds
+FOOT = (0x0A000001, 0x0A000002)       # loose bank CHR_SynthFootsteps_Media
+FOLEY = (0x0B000001, 0x0B000002)      # bank CHR_SynthFoley, copied into both packages
+GUI = (0x0C000001,)                   # bank GUI_Synth (package B only)
+LINE_MEDIA = (0x0D000000, 0x0D000001, 0x0D000002)   # INT media ids of the SPEECH_ENTRY variations
+HEAD = 0x200                          # prefetch head size of variation 1
+
+
+def make_banks(cooked, speech_wems):
+    """Loose bank + two packages (one bank in both) + a prefetch head of INT SPEECH_ENTRY variation 1."""
+    from ackit.bnk import fnv1
+    names = ["Init", "CHR_SynthFootsteps_Media", "CHR_SynthFoley", "GUI_Synth", "Speech_Synth",
+             # banks whose streamed media are the WAD entries (entry ID = FNV-1 of the bank name)
+             "CHR_RadioScanner", "Speech_TAPE_JOKER_1", "Music", "AMB_Steel_B3"]
+    with open(os.path.join(cooked, "CookedBanks.txt"), "w", newline="\r\n") as f:
+        f.write("\n".join(names) + "\n")
+    foot = {FOOT[0]: wem(0.4, 800), FOOT[1]: wem(0.3, 900)}
+    foley = {FOLEY[0]: wem(0.5, 1000), FOLEY[1]: wem(0.6, 1100, 32000)}
+    gui = {GUI[0]: wem(0.5, 1200)}
+    open(os.path.join(cooked, "SFX", "%08X.bnk" % fnv1(names[1])), "wb").write(
+        bank(fnv1(names[1]), foot, name=names[1]))
+    # BKPK table of SPEECH_ENTRY for INT (media ids differ per language; offsets as in the WAD entry)
+    offs, pos = [], 0
+    for w in speech_wems:
+        pos = _a(pos, 0x800)
+        offs.append(pos)
+        pos += len(w)
+    tbl = bytearray(0x48)
+    struct.pack_into("<4sII", tbl, 0, b"BKPK", 0x01020000, SPEECH_ENTRY)
+    struct.pack_into("<IIIIII", tbl, 0x10, 3, 0, 36, 0x48, _a(pos, 0x800), 0)
+    tbl += b"".join(struct.pack("<III", m, len(w), o) for m, w, o in zip(LINE_MEDIA, speech_wems, offs))
+    speech = bank(fnv1(names[4]), {LINE_MEDIA[1]: speech_wems[1][:HEAD]}, prefetch=(LINE_MEDIA[1],))
+    filler = bytes(range(256)) * 9
+    synth_a = filler + bank(fnv1(names[2]), foley) + filler[:777] + bytes(tbl) + speech + filler
+    synth_b = filler[:1000] + bank(fnv1(names[3]), gui) + filler + bank(fnv1(names[2]), foley) + filler
+    package(os.path.join(cooked, "Synth_A.upk"), synth_a)
+    package(os.path.join(cooked, "Synth_B.upk"), synth_b)
+    # dialogue (subtitles) in two packages; the first one also holds a bank with an in-memory sound and the
+    # event that links the dialogue's speech event to INT variation 1 of SPEECH_ENTRY
+    dlg = dialogue(SPEECH_EVENT, SUBTITLES, wide=("RUS",))
+    dlg_wem = wem(0.3, 1400)
+    ev_bank = bank(fnv1("NPC_SynthDialogue"), {DLG_MEDIA: dlg_wem}, name="NPC_SynthDialogue",
+                   events=[(SPEECH_EVENT, LINE_MEDIA[1])])
+    objs = [("AkEvent", SPEECH_EVENT, bytes(8)), ("RDialogueEvent", "Radio_02", dlg)]
+    # a localized bank (one AkBank export: language list + one bank per language) with an in-memory line
+    loc = localized_bank(GRUNT_BANK, {lang: {m: wem(0.4, 500 + 100 * i)} for i, (lang, m) in enumerate(GRUNT_MEDIA.items())},
+                         GRUNT_EVENT)
+    grunt = [("AkEvent", GRUNT_EVENT, bytes(8)), ("RDialogueEvent", "Grunt_01", dialogue(GRUNT_EVENT, GRUNT_SUBS)),
+             ("AkBank", GRUNT_BANK, loc)]
+    ue_package(os.path.join(cooked, "Synth_Dialogue.upk"),
+               objs + [("AkBank", "NPC_SynthDialogue", ev_bank)] + grunt)
+    ue_package(os.path.join(cooked, "Synth_Dialogue2.upk"), objs, prefix=bytes(3000))
+    with open(os.path.join(cooked, "CookedBanks.txt"), "a", newline="\r\n") as f:
+        f.write("NPC_SynthDialogue\n%s\n" % GRUNT_BANK)
+    out = {"%08X" % m: w for m, w in list(foot.items()) + list(foley.items()) + list(gui.items())}
+    out["%08X" % DLG_MEDIA] = dlg_wem
+    return out
+
+
+SPEECH_EVENT = "Speech_Synth_Radio_02"
+GRUNT_BANK, GRUNT_EVENT = "Speech_Synth_Grunt", "Speech_Synth_Grunt_01"
+GRUNT_MEDIA = {"INT": 0x0F000001, "ITA": 0x0F000002}       # one media per language (localized bank)
+GRUNT_SUBS = {"INT": ["Hey you!"], "ITA": ["Ehi, tu!"]}
+
+
+def localized_bank(name, per_lang, event):
+    """AkBank export data of a localized bank: language list, then one bank per language (same id)."""
+    from ackit.bnk import fnv1
+    langs = list(per_lang)
+    lst = struct.pack("<I", len(langs)) + b"".join(_fstr(l) for l in langs)
+    head = bytes(4) + struct.pack("<HHiiii", 9, 0x84, 0, 0, len(lst), 0) + lst
+    blobs = b"".join(bank(fnv1(name), media, name=name, mem_events=[(event, next(iter(media)))]) + bytes(16)
+                     for media in per_lang.values())
+    return head + blobs
+SUBTITLES = {"INT": ["Scanner line two.", "Second part."], "ITA": ["Riga due dello scanner."],
+             "RUS": ["Строка два"]}
+DLG_MEDIA = 0x0E000001
+MENU = {"INT": ("FRONTEND.int", "\r\n[Menu]\r\nStart=Press any key\r\nQuit=\"Quit game\"\r\n; comment\r\n"
+                                "[Tips]\r\nTip=Use the grapnel\r\nTip=Glide over the city\r\n"),
+        "ITA": ("FRONTEND.ITA", "\r\n[Menu]\r\nStart=Premi un tasto\r\nQuit=\"Esci\"\r\n"
+                                "[Tips]\r\nTip=Usa il rampino\r\nTip=Plana sulla citta\r\n")}
+
+
+def make_localization(root):
+    for lang, (fname, text) in MENU.items():
+        d = os.path.join(root, "BmGame", "Localization", lang)
+        os.makedirs(d, exist_ok=True)
+        open(os.path.join(d, fname), "wb").write(b"\xff\xfe" + text.encode("utf-16-le"))
+    open(os.path.join(root, "BmGame", "Localization", "INT", "Version.int"), "wb").write(b"[ID]\r\nUnreal=SYNTH\r\n")

tests/test_app.py

@@ -2,6 +2,7 @@
 import hashlib
 import os
 
+import numpy as np
 import pytest
 
 pytest.importorskip("PySide6")
@@ -53,10 +54,18 @@ def type_search(w, text):
     w._filter_text()
 
 
+def edit(w, name):
+    """Select a sound in the Sounds tab and open it in the Edit tab (what Enter / the Edit button do)."""
+    assert w.select_name(name)
+    w.edit_selected()
+    assert w.tabs.currentWidget() is w.edit and w.editor.sound.name == name
+    return w.editor
+
+
 def test_browse_search_filter(win):
     w, root, _ = win
     assert w.model.rowCount() == len(w.ws.catalog.sounds)
-    type_search(w, "radio scanner 2")
+    type_search(w, "synth radio 02")
     assert w.proxy.rowCount() == 1
     type_search(w, "")
     w.cmb_group.setCurrentText("Music")
@@ -66,13 +75,34 @@ def test_browse_search_filter(win):
     assert w.proxy.rowCount() == w.model.rowCount()
 
 
+def test_sort_order_survives_filter_changes(win):
+    """Rows that a filter shows again go to their sorted place (not appended at the end)."""
+    from PySide6.QtCore import Qt
+    from app.models import C_NAME, SORT_ROLE
+    w, _, _ = win
+
+    def keys():
+        return [w.proxy.data(w.proxy.index(r, C_NAME), SORT_ROLE) for r in range(w.proxy.rowCount())]
+
+    w.table.sortByColumn(C_NAME, Qt.SortOrder.AscendingOrder)
+    assert keys() == sorted(keys())
+    w.cmb_group.setCurrentText("Music")
+    w.clear_filters()
+    type_search(w, "radio")
+    type_search(w, "")
+    assert w.proxy.rowCount() == w.model.rowCount() and keys() == sorted(keys())
+    w.table.sortByColumn(C_NAME, Qt.SortOrder.DescendingOrder)
+    w.cmb_group.setCurrentText("Speech")
+    w.clear_filters()
+    assert keys() == sorted(keys(), reverse=True)
+
+
 def test_replace_apply_revert_byte_identical(win, tmp_path):
     w, root, originals = win
     before = files(root)
-    type_search(w, "CHR_RadioScanner_2")
+    type_search(w, "Speech_Synth_Radio_02")
-    assert w.select_name("CHR_RadioScanner_2")
+    ed = edit(w, "Speech_Synth_Radio_02")
-    ed = w.editor
+    assert ed.orig is not None
-    assert ed.sound.name == "CHR_RadioScanner_2" and ed.orig is not None
     wav = str(tmp_path / "new.wav")
     x, sr = synth.tone(1.0, 44100, 990)
     audio.save_wav(wav, x, sr)
@@ -93,10 +123,186 @@ def test_replace_apply_revert_byte_identical(win, tmp_path):
     assert not w.ws.store.entries
 
 
+def test_bank_sound_apply_revert_in_editor(win, tmp_path):
+    w, root, originals = win
+    before = files(root)
+    w.cmb_arch.setCurrentText("BANK")
+    assert w.proxy.rowCount() == 7                                  # 6 neutral + the INT copy of a localized line
+    ed = edit(w, "CHR_SynthFoley_1")
+    assert "bank CHR_SynthFoley (2 copies)" in ed.meta.text()
+    wav = str(tmp_path / "foley.wav")
+    audio.save_wav(wav, *synth.tone(0.2, 44100, 1300))
+    assert ed.load_file(wav)
+    ed._update_new()
+    ed.apply()
+    assert w.ws.state(ed.sound) == W.MODIFIED and files(root) != before
+    ed.revert()
+    assert files(root) == before and not w.ws.store.entries
+
+
+def test_subtitles_tab_edit_and_link(win):
+    w, root, _ = win
+    before = files(root)
+    w.tabs.setCurrentWidget(w.subs)
+    p = w.subs
+    assert p.lang == "INT" and p.model.rowCount() == 3
+    p.search.setText("scanner")
+    p._filter()
+    assert p.proxy.rowCount() == 1
+    p.table.selectRow(0)
+    assert p.edit.text() == "Scanner line two." and "/ 17 characters" in p.lbl_info.text()
+    p.edit.setText("x" * 30)
+    assert not p.b_apply.isEnabled()                               # longer than the room
+    p.edit.setText("Scan two.")
+    p.apply_current()
+    key = p.current.key
+    assert w.ws.texts.state(key) == "modified" and p.model.modified[key] == "Scan two."
+    p.go_to_sound.emit(p.current.sound)                             # "Go to the sound"
+    assert w.tabs.currentIndex() == 0 and w.preview.sound.name == "Speech_Synth_Radio_02"
+    assert "Scan two." in w.preview.sub.text()                      # the browser shows it, read-only
+    w.edit_selected()
+    box = w.editor.subtitle
+    assert [e.text() for e in box.edits] == ["Scan two.", "Second part."]
+    box.edits[1].setText("Part two.")
+    box.apply()
+    assert w.ws.texts.current(key.replace(":0", ":1")) == "Part two."
+    box.open_in_list.emit(box.keys[1])                              # back to the Subtitles tab
+    assert w.tabs.currentWidget() is p and p.current.key == box.keys[1]
+    box.revert()
+    assert files(root) == before and not w.ws.texts.records()
+
+
+def test_one_language_for_everything(win):
+    w, root, _ = win
+    assert w.language == "INT" and w.cmb_lang.currentText() == "INT" and not w.lbl_voices.text()
+    assert {"INT", "ITA", "RUS"} <= {w.cmb_lang.itemText(i) for i in range(w.cmb_lang.count())}
+    w.load_game(root, "RUS")                                  # no Russian voices: INT speech, RUS texts
+    assert w.language == "RUS" and "voices: INT" in w.lbl_voices.text()
+    assert "INT" in w.ws.catalog.archives
+    w.tabs.setCurrentWidget(w.subs)
+    assert w.subs.lang == "RUS" and "RUS" in w.subs.lbl_lang.text()
+    w.tabs.setCurrentIndex(0)
+    edit(w, "Speech_Synth_Radio_02")
+    assert [e.text() for e in w.editor.subtitle.edits] == ["Строка два"]
+    w.tabs.setCurrentWidget(w.menu)
+    assert w.menu.lang is None and w.menu.model.rowCount() == 0   # no RUS menu files in the synthetic game
+
+
+def test_subtitles_find_replace(win):
+    w, root, _ = win
+    before = files(root)
+    w.tabs.setCurrentWidget(w.subs)
+    p = w.subs
+    assert p.find_replace("line", "row") == 1
+    assert w.ws.texts.current("SUB:INT:%s:0" % synth.SPEECH_EVENT) == "Scanner row two."
+    p.revert_all()
+    assert files(root) == before
+
+
+def test_menu_text_tab(win):
+    w, root, _ = win
+    before = files(root)
+    w.tabs.setCurrentWidget(w.menu)
+    p = w.menu
+    assert p.model.rowCount() == 4
+    p.filters["File"].setCurrentText("FRONTEND.int")
+    p.search.setText("press")
+    p._filter()
+    assert p.proxy.rowCount() == 1
+    p.table.selectRow(0)
+    p.edit.setText("Press Enter to start")
+    p.apply_current()
+    assert w.ws.texts.current("MENU:INT:FRONTEND.int:Menu:Start") == "Press Enter to start"
+    p.cb_mod.setChecked(True)
+    assert p.proxy.rowCount() == 1
+    p.revert_all()
+    assert files(root) == before
+
+
+def test_mod_with_texts(win, tmp_path):
+    w, root, _ = win
+    tx = w.ws.texts
+    tx.apply({"SUB:INT:%s:0" % synth.SPEECH_EVENT: "Mod line.", "MENU:INT:FRONTEND.int:Menu:Quit": "Leave"})
+    pkg = str(tmp_path / "texts.acmod")
+    w.export_mod_dialog(path=pkg, keys=tx.modified(), title="texts")
+    tx.revert_all()
+    w.import_mod_dialog(path=pkg, confirm=False)
+    assert tx.current("MENU:INT:FRONTEND.int:Menu:Quit") == "Leave"
+    assert tx.current("SUB:INT:%s:0" % synth.SPEECH_EVENT) == "Mod line."
+    tx.revert_all()
+
+
+def test_revert_all_reverts_sounds_and_texts(win):
+    w, root, _ = win
+    before = files(root)
+    tx = w.ws.texts
+    tx.apply({"SUB:INT:%s:0" % synth.SPEECH_EVENT: "Undo me.", "MENU:INT:FRONTEND.int:Menu:Quit": "Leave"})
+    s = w.ws.sound("CHR_RadioScanner_1")
+    w.ws.apply([(s, synth.wem(0.5, 1200), {})])
+    w.revert_all()
+    assert files(root) == before and not w.ws.store.entries and not tx.records()
+
+
+def test_mod_import_asks_before_overwriting(win, tmp_path):
+    """Files changed outside the editor are overwritten by a mod only when the user confirms."""
+    w, root, originals = win
+    s = w.ws.sound("CHR_RadioScanner_1")
+    w.ws.apply([(s, synth.wem(0.5, 1200), {})])
+    pkg = str(tmp_path / "m.acmod")
+    w.export_mod_dialog(path=pkg, keys=[s.key], title="m")
+    other = synth.wem(0.4, 150)
+    inject(w.ws.wad_path(s), {s.target: other}, {s.target: w.ws.limit(s)})      # another tool writes the slot
+    asked = []
+    w.confirm = lambda *a: asked.append(a) or False
+    w.import_mod_dialog(path=pkg, confirm=False)
+    assert asked and w.ws.current_bytes(s) == other                            # declined: left alone
+    w.confirm = lambda *a: True
+    w.import_mod_dialog(path=pkg, confirm=False)
+    assert w.ws.state(s) == W.MODIFIED
+    w.revert_all()
+
+
+def test_store_cannot_change_while_changes_are_applied(win, tmp_path, monkeypatch):
+    from app import mainwindow
+    w, root, _ = win
+    s = w.ws.sound("CHR_RadioScanner_1")
+    w.ws.apply([(s, synth.wem(0.5, 1200), {})])
+    store = w.ws.store.root
+
+    class Dlg:
+        def __init__(self, settings, parent):
+            pass
+
+        def exec(self):
+            return mainwindow.QDialog.DialogCode.Accepted
+
+        def values(self):
+            return dict(w.settings, store_root=str(tmp_path / "elsewhere"))
+    monkeypatch.setattr(mainwindow, "SettingsDialog", Dlg)
+    told = []
+    w.inform = lambda *a: told.append(a)
+    w.settings_dialog()
+    assert told and w.settings.get("store_root") != str(tmp_path / "elsewhere") and w.ws.store.root == store
+    w.revert_all()
+
+
+def test_text_undone_outside_shows_banner(win):
+    w, root, _ = win
+    tx = w.ws.texts
+    k = "MENU:INT:FRONTEND.int:Menu:Start"
+    tx.apply({k: "Go"})
+    path = tx.files(k)[0]
+    open(path, "wb").write(open(w.ws.store.full_copy(w.ws.game.rel(path)), "rb").read())   # like 'verify files'
+    w.check_problems()
+    assert not w.banner.isHidden() and "text changes" in w.banner_text.text()
+    w.reapply_lost()
+    assert tx.current(k) == "Go" and tx.state(k) == "modified"
+    tx.revert_all()
+
+
 def test_record_simulated_input(win):
     w, root, _ = win
-    w.select_name("CHR_RadioScanner_1")
+    ed = edit(w, "CHR_RadioScanner_1")
-    ed = w.editor
     x, sr = synth.tone(0.5, 48000, 500)
     ed.recorder.start_simulated(x, sr, realtime=True)
     assert ed.recorder.is_recording()
@@ -129,7 +335,7 @@ def test_notes_tags_favorites(win):
 
 def test_mod_export_import(win, tmp_path):
     w, root, _ = win
-    w.select_name("CHR_RadioScanner_2")
+    edit(w, "Speech_Synth_Radio_02")
     x, sr = synth.tone(1.0, 24000, 1200)
     w.editor.set_source(x, sr, "tone")
     w.editor.apply()
@@ -146,7 +352,7 @@ def test_mod_export_import(win, tmp_path):
 
 def test_changed_underneath_banner(win):
     w, root, originals = win
-    w.select_name("CHR_RadioScanner_1")
+    edit(w, "CHR_RadioScanner_1")
     x, sr = synth.tone(0.5, 24000, 1500)
     w.editor.set_source(x, sr, "tone")
     w.editor.apply()
@@ -174,13 +380,50 @@ class HeldRunner:
             self.real.start(*self.held.pop(0))
 
 
+def test_selection_survives_sort_and_filter(win):
+    from PySide6.QtCore import Qt
+    from app.models import C_LEN
+    w, _, _ = win
+    assert w.select_name("Speech_Synth_Radio_02")
+    s = w.preview.sound
+    w.table.sortByColumn(C_LEN, Qt.SortOrder.DescendingOrder)
+    type_search(w, "radio")
+    cur = w.table.currentIndex()
+    assert cur.isValid() and w.proxy.sound_at(cur) is s and [x.key for x in w.selected_sounds()] == [s.key]
+    type_search(w, "music")                              # the row is filtered out: no current row
+    assert not w.table.currentIndex().isValid() and not w.selected_sounds()
+
+
+def test_list_spinners_while_loading(win):
+    """The sound list and the text lists show the loading overlay while they load (background jobs); a subtitle
+    asked for meanwhile is selected when the rows arrive."""
+    from ackit import subtitles
+    w, root, _ = win
+    held = w.runner = HeldRunner(w.runner)
+    w.load_game(root, "INT")
+    assert w.table_busy.busy and w.table_busy.timer.isActive()
+    held.release()
+    assert not w.table_busy.busy and w.model.rowCount() == len(w.ws.catalog.sounds)
+    w._text_loaded.clear()
+    sub = w.ws.texts.subtitles("INT")[0]
+    key = subtitles.key("INT", sub.speech, sub.line)
+    w.show_subtitle(key)
+    assert w.subs.busy.busy and w.subs.busy.timer.isActive()
+    held.release()
+    assert not w.subs.busy.busy and w.subs.current is not None and w.subs.current.key == key
+
+
 def test_spinners_while_loading(win, tmp_path):
     w, root, _ = win
     held = w.runner = HeldRunner(w.runner)
     w.select_name("CHR_RadioScanner_1")
-    assert w.editor.wf_orig.busy and w.editor.wf_orig._timer.isActive()
+    assert w.preview.wf_orig.busy and w.preview.wf_orig._timer.isActive()
     held.release()
-    assert not w.editor.wf_orig.busy and not w.editor.wf_orig._timer.isActive()
+    assert not w.preview.wf_orig.busy and not w.preview.wf_orig._timer.isActive()
+    w.edit_selected()
+    assert w.editor.wf_orig.busy
+    held.release()
+    assert not w.editor.wf_orig.busy
     wav = str(tmp_path / "a.wav")
     audio.save_wav(wav, *synth.tone(0.3, 22050, 700))
     w.editor.load_file(wav)
@@ -202,11 +445,130 @@ def test_background_toggle_and_filters_look_alike(win):
     assert not any(c.isEditable() for c in (w.cmb_group, w.cmb_cat, w.cmb_char, w.cmb_arch, w.cmb_tag))
 
 
-def test_editor_panel_fits_its_contents(win):
+def test_browser_only_browses_edit_tab_fits(win):
+    """The Sounds tab has no editing tools (player + Edit); the Edit tab needs no scrolling at 1400 x 900 and
+    1280 x 800, even for a speech line with a two-part subtitle."""
+    from PySide6.QtWidgets import QPushButton
     w, root, _ = win
-    w.resize(1400, 900)
+    texts = {b.text() for b in w.vsplit.findChildren(QPushButton)}
+    assert {"Edit", "Add to edit list", "Play original", "Play installed", "Stop"} <= texts
+    assert not texts & {"Record", "Apply to game", "Load file…", "Revert"}
     w.show()
+    for size in ((1400, 900), (1280, 800)):
+        w.resize(*size)
+        edit(w, "Speech_Synth_Radio_02")                            # two subtitle parts: the tallest case
         QTest.qWait(100)
-    need = w.editor.sizeHint().height()
+        assert w.editor.subtitle_box.isVisible() and len(w.editor.subtitle.edits) == 2
-    assert w.scroll.viewport().height() >= need - 2, (w.scroll.viewport().height(), need)
+        for area in (w.editor.parentWidget().parentWidget(), w.edit.rack.parentWidget().parentWidget()):
-    assert w.vsplit.sizes()[0] >= 160
+            assert not area.verticalScrollBar().isVisible() and not area.horizontalScrollBar().isVisible(), size
+    edit(w, "CHR_SynthFoley_1")
+    assert not w.editor.subtitle_box.isVisible()                    # no subtitle: no subtitle box
+
+
+def test_recorder_sample_formats(qapp):
+    """Input devices may deliver 16/32-bit integer, 8-bit or float samples, and reads can split a frame."""
+    import numpy as np
+    from app.audioio import Recorder
+    r = Recorder()
+    x = np.array([[0.5, -0.25], [0.125, -1.0]], np.float32)
+    for sample, raw in (("<i2", (x * 32768).astype("<i2").tobytes()), ("<i4", (x * 2147483648.0).astype("<i4").tobytes()),
+                        ("u1", (x * 128 + 128).astype("u1").tobytes()), ("<f4", x.astype("<f4").tobytes())):
+        r.chunks, r._rest, r.channels, r.sample = [], b"", 2, sample
+        r._ingest(raw[:3])
+        r._ingest(raw[3:])
+        got, _ = r.stop()
+        assert np.allclose(got, x, atol=1 / 128), sample
+
+
+def test_edit_list_replacement_effects_apply_revert(win, tmp_path):
+    """Edit tab: sounds come from the browser onto the edit list; a replacement and effect settings are kept per sound
+    (user data), applied with the effects, reverted byte-identically; Previous / Next walk the list."""
+    from ackit import fx
+    w, root, _ = win
+    e = w.edit
+    before = files(root)
+    a, b = w.ws.sound("Speech_Synth_Radio_02"), w.ws.sound("Speech_TAPE_JOKER_1")
+    w.edit_sounds([a, b])
+    assert w.tabs.currentWidget() is e and [s.key for s in e.model.sounds] == [a.key, b.key]
+    ed = w.editor
+    assert ed.sound is a and not ed.b_prev.isEnabled() and ed.b_next.isEnabled()
+    assert ed.b_apply.isEnabled() and ed.b_export.isEnabled() and ed.b_rec.isEnabled()
+    wav = str(tmp_path / "new.wav")
+    audio.save_wav(wav, *synth.tone(1.2, 44100, 440))
+    assert ed.load_file(wav)
+    assert e.store.has_replacement(a.key) and e.model.state(a) == "Replacement saved"
+    dry = ed.processed()[0].copy()
+    e.rack.apply_look("Radio")                                       # EQ, distortion, static
+    assert fx.summary(ed.effects) == "Band-pass EQ, Distortion, Noise"
+    wet = ed.processed()[0]
+    assert not np.allclose(wet[:len(dry)], dry, atol=1e-3)
+    e.flush()
+    assert e.store.get(a.key)["fx"]["eq"]["on"] is True
+    ed.apply()
+    assert w.ws.state(a) == W.MODIFIED and w.ws.store.get(a.key)["meta"]["effects"] == fx.summary(ed.effects)
+    assert a.key in w.model.modified and e.model.state(a) == "Applied"
+    QTest.qWait(50)
+    ed.b_next.click()                                                # the next sound: no replacement, no effects
+    assert ed.sound is b and ed.src is None and not e.rack.is_on("eq") and ed.b_prev.isEnabled()
+    ed.b_prev.click()                                                # back: replacement, values and effects return
+    assert ed.sound is a and ed.src is not None and e.rack.is_on("eq") and ed.cb_match.isChecked()
+    ed.revert()
+    QTest.qWait(50)
+    assert files(root) == before and not w.ws.store.entries
+    assert e.store.has_replacement(a.key)                            # the replacement stays for later
+    e.table.selectRow(1)
+    e.remove_selected()
+    assert [s.key for s in e.model.sounds] == [a.key] and e.store.list == [a.key]
+
+
+def test_effects_panel_looks_presets_sliders(win):
+    from PySide6.QtCore import QItemSelectionModel
+    from ackit import fx
+    w, _, _ = win
+    w.edit_sounds([w.ws.sound("CHR_RadioScanner_1"), w.ws.sound("CHR_RadioScanner_3")])
+    rack = w.edit.rack
+    rack.apply_preset("reverb", "Hall")
+    s = rack.settings()
+    assert s["reverb"]["on"] and s["reverb"]["decay"] == 2.0 and rack.stack.currentWidget() is rack.pages["reverb"]
+    rack.show_block("echo")
+    rack.controls[("echo", "time")].slider.setValue(100)            # moving a slider turns its effect on
+    assert rack.settings()["echo"]["on"] and rack.items["echo"].font().bold()
+    rack.set_on("echo", False)
+    assert not rack.settings()["echo"]["on"]
+    c = rack.controls[("eq", "low_cut")]                             # log slider, readable value
+    c.slider.setValue(c.slider.maximum())
+    assert rack.settings()["eq"]["low_cut"] == 2000 and c.value_lbl.text() == "2.0 kHz"
+    c.slider.reset.emit()                                            # double-click: back to the default
+    assert rack.settings()["eq"]["low_cut"] == 300
+    rack.cmb_preset.activated.emit(rack.cmb_preset.findText("PA announcement"))   # the preset list
+    assert rack.cmb_preset.currentIndex() == 0
+    s = rack.settings()
+    assert [b.key for b in fx.BLOCKS if s[b.key]["on"]] == ["eq", "echo", "channels"] and s["echo"]["time"] == 160
+    rack.b_copy.click()
+    rack.b_reset.click()
+    assert not fx.active(rack.settings())
+    rack.b_paste.click()
+    assert rack.settings()["echo"]["time"] == 160
+    sm = w.edit.table.selectionModel()
+    for r in range(w.edit.model.rowCount()):
+        sm.select(w.edit.model.index(r, 0), QItemSelectionModel.SelectionFlag.Select |
+                  QItemSelectionModel.SelectionFlag.Rows)
+    w.edit.use_for_selected()
+    assert all(fx.summary(w.edit.store.get(k)["fx"]) == "Band-pass EQ, Echo, Channels"
+               for k in w.edit.store.list)
+    rack.set_channels(5)
+    assert "L R C SL SR" in rack.items["channels"].text()
+
+
+def test_edit_store_moves_data_of_the_first_versions(qapp, tmp_path):
+    """Replacements saved by the first versions (a "take" field, the takes/ folder) are found under the new names."""
+    import json
+    from app.edit import EditStore
+    root = tmp_path / "edits"
+    (root / "takes").mkdir(parents=True)
+    key = "INT:B49CDF48:1"
+    audio.save_wav(str(root / "takes" / "INT_B49CDF48_1.wav"), *synth.tone(0.2, 22050, 300))
+    json.dump({key: {"take": "recording", "gain": 1.5}}, open(root / "index.json", "w"))   # first version: no list
+    st = EditStore(str(root))
+    assert st.list == [key] and st.get(key)["source"] == "recording" and "take" not in st.get(key)
+    assert st.has_replacement(key) and st.load_replacement(key) is not None and not (root / "takes").exists()

tests/test_banks.py

@@ -0,0 +1,207 @@
+"""In-memory sounds (archive BANK) and prefetch heads: catalog scan, apply/revert through Workspace."""
+import hashlib
+import os
+
+import pytest
+
+import synth
+from ackit import banks
+from ackit.game import Game
+from ackit.inject import SizeLimitError
+from ackit.upk import Package
+from ackit.wem import fill, unfill
+from ackit.workspace import Workspace, ConflictError, MODIFIED, ORIGINAL
+
+
+def _sha(p):
+    return hashlib.sha1(open(p, "rb").read()).hexdigest()
+
+
+def _ws(root, tmp_path):
+    return Workspace(root, store_root=str(tmp_path / "store"), archives=["INT", "SFX", "BANK"])
+
+
+def _cooked(root):
+    return Game(root).cooked
+
+
+def _copy(ws, l):
+    return banks.open_file(ws.bank_path(l.file)).read(l.offset, l.size)
+
+
+def test_bank_catalog(synth_game, tmp_path):
+    root, originals = synth_game
+    ws = _ws(root, tmp_path)
+    bank = {s.target: s for s in ws.catalog.sounds if s.archive == "BANK"}
+    neutral = synth.FOOT + synth.FOLEY + synth.GUI + (synth.DLG_MEDIA,)
+    assert set(bank) == {"%08X" % m for m in neutral + (synth.GRUNT_MEDIA["INT"],)}   # INT catalog: no ITA copy
+    grunt = bank["%08X" % synth.GRUNT_MEDIA["INT"]]
+    assert grunt.language == "INT" and grunt.name == synth.GRUNT_EVENT and grunt.group == "Speech"
+    ita = Workspace(root, store_root=str(tmp_path / "s2"), archives=["BANK"]).catalog   # no language: all
+    assert {s.language for s in ita.sounds if s.bank == synth.GRUNT_BANK} == {"INT", "ITA"}
+    foot = bank["%08X" % synth.FOOT[0]]
+    assert foot.bank == "CHR_SynthFootsteps_Media" and foot.name == "CHR_SynthFootsteps_Media_1"
+    assert (foot.group, foot.category) == ("SFX", "Footsteps")
+    assert bank["%08X" % synth.GUI[0]].category == "Interface"
+    foley = bank["%08X" % synth.FOLEY[1]]
+    assert foley.rate == 32000 and foley.size == len(originals[foley.target])
+    assert sorted(l.file for l in ws.locations(foley)) == ["Synth_A.upk", "Synth_B.upk"]
+    for s in bank.values():
+        if s.language:
+            continue
+        assert ws.current_bytes(s) == originals[s.target]
+        assert all(_copy(ws, l) == originals[s.target] for l in ws.locations(s))
+    # the prefetch head belongs to INT variation 1 of the speech entry, not to the BANK archive
+    line = ws.catalog.by_key["INT:%08X:1" % synth.SPEECH_ENTRY]
+    heads = ws.locations(line)
+    assert [(l.file, l.size, l.kind) for l in heads] == [("Synth_A.upk", synth.HEAD, "p")]
+    assert _copy(ws, heads[0]) == originals["%08X:1" % synth.SPEECH_ENTRY][:synth.HEAD]
+    assert ws.locations(ws.catalog.by_key["INT:%08X:0" % synth.SPEECH_ENTRY]) == []
+
+
+def test_bank_apply_every_copy_and_revert_byte_identical(synth_game, tmp_path):
+    root, originals = synth_game
+    ws = _ws(root, tmp_path)
+    cooked = _cooked(root)
+    files = [os.path.join(cooked, f) for f in ("Synth_A.upk", "Synth_B.upk")]
+    before = {f: _sha(f) for f in files}
+    s = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[0]]
+    new = synth.fitting_wem(ws, s, 0.2, 1500)
+    assert len(new) < ws.limit(s)
+    ws.apply([(s, new, {"source": "test"})])
+    assert ws.state(s) == MODIFIED and ws.current_bytes(s) == new
+    for l in ws.locations(s):               # every copy filled exactly: no zero padding after the WEM
+        assert _copy(ws, l) == fill(new, l.size) and len(_copy(ws, l)) == l.size
+        assert unfill(_copy(ws, l)) == new
+    other = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[1]]
+    assert all(_copy(ws, l) == originals[other.target] for l in ws.locations(other))
+    assert all(_sha(f) != before[f] for f in files)
+    assert Package(files[1]).size == Package(ws.store.full_copy(ws.game.rel(files[1]))).size  # layout unchanged
+    ws.revert([s])
+    assert ws.state(s) == ORIGINAL and {f: _sha(f) for f in files} == before
+
+
+def test_fill_slot_exactly():
+    """The game plays an in-memory WEM followed by zero padding as silence: a smaller replacement is grown to
+    its slot with a label list chunk (wem.fill), which the game's own WEMs use too."""
+    from ackit import audio
+    from ackit.wem import FILL_MIN, Wem, fillable
+    w = synth.wem(0.2, 1500)
+    for size in (len(w), len(w) + FILL_MIN, len(w) + 1000):
+        f = fill(w, size)
+        assert len(f) == size and 8 + int.from_bytes(f[4:8], "little") == size     # RIFF covers the slot
+        tags = [b"fmt ", b"data"] if size == len(w) else [b"fmt ", b"LIST", b"data"]
+        assert unfill(f) == w and [c.tag for c in Wem.parse(f).chunks] == tags
+        assert (audio.decode_wem(f)[0] == audio.decode_wem(w)[0]).all()
+    assert not fillable(len(w), len(w) + 3) and not fillable(len(w), len(w) + FILL_MIN - 2)
+    with pytest.raises(ValueError):
+        fill(w, len(w) + 1)
+    f = fill(w, len(w) + 40)
+    a = f.index(b"adtllabl")
+    game_like = f[:a + 12] + (1).to_bytes(4, "little") + b"END\0" + f[a + 20:]   # a real marker label: kept
+    assert len(game_like) == len(f) and unfill(game_like) == game_like
+
+
+def test_bank_apply_refuses_unfillable_and_prepare_fits(synth_game, tmp_path):
+    from ackit.game import GameError
+    root, _ = synth_game
+    ws = _ws(root, tmp_path)
+    s = ws.catalog.by_key["BANK:%08X" % synth.FOOT[1]]
+    bad = next(w for w in (synth.wem(0.1 * (1 - 0.01 * k), 2000) for k in range(40)) if not ws.fits(s, w)
+               and len(w) <= ws.limit(s))
+    with pytest.raises(GameError):
+        ws.apply([(s, bad, {})])
+    assert ws.store.entries == {}
+    x, sr = synth.tone(0.1, 24000, 2000, 1)
+    wem, _ = ws.prepare(s, x, sr)
+    assert ws.fits(s, wem)
+
+
+def test_loose_bank_in_place(synth_game, tmp_path):
+    root, originals = synth_game
+    ws = _ws(root, tmp_path)
+    s = ws.catalog.by_key["BANK:%08X" % synth.FOOT[1]]
+    path = ws.files(s)[0]
+    before, size = _sha(path), os.path.getsize(path)
+    ws.apply([(s, synth.fitting_wem(ws, s, 0.1, 2000), {})])
+    assert os.path.getsize(path) == size and _sha(path) != before
+    ws.revert_all()
+    assert _sha(path) == before
+
+
+def test_partial_revert_keeps_other_changes(synth_game, tmp_path):
+    root, originals = synth_game
+    ws = _ws(root, tmp_path)
+    path = os.path.join(_cooked(root), "Synth_B.upk")
+    before = _sha(path)
+    a = ws.catalog.by_key["BANK:%08X" % synth.FOLEY[0]]
+    b = ws.catalog.by_key["BANK:%08X" % synth.GUI[0]]
+    na, nb = synth.fitting_wem(ws, a, 0.2, 1500), synth.fitting_wem(ws, b, 0.2, 1700)
+    ws.apply([(a, na, {}), (b, nb, {})])
+    ws.revert([a])
+    assert ws.current_bytes(a) == originals[a.target]
+    assert ws.current_bytes(b) == nb and ws.state(b) == MODIFIED
+    ws.revert([b])
+    assert _sha(path) == before
+
+
+def test_speech_line_with_prefetch_head(synth_game, tmp_path):
+    root, originals = synth_game
+    ws = _ws(root, tmp_path)
+    line = ws.catalog.by_key["INT:%08X:1" % synth.SPEECH_ENTRY]
+    files = ws.files(line)
+    assert [os.path.basename(f) for f in files] == ["28CCF006_LOC_INT.wad", "Synth_A.upk"]
+    before = {f: _sha(f) for f in files}
+    new = synth.wem(0.3, 640)
+    ws.apply([(line, new, {})])
+    head = ws.locations(line)[0]
+    assert _copy(ws, head) == (new + bytes(synth.HEAD))[:synth.HEAD]
+    assert ws.current_bytes(line).startswith(new)
+    ws.revert([line])
+    assert {f: _sha(f) for f in files} == before
+
+
+def test_bank_size_limit(synth_game, tmp_path):
+    root, _ = synth_game
+    ws = _ws(root, tmp_path)
+    s = ws.catalog.by_key["BANK:%08X" % synth.FOOT[1]]
+    with pytest.raises(SizeLimitError):
+        ws.apply([(s, synth.wem(3.0, 440), {})])
+    assert ws.store.entries == {}
+
+
+def test_bank_file_changed_underneath(synth_game, tmp_path):
+    root, originals = synth_game
+    ws = _ws(root, tmp_path)
+    s = ws.catalog.by_key["BANK:%08X" % synth.GUI[0]]
+    ws.apply([(s, synth.fitting_wem(ws, s, 0.2, 1500), {})])
+    path = ws.files(s)[0]
+    # another tool rewrites the package (different bytes elsewhere in it)
+    pk = Package(path)
+    other = pk.patched([(pk.header_size + 10, b"OTHER-TOOL")])
+    open(path, "wb").write(other)
+    with pytest.raises(ConflictError):
+        ws.apply([(s, synth.fitting_wem(ws, s, 0.2, 1600), {})])
+    ws.apply([(s, synth.fitting_wem(ws, s, 0.2, 1600), {})], force=True)
+    ws.revert([s])
+    assert _sha(path) == _sha(ws.store.full_copy(ws.game.rel(path)))
+
+
+def test_event_names():
+    sc = banks.Scan()
+    sc.names = {"Play_Synth_Punch", "Stop_Synth_Punch", "Unrelated"}
+    sc.bank_names = {7: "CHR_SynthCombat"}
+    for m in (1, 2, 3):
+        sc.media[m] = {"size": 10, "sha1": {"x"}, "head": b"", "banks": {7}, "kinds": {"m"}}
+    # event -> action -> random container (children 0x50, 0x51) -> sounds -> media 1, 2; media 3 unreferenced
+    from ackit.bnk import fnv1
+    import struct
+    sc.objects = {0x50: 2, 0x51: 2, 0x60: 5, 0x70: 3, fnv1("Play_Synth_Punch"): 4}
+    sc.containers = {0x60: struct.pack("<I", 0x60) + bytes(9) + struct.pack("<3I", 2, 0x50, 0x51) + bytes(5)}
+    sc.actions = {0x70: 0x60}
+    sc.events = {fnv1("Play_Synth_Punch"): [0x70]}
+    sc.sound_media = {0x50: {1}, 0x51: {2}}
+    names = banks.media_names(sc)
+    assert names[1] == ("Play_Synth_Punch_1", "CHR_SynthCombat")
+    assert names[2] == ("Play_Synth_Punch_2", "CHR_SynthCombat")
+    assert names[3] == ("CHR_SynthCombat_1", "CHR_SynthCombat")

tests/test_catalog.py

@@ -1,10 +1,11 @@
 """Catalog: classification rules, naming, character labels, search; the shipped CSVs."""
+import collections
 import csv
 import os
 
 import pytest
 
-from ackit import classify
+from ackit import banks, classify
 from ackit.catalog import Catalog, read_csv, load_characters
 from ackit.game import Game
 from ackit.paths import catalog_dir
@@ -38,9 +39,10 @@ def test_synthetic_catalog_names_and_search(synth_game):
     root, _ = synth_game
     cat = Catalog(Game(root), ["INT", "SFX"])
     names = {s.name for s in cat.sounds}
-    assert {"CHR_RadioScanner_1", "CHR_RadioScanner_2", "CHR_RadioScanner_3", "Music_393BC53A"} <= names
+    assert {"CHR_RadioScanner_1", "Speech_Synth_Radio_02", "CHR_RadioScanner_3", "Music_393BC53A"} <= names
     s = cat.resolve("CHR_RadioScanner#2")
-    assert s.target.endswith(":1") and cat.resolve("chr_radioscanner_2.wav") is s
+    assert s.target.endswith(":1") and cat.resolve("speech_synth_radio_02.wav") is s
+    assert s.line and s.name == "Speech_Synth_Radio_02"        # the game's line: named after its event
     assert cat.resolve(s.key) is s
     assert [x.name for x in cat.search("radio scanner 3")] == ["CHR_RadioScanner_3"]
     with pytest.raises(LookupError) as e:
@@ -49,29 +51,148 @@ def test_synthetic_catalog_names_and_search(synth_game):
 
 
 def test_characters_csv_covers_user_lists():
-    chars = load_characters()
+    chars = load_characters(lang="ITA")
-    assert len(chars) > 3000
+    assert len(chars) > 3000 and len(load_characters()) > 2500     # most labels apply in every language
+    # the lists name ITA sounds by position (wemscan): NAME_i = variation i of the ITA WAD entry (alias NAME#i)
     src = os.path.join(catalog_dir(), "source", "ita", "Joker.txt")
     first = open(src).readline().strip()[:-4]
-    assert "Joker" in chars[first.lower()]
+    base, i = first.rsplit("_", 1)
+    ita = read_csv(os.path.join(catalog_dir(), "ITA.csv"))
+    s = next(s for s in ita if first.lower() in {n.lower() for n in s.names()} or
+             "%s#%s" % (base, i) in s.aliases)
+    assert "Joker" in s.character
+    # every labelled ITA sound keeps its label (takes of one line, NAME#k, share it), and most labels reach INT
+    assert sum(1 for x in ita if x.character) >= 3366
+    assert sum(1 for x in read_csv(os.path.join(catalog_dir(), "INT.csv")) if x.character) > 2900
+
+
+def test_line_names_agree_across_languages_and_subtitles():
+    """Speech sounds carry the game's line name: the same names in every language, and a linked subtitle's
+    sound has the subtitle's line name (positions inside WAD entries differ per language: findings 20)."""
+    from ackit import subtitles
+    ita = {s.name for s in read_csv(os.path.join(catalog_dir(), "ITA.csv")) if s.group == "Speech"}
+    intl = {s.name for s in read_csv(os.path.join(catalog_dir(), "INT.csv")) if s.group == "Speech"}
+    assert len(ita & intl) > 0.97 * len(ita)
+    snd = {s.key: s for s in read_csv(os.path.join(catalog_dir(), "ITA.csv"))}
+    rows = [r for r in subtitles.read_texts(os.path.join(catalog_dir(), "SUB_ITA.csv"), "ITA") if r.sound in snd]
+    same = sum(1 for r in rows if snd[r.sound].name.split("#")[0] == r.speech)
+    assert len(rows) > 8000 and same > 0.99 * len(rows)
+    hdr = next(csv.reader(open(os.path.join(catalog_dir(), "characters.csv"), encoding="utf-8")))
+    assert hdr == ["name", "character", "language"]
 
 
 def test_shipped_csvs_are_consistent():
+    from ackit import subtitles
     for f in os.listdir(catalog_dir()):
+        if f.startswith("SUB_") and f.endswith(".csv"):
+            lines = subtitles.read_texts(os.path.join(catalog_dir(), f), f[4:7])
+            assert len({(s.speech, s.line) for s in lines}) == len(lines), f
+            assert all(len(s.text) <= s.room for s in lines), f
+            continue
         if f.endswith(".csv") and f != "characters.csv":
             rows = read_csv(os.path.join(catalog_dir(), f))
             keys = {r.key for r in rows}
             assert len(keys) == len(rows), f
             assert all(r.group in classify.GROUPS for r in rows)
-            assert all(len(r.sha1) == 16 for r in rows)
+            assert len({(r.language, r.name.lower()) for r in rows}) == len(rows), f   # per language
+            # no checksum only where the copies of a BANK sound differ (1C898731)
+            assert sum(len(r.sha1) != 16 for r in rows) <= (1 if f == "BANK.csv" else 0), f
     hdr = next(csv.reader(open(os.path.join(catalog_dir(), "ITA.csv"), encoding="utf-8")))
     assert hdr[:3] == ["archive", "target", "name"]
+    locs = banks.read_locations(os.path.join(catalog_dir(), banks.LOCATIONS))
+    bank = {r.target for r in read_csv(os.path.join(catalog_dir(), "BANK.csv"))}
+    assert {"%08X" % m for m, ls in locs.items() if ls[0].kind == "m"} == bank
+    assert all(l.sound for ls in locs.values() for l in ls if l.kind == "p")
+
+
+def test_shipped_catalog_cross_references():
+    """Every reference between the shipped files points to something that exists, and every line has one
+    recording and one name per language (copies aside)."""
+    from ackit import subtitles
+    voices = ["INT", "FRA", "ITA", "DEU", "ESN"]
+    rows = {a: read_csv(os.path.join(catalog_dir(), a + ".csv")) for a in voices + ["SFX", "BANK"]}
+    keys = {s.key: s for r in rows.values() for s in r}
+    bank = rows["BANK"]
+    # subtitles -> a voice recording of that language (INT voices for the others)
+    for f in os.listdir(catalog_dir()):
+        if f.startswith("SUB_") and f.endswith(".csv"):
+            lang = f[4:7]
+            voice = lang if lang in voices else "INT"
+            for s in subtitles.read_texts(os.path.join(catalog_dir(), f), lang):
+                for k in filter(None, s.sound.split(";")):
+                    assert k in keys, (f, s.speech, k)
+                    snd = keys[k]
+                    assert snd.archive == voice or (snd.archive == "BANK" and snd.in_language(voice)), (f, k)
+    # heads of WAD recordings -> a WAD recording, never larger than it; no BANK sound is also a head
+    locs = banks.read_locations(os.path.join(catalog_dir(), banks.LOCATIONS))
+    for m, ls in locs.items():
+        for l in ls:
+            if l.kind == "p":
+                assert l.sound in keys and keys[l.sound].archive in voices and l.size <= keys[l.sound].size, l
+                assert "BANK:%08X" % m not in keys
+    # speaker labels -> line names that exist in every voice language
+    names = {L: {s.name for s in rows[L]} for L in voices}
+    with open(os.path.join(catalog_dir(), "characters.csv"), encoding="utf-8") as f:
+        for r in csv.DictReader(f):
+            assert all(r["name"] in names[L] for L in ([r["language"]] if r["language"] else voices)), r
+    # one recording per language per line (copies: a second place of the same recording, same line)
+    for L in voices:
+        per = collections.defaultdict(list)
+        for s in rows[L] + [b for b in bank if b.line and b.in_language(L)]:
+            if s.line:
+                per[s.line].append(s)
+        for lid, ss in per.items():
+            assert len({s.sha1 for s in ss}) == 1, (L, lid, [s.key for s in ss])    # copies are identical
+            assert len([s for s in ss if s.archive == "BANK"]) <= 1, (L, lid)
+    # the same line has the same name in every language (the second place of a copy keeps its position name)
+    common = {}
+    for L in voices:
+        per = collections.defaultdict(set)
+        for s in rows[L]:
+            if s.line:
+                per[s.line].add(s.name)
+        for lid, ns in per.items():
+            common[lid] = common[lid] & ns if lid in common else ns
+    assert all(common.values()), [k for k, v in common.items() if not v][:5]
 
 
 @pytest.mark.game
 def test_real_game_uses_shipped_catalog(real_game_dir):
     g = Game(real_game_dir)
-    cat = Catalog(g, ["ITA", "SFX"])
+    cat = Catalog(g, ["ITA", "SFX", "BANK"])
-    assert len(cat.sounds) == 10816 + 651
+    bank = read_csv(os.path.join(catalog_dir(), "BANK.csv"))
-    s = cat.resolve("Speech_TAPE_JOKER_1_4")
+    assert len(cat.sounds) == 10816 + 651 + sum(1 for s in bank if s.in_language("ITA"))
-    assert s.category == "Interview tapes" and s.size == 44985
+    assert all(s.in_language("ITA") for s in cat.sounds if s.archive == "BANK")
+    assert all(s.line for s in cat.sounds if s.archive == "ITA" and s.group == "Speech")   # every voice line
+    s = cat.resolve("Speech_TAPE_JOKER_1#4")             # position 4 in the ITA entry (the user's list name)
+    assert s.category == "Interview tapes" and s.size == 44985 and s.name.startswith("Speech_TAPE_JOKER_1_")
+    # bank copies of WAD recordings: 396 prefetch heads + 49 recordings also stored whole in banks
+    assert len(cat.heads) == 445 and sum(len(v) for v in cat.heads.values()) == 2185
+    b = cat.resolve("Play_BatmanDeviceImpact_Wood_1")
+    assert b.archive == "BANK" and b.group == "SFX"
+
+
+@pytest.mark.game
+def test_every_name_resolves_to_its_sound(real_game_dir):
+    """A line named after its INT position (CHR_RadioScanner#1) can equal another recording's position alias
+    in another language: the name wins, in every language."""
+    g = Game(real_game_dir)
+    for lang in g.languages:
+        cat = Catalog(g, [lang, "SFX", "BANK"])
+        assert all(cat.resolve(s.name) is s for s in cat.sounds), lang
+
+
+def test_build_catalog_stops_on_changed_subtitles(tmp_path):
+    """tools/build_catalog.py refuses to record a modified game: changed subtitles count like changed sounds."""
+    import importlib.util
+    from types import SimpleNamespace
+    from ackit import subtitles
+    spec = importlib.util.spec_from_file_location("build_catalog", os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "tools", "build_catalog.py"))
+    bc = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(bc)
+    row = subtitles.SubLine("INT", "Speech_X", 0, "Original.", 12, "a", "")
+    subtitles.write_texts(str(tmp_path / "SUB_INT.csv"), [row])
+    game = SimpleNamespace(archives={})
+    assert bc.changed_since_catalog(game, str(tmp_path), [], {"INT": [row]}) == []
+    assert bc.changed_since_catalog(game, str(tmp_path), [], {"INT": [row._replace(text="Edited.")]}) == \
+        ["SUB:INT:Speech_X:0"]

tests/test_catalogcheck.py

@@ -0,0 +1,40 @@
+"""`ackit verify --catalog`: the catalog in use against a fresh scan of the game."""
+import glob
+import os
+
+from ackit.catalog import read_csv, write_csv
+from ackit.game import Game
+from ackit.paths import user_dir
+from ackit.verify import catalogcheck
+
+
+def test_clean_synthetic_catalog_passes(synth_game):
+    root, _ = synth_game
+    rep = catalogcheck.run(Game(root), log=lambda *a: None)
+    assert rep.ok, rep.issues
+    assert rep.checked["BANK"] == 8 and rep.checked["INT"] == 4      # all languages of the localized line
+
+
+def test_corrupted_catalog_is_reported(synth_game):
+    root, _ = synth_game
+    game = Game(root)
+    catalogcheck.run(game, ["INT", "BANK"], log=lambda *a: None)       # builds the cached scans
+    cache = glob.glob(os.path.join(user_dir("cache"), "scan_INT_*.csv"))[0]
+    rows = read_csv(cache)
+    rows[1].category = "Weapons"                                       # wrong category
+    rows.append(rows[0])                                               # duplicate
+    del rows[2]                                                        # missing
+    write_csv(cache, rows)
+    bank = glob.glob(os.path.join(user_dir("cache"), "scan_BANK_*.csv"))[0]
+    brows = read_csv(bank)
+    brows[0].name = brows[1].name                                      # duplicate name
+    write_csv(bank, brows[:-1])                                        # one sound missing
+    rep = catalogcheck.run(game, ["INT", "BANK"], log=lambda *a: None)
+    kinds = {(a, k) for a, k, _, _ in rep.issues}
+    assert {("INT", "category"), ("INT", "duplicate"), ("INT", "missing"), ("BANK", "duplicate"),
+            ("BANK", "missing")} <= kinds
+
+
+def test_real_catalog_quick(real_game_dir):
+    rep = catalogcheck.run(Game(real_game_dir), quick=True, log=lambda *a: None)
+    assert rep.ok, rep.issues[:10]

tests/test_cli.py

@@ -15,24 +15,60 @@ def run(capsys, *args):
 def test_cli_workflow(synth_game, tmp_path, capsys):
     root, originals = synth_game
     g = ["--game", root, "--lang", "INT"]
-    out = run(capsys, "find", "radioscanner", *g)
+    out = run(capsys, "find", "synth radio", *g)
-    assert "CHR_RadioScanner_2" in out
+    assert "Speech_Synth_Radio_02" in out
     wav = str(tmp_path / "line.wav")
     x, sr = synth.tone(1.0, 44100, 660)
     audio.save_wav(wav, x, sr)
-    out = run(capsys, "replace", "CHR_RadioScanner_2", wav, *g)
+    out = run(capsys, "replace", "Speech_Synth_Radio_02", wav, *g)
-    assert "replaced CHR_RadioScanner_2" in out
+    assert "replaced Speech_Synth_Radio_02" in out
-    assert "CHR_RadioScanner_2" in run(capsys, "status", *g)
+    assert "Speech_Synth_Radio_02" in run(capsys, "status", *g)
-    assert "modified" in run(capsys, "info", "CHR_RadioScanner_2", *g)
+    assert "modified" in run(capsys, "info", "Speech_Synth_Radio_02", *g)
     orig = str(tmp_path / "orig.wem")
-    run(capsys, "extract", "CHR_RadioScanner_2", orig, *g)
+    run(capsys, "extract", "Speech_Synth_Radio_02", orig, *g)
     assert open(orig, "rb").read() == originals["%08X:1" % synth.SPEECH_ENTRY]
     mod = str(tmp_path / "m.acmod")
     assert "1 sounds" in run(capsys, "export-mod", mod, *g)
     assert "reverted 1" in run(capsys, "revert", "--all", *g)
     assert "original" in run(capsys, "status", *g)
     assert "exact" in run(capsys, "import-mod", mod, *g)
-    run(capsys, "revert", "CHR_RadioScanner_2", *g)
+    run(capsys, "revert", "Speech_Synth_Radio_02", *g)
+
+
+def test_cli_replace_with_voice_effects(synth_game, tmp_path, capsys):
+    import json
+    root, _ = synth_game
+    g = ["--game", root, "--lang", "INT", "--store", str(tmp_path / "store")]
+    tpl = str(tmp_path / "fx.json")
+    assert "wrote" in run(capsys, "fx-template", tpl)
+    s = json.load(open(tpl))
+    s["eq"]["on"] = True
+    json.dump(s, open(tpl, "w"))
+    wav = str(tmp_path / "line.wav")
+    audio.save_wav(wav, *synth.tone(1.0, 44100, 660))
+    assert "replaced Speech_Synth_Radio_02" in run(capsys, "replace", "Speech_Synth_Radio_02", wav, "--fx", tpl, *g)
+    run(capsys, "revert", "--all", *g)
+    assert "game files are original" in run(capsys, "status", *g)
+
+
+def test_cli_text_language_without_voices(synth_game, capsys):
+    """--lang works for the languages that have only texts (INT voices are loaded for them, like the editor)."""
+    root, _ = synth_game
+    out = run(capsys, "text", "find", "--kind", "subs", "--game", root, "--lang", "RUS", "Строка")
+    assert "SUB:RUS:%s:0" % synth.SPEECH_EVENT in out and "1 match" in out
+
+
+def test_cli_restore_backup(synth_game, tmp_path, capsys):
+    root, _ = synth_game
+    g = ["--game", root, "--lang", "INT", "--store", str(tmp_path / "store")]
+    wav = str(tmp_path / "line.wav")
+    audio.save_wav(wav, *synth.tone(1.0, 44100, 660))
+    run(capsys, "replace", "Speech_Synth_Radio_02", wav, *g)
+    out = run(capsys, "restore-backup", *g)
+    assert "would copy back" in out and "--yes" in out                 # lists only
+    assert "Speech_Synth_Radio_02" in run(capsys, "status", *g)
+    assert "restored" in run(capsys, "restore-backup", "--yes", *g)
+    assert "game files are original" in run(capsys, "status", *g)
 
 
 def test_cli_errors_are_friendly(synth_game, capsys):

tests/test_codec.py

@@ -59,6 +59,13 @@ def test_audio_helpers():
     assert audio.to_channels(x, 1).shape == (48000, 1)
     assert audio.to_channels(x[:, :1], 6).shape == (48000, 6)
     assert abs(audio.peak_db(audio.normalize(x, -3.0)) + 3.0) < 0.01
+    # down-mix keeps every channel: a voice only in the centre of 5.1 goes to both stereo sides at -3 dB (the
+    # standard fold-down) and to mono at its own level
+    c = np.zeros((100, 6), np.float32)
+    c[:, 2] = 0.5
+    assert np.allclose(audio.to_channels(c, 2), 0.5 * 0.7071) and np.allclose(audio.to_channels(c, 1), 0.5)
+    same = np.repeat(x[:, :1], 2, axis=1)                          # identical L/R: mono = either side, no clipping
+    assert np.allclose(audio.to_channels(same, 1)[:, 0], x[:, 0], atol=1e-6)
     assert audio.trim(x, sr, 0.25, 0.75).shape[0] == 24000
     assert audio.resample(x, 48000, 24000).shape[0] == 24000
 

tests/test_containers.py

@@ -0,0 +1,110 @@
+"""LZO, UE3 packages and soundbanks (ackit/lzo.py, upk.py, bnk.py)."""
+import os
+
+import pytest
+
+import synth
+from ackit import lzo
+from ackit.bnk import Bank, find_banks, fnv1, IN_MEMORY
+from ackit.upk import Package
+
+
+# --- LZO ----------------------------------------------------------------------------------------
+@pytest.mark.parametrize("n", [0, 1, 3, 4, 17, 18, 19, 238, 239, 273, 274, 529, 0x20000])
+def test_lzo_stored_roundtrip(n):
+    data = os.urandom(n)
+    assert lzo.decompress(lzo.compress_stored(data), n) == data
+
+
+def test_lzo_matches():
+    # "abc" + M3 match (9 bytes at distance 3) + end
+    assert lzo.decompress(bytes([20]) + b"abc" + bytes([39, 0x08, 0x00]) + lzo.EOF) == b"abc" * 4
+    # "abcd" + M2 match (4 bytes at distance 4) + end
+    assert lzo.decompress(bytes([21]) + b"abcd" + bytes([108, 0]) + lzo.EOF) == b"abcd" * 2
+
+
+def test_lzo_errors():
+    with pytest.raises(lzo.LzoError):
+        lzo.decompress(bytes([25]) + b"abc")                       # truncated
+    with pytest.raises(lzo.LzoError):
+        lzo.decompress(lzo.compress_stored(b"hello") + b"x")        # trailing bytes
+    with pytest.raises(lzo.LzoError):
+        lzo.decompress(lzo.compress_stored(b"hello"), 6)             # wrong size
+
+
+# --- packages -----------------------------------------------------------------------------------
+@pytest.fixture
+def pkg(tmp_path):
+    body = bytes(range(256)) * 80                                     # 20480 bytes: 5 blocks, 2 chunks
+    p = str(tmp_path / "Synth.upk")
+    start = synth.package(p, body)
+    return p, body, start
+
+
+def test_package_read(pkg):
+    p, body, start = pkg
+    pk = Package(p)
+    assert pk.compressed and pk.version == 805 and pk.header_size == start
+    assert pk.size == start + len(body) and len(pk.chunks) == 2 and len(pk.blocks) == 5
+    assert pk.read(start + 4000, 300) == body[4000:4300]                # crosses a block boundary
+    assert pk.read_all() == (start, body)
+
+
+def test_package_patch(pkg):
+    p, body, start = pkg
+    raw = open(p, "rb").read()
+    pk = Package(p)
+    assert pk.patched([]) == raw                                       # nothing changed -> same bytes
+    new = pk.patched([(start + 16334, b"X" * 100)])                    # second chunk, across two blocks
+    q = p + ".new"
+    open(q, "wb").write(new)
+    pk2 = Package(q)
+    exp = bytearray(body)
+    exp[16334:16434] = b"X" * 100
+    assert pk2.read_all()[1] == bytes(exp)
+    # the first chunk is copied verbatim
+    c0 = pk.chunks[0]
+    assert new[c0[2]:c0[2] + c0[3]] == raw[c0[2]:c0[2] + c0[3]]
+    with pytest.raises(ValueError):
+        pk.patched([(start + len(body) - 10, b"too long" * 4)])
+
+
+# --- banks --------------------------------------------------------------------------------------
+def test_bank_parse():
+    media = {0x111: b"RIFF" + bytes(41), 0x222: b"RIFF" + bytes(100)}
+    b = synth.bank(fnv1("CHR_Test"), media, name="CHR_Test")
+    bk = Bank(b)
+    assert bk.id == fnv1("CHR_Test") and bk.version == 56 and bk.size == len(b)
+    assert bk.names() == {bk.id: "CHR_Test"}
+    m = {x.id: x for x in bk.media()}
+    assert b[m[0x222].off:m[0x222].off + m[0x222].size] == media[0x222]
+    assert m[0x222].off % 16 == (bk.chunks["DATA"][0]) % 16
+    src = {s.media: s for s in bk.sources()}
+    assert src[0x111].stream == IN_MEMORY and (src[0x111].off, src[0x111].size) == (m[0x111].off, m[0x111].size)
+
+
+def test_find_banks_in_buffer():
+    a = synth.bank(1, {5: b"x" * 20})
+    b = synth.bank(2, {6: b"y" * 30})
+    buf = b"junkBKHDjunk" + a + bytes(7) + b
+    assert [x.id for x in find_banks(buf)] == [1, 2]
+
+
+def test_fnv1():
+    assert fnv1("Init") == 0x50C63A23 and fnv1("Music") == 0xEDF036D6
+
+
+# --- real game (read-only) ----------------------------------------------------------------------
+def test_real_package_blocks(real_game_dir):
+    from ackit.game import Game
+    p = os.path.join(Game(real_game_dir).cooked, "BmGame.upk")
+    pk = Package(p)
+    assert pk.compressed and pk.patched([]) == open(p, "rb").read()
+    start, data = pk.read_all()
+    banks = list(find_banks(data))
+    assert len(banks) > 20
+    bk = banks[0]
+    m = bk.media()[0]
+    off = start + bk.at + m.off
+    assert pk.read(off, m.size) == data[bk.at + m.off:bk.at + m.off + m.size]
+    assert pk.read(off, 4) == b"RIFF"

tests/test_fx.py

@@ -0,0 +1,225 @@
+"""Effects (ackit/fx.py): each block measured back on synthetic signals."""
+import numpy as np
+import pytest
+
+from ackit import fx
+
+SR = 44100
+
+
+def tone(freq, seconds=1.0, amp=0.3):
+    t = np.arange(int(SR * seconds)) / SR
+    return (amp * np.sin(2 * np.pi * freq * t)).astype(np.float32)[:, None]
+
+
+def impulse(seconds=1.5, at=0.1):
+    x = np.zeros((int(SR * seconds), 1), np.float32)
+    x[int(SR * at)] = 1.0
+    return x
+
+
+def only(key, **params):
+    s = fx.defaults()
+    s[key]["on"] = True
+    s[key].update(params)
+    return s
+
+
+def peak_freq(x, lo=20, hi=None):
+    spec = np.abs(np.fft.rfft(x[:, 0] * np.hanning(len(x))))
+    f = np.fft.rfftfreq(len(x), 1 / SR)
+    m = (f >= lo) & (f <= (hi or SR / 2))
+    return f[m][np.argmax(spec[m])]
+
+
+def band_db(x, f0, width=0.05):
+    spec = np.abs(np.fft.rfft(x[:, 0] * np.hanning(len(x)))) ** 2
+    f = np.fft.rfftfreq(len(x), 1 / SR)
+    m = (f > f0 * (1 - width)) & (f < f0 * (1 + width))
+    return 10 * np.log10(spec[m].sum() + 1e-20)
+
+
+def test_defaults_off_and_identity():
+    s = fx.defaults()
+    assert not fx.active(s) and fx.summary(s) == ""
+    x = tone(440)
+    assert np.array_equal(fx.render(x, SR, s), x)
+
+
+def test_normalize_fills_clamps_and_drops():
+    s = fx.normalize({"eq": {"on": 1, "low_cut": 5, "bogus": 3}, "pitch": {"shift": "x"}, "nope": {}})
+    assert s["eq"]["on"] is True and s["eq"]["low_cut"] == 20 and "bogus" not in s["eq"]
+    assert s["pitch"]["shift"] == 0.0 and "nope" not in s
+    assert s["glitch"]["repeats"] == 2 and isinstance(s["glitch"]["repeats"], int)
+    assert fx.summary(s) == "Band-pass EQ"
+
+
+def test_pitch_shift_octave_up():
+    y = fx.render(tone(220), SR, only("pitch", shift=12))
+    assert abs(peak_freq(y) - 440) < 10
+
+
+def test_formant_shift_keeps_pitch():
+    t = np.arange(SR) / SR
+    x = sum(np.sin(2 * np.pi * 150 * k * t) / k for k in range(1, 20)).astype(np.float32)[:, None] * 0.1
+    y = fx.render(x, SR, only("pitch", formant=6))
+    assert abs(peak_freq(y, 100, 200) - 150) < 5
+    assert not np.allclose(y, x, atol=1e-3)
+
+
+def test_pitch_drift_moves_over_time():
+    y = fx.render(tone(440, 4), SR, only("pitch", drift=4, drift_rate=0.5))
+    peaks = [peak_freq(y[i:i + 8192]) for i in range(SR // 2, len(y) - 8192, SR // 4)]
+    assert max(peaks) - min(peaks) > 100
+
+
+def test_octave_layer_below():
+    y = fx.render(tone(440), SR, only("octaves", interval1=-12, level1=0))
+    assert band_db(y, 220) > band_db(tone(440), 220) + 30
+
+
+def test_ring_modulator_sidebands():
+    y = fx.render(tone(1000), SR, only("robot", ring_freq=100, ring_mix=1.0))
+    assert band_db(y, 900, 0.01) > band_db(y, 1000, 0.01) + 20
+    assert band_db(y, 1100, 0.01) > band_db(y, 1000, 0.01) + 20
+
+
+def test_glitch_longer_and_seeded():
+    x = tone(300, 3)
+    a = fx.render(x, SR, only("glitch", rate=5, length=100, repeats=2, pitch=0, seed=1))
+    b = fx.render(x, SR, only("glitch", rate=5, length=100, repeats=2, pitch=0, seed=1))
+    c = fx.render(x, SR, only("glitch", rate=5, length=100, repeats=2, pitch=0, seed=2))
+    assert np.array_equal(a, b)
+    assert len(a) > len(x) and len(a) - len(x) == int(SR * 0.1) * 2 * round(5 * 3 / 10)
+    assert not np.array_equal(a, c[:len(a)]) or len(a) != len(c)
+
+
+def test_eq_band_pass():
+    x = (tone(100) + tone(1500) + tone(8000)) / 3
+    y = fx.render(x, SR, only("eq", low_cut=300, high_cut=3400, slope=24, mid_gain=0))
+    assert band_db(y, 100) < band_db(x, 100) - 20
+    assert band_db(y, 8000) < band_db(x, 8000) - 20
+    assert abs(band_db(y, 1500) - band_db(x, 1500)) < 0.5
+
+
+def test_distortion_adds_harmonics_same_loudness():
+    x = tone(200)
+    y = fx.render(x, SR, only("distortion", drive=20))
+    assert band_db(y, 600) > band_db(x, 600) + 20
+    assert abs(np.sqrt(np.mean(y ** 2)) - np.sqrt(np.mean(x ** 2))) < 1e-3
+
+
+@pytest.mark.parametrize("opt", [{"clip": -12, "drive": 0}, {"bits": 4, "drive": 0}, {"gsm": True, "drive": 0}])
+def test_distortion_options_change_audio(opt):
+    x = tone(200)
+    assert not np.allclose(fx.render(x, SR, only("distortion", **opt)), x, atol=1e-3)
+
+
+def test_noise_level_relative_to_voice():
+    x = tone(500, 2)
+    y = fx.render(x, SR, only("noise", level=-20, colour=2000))
+    n = y - x
+    ratio = 20 * np.log10(np.sqrt(np.mean(n ** 2)) / np.sqrt(np.mean(x ** 2)))
+    assert abs(ratio + 20) < 1
+
+
+def test_echo_delay_and_tail():
+    x = impulse()
+    y = fx.render(x, SR, only("echo", time=160, feedback=0, mix=0.5))
+    assert len(y) >= len(x)
+    d = np.argsort(np.abs(y[:, 0]))[-2:]
+    assert abs(sorted(d)[1] - sorted(d)[0] - int(SR * 0.16)) <= 1      # pedalboard rounds the delay
+
+
+def test_reverb_tail_and_reverse_swell():
+    x = impulse()
+    y = fx.render(x, SR, only("reverb", mix=0.5, decay=1.0))
+    assert np.sum(y[int(SR * 0.3):] ** 2) > 1e-4
+    v = np.zeros((SR * 2, 1), np.float32)
+    v[SR:SR + SR // 2] = tone(300, 0.5)
+    r = fx.render(v, SR, only("reverb", mix=0.0, reverse=1.0, reverse_every=1.0))
+    assert np.sum(r[SR // 2:SR] ** 2) > 1e-3 and np.sum(v[SR // 2:SR] ** 2) == 0
+
+
+def test_channels_pa_layout():
+    x = impulse(1.0)
+    s = only("channels", front=-12, surround=-6, surround_delay=160, width=0)
+    y = fx.render(x, SR, s, channels=5)                # L R C SL SR
+    assert y.shape[1] == 5
+    i = int(SR * 0.1)
+    assert y[i, 2] == pytest.approx(1.0)
+    assert y[i, 0] == pytest.approx(10 ** (-12 / 20), rel=1e-3)
+    assert np.argmax(np.abs(y[:, 3])) == i + int(SR * 0.16)
+    assert np.max(np.abs(y[:, 3])) == pytest.approx(10 ** (-6 / 20), rel=1e-3)
+    assert fx.render(x, SR, s, channels=1).shape[1] == 1     # mono target: block has no effect
+
+
+def test_channels_width_and_pan():
+    y = fx.render(impulse(1.0), SR, only("channels", width=10), channels=2)
+    assert np.argmax(np.abs(y[:, 1])) - np.argmax(np.abs(y[:, 0])) == int(SR * 0.01)
+    p = fx.render(tone(300, 2), SR, only("channels", pan_rate=1, pan_depth=1), channels=2)
+    left = [np.sqrt(np.mean(p[i:i + 2000, 0] ** 2)) for i in range(0, len(p) - 2000, 4410)]
+    assert max(left) > 2 * min(left) + 1e-3
+
+
+def test_all_blocks_together_finite():
+    s = fx.defaults()
+    for b in fx.BLOCKS:
+        s[b.key]["on"] = True
+    s["distortion"]["gsm"] = True
+    x = tone(220, 2) + tone(1300, 2) * 0.3
+    y = fx.render(x, SR, s, channels=4)
+    assert y.shape[1] == 4 and np.isfinite(y).all() and len(y) >= len(x)
+    assert np.array_equal(y, fx.render(x, SR, s, channels=4))     # deterministic
+
+
+def rt60(ir):
+    """Reverb time from the Schroeder backward integral (-5 to -25 dB, extrapolated to -60)."""
+    e = np.cumsum((ir ** 2)[::-1])[::-1]
+    db = 10 * np.log10(e / e[0] + 1e-30)
+    a, b = np.argmax(db < -5), np.argmax(db < -25)
+    return (b - a) / SR * 3
+
+
+@pytest.mark.parametrize("decay", [0.3, 1.0, 3.0])
+def test_room_response_decay_measured_back(decay):
+    ir = fx.room_response(SR, 1, decay, predelay=0, early=0.0, damping=0.0)
+    assert abs(rt60(ir[:, 0]) - decay) < 0.15 * decay
+    assert np.sum(ir ** 2) == pytest.approx(1.0, rel=1e-3)
+
+
+def test_room_response_damping_predelay_width():
+    ir = fx.room_response(SR, 1, 1.5, predelay=0, early=0.0, damping=1.0)[:, 0]
+    late = ir[int(0.5 * SR):][:, None]
+    assert band_db(late, 8000, 0.3) < band_db(late, 300, 0.3) - 20       # the highs die first
+    pre = fx.room_response(SR, 1, 0.5, predelay=40)[:, 0]
+    assert np.max(np.abs(pre[:int(0.039 * SR)])) == 0 and np.max(np.abs(pre[int(0.04 * SR):])) > 0
+    wide = fx.room_response(SR, 2, 0.5, width=1.0)
+    mono = fx.room_response(SR, 2, 0.5, width=0.0)
+    assert abs(np.corrcoef(wide.T)[0, 1]) < 0.2 and np.corrcoef(mono.T)[0, 1] > 0.99
+
+
+def test_presets_and_looks_are_valid():
+    for block, items in fx.PRESETS.items():
+        keys = {p.key for p in fx.BY_KEY[block].params}
+        for name, values in items:
+            assert set(values) <= keys, (block, name)
+            s = fx.with_preset(fx.defaults(), block, name)
+            assert s[block]["on"] and all(s[block][k] == values[k] for k in values), (block, name)
+    x = tone(300, 1)
+    for name, _ in fx.LOOKS:
+        s = fx.look(name)
+        assert fx.active(s)
+        y = fx.render(x, SR, s, channels=5)
+        assert np.isfinite(y).all() and len(y) >= len(x)
+
+
+def test_reverb_mix_dry_to_wet_and_old_name():
+    x = impulse(1.0)
+    i = int(SR * 0.1)
+    assert np.array_equal(fx.render(x, SR, only("reverb", mix=0.0)), x)                   # 0: dry only
+    wet = fx.render(x, SR, only("reverb", mix=1.0, predelay=30, early=0.0))
+    assert abs(wet[i, 0]) < 1e-3 and np.max(np.abs(wet[i + int(SR * 0.03):])) > 0       # 1: no dry impulse left
+    half = fx.render(x, SR, only("reverb", mix=0.5, predelay=30, early=0.0))
+    assert half[i, 0] == pytest.approx(np.cos(np.pi / 4), rel=1e-3)                      # equal power
+    assert fx.normalize({"reverb": {"on": True, "wet": 0.4}})["reverb"]["mix"] == 0.4    # older settings

tests/test_game.py

@@ -46,3 +46,17 @@ def test_not_a_game_folder(tmp_path):
     import pytest
     with pytest.raises(G.GameError):
         G.Game(str(tmp_path))
+
+
+def test_language_without_voices(synth_game, tmp_path, monkeypatch):
+    """A game set to a language with texts only (e.g. JPN) is in that language, with the INT voices."""
+    root, _ = synth_game
+    os.makedirs(os.path.join(root, "BmGame", "Localization", "JPN"), exist_ok=True)
+    cfg = tmp_path / "cfg"
+    cfg.mkdir()
+    (cfg / "UserEngine.ini").write_bytes("[Engine.Engine]\r\nLanguage=Jpn\r\n".encode("utf-16"))
+    monkeypatch.setattr(G.Game, "config_dirs", lambda self: [str(cfg)])
+    g = G.Game(root)
+    assert g.language() == "JPN" and g.voice_archive(g.language()) == "INT"
+    from ackit.catalog import Catalog
+    assert "INT" in Catalog(g).archives                       # the default catalog loads the INT voices