From 3006b04b19e59d737ea3c3da3fa2de4c94d0c8ae Mon Sep 17 00:00:00 2001 From: Frotty Date: Tue, 13 Oct 2026 11:39:16 +0200 Subject: [PATCH 1/2] Backlog: what a large map's build still spends, and the updateModel scan off the critical path Records the costs measured but not changed while profiling a large map's build: the two per-file linear lookups in JMPQ3 behind the map config and the map data injection, single-threaded recompression of a release map, the Guava collections and copied dataflow states in the type check, and the per-subclass member lookups when translating methods to IM. And that keeping the WFile hash did not speed up a load, as the scan is off the critical path while parsing is parallel. --- BACKLOG.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/BACKLOG.md b/BACKLOG.md index fac80b7f2..74df8b6db 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -115,6 +115,25 @@ Notes; finished-work narrative does not. `wFile`, which canonicalises the path of each unit once. An index from file to unit would remove the scan, but the list of the model has several writers (`updateModel`, `removeCompilationUnit`, `retainCompilationUnits`, the incremental checks), so it has to stay in step with all of them; that is about 0.25 s of a castle fight build. + On a project of 1,625 files the comparisons were 40% of the loading thread's samples, yet keeping the hash of a + `WFile` (which made them integer comparisons) changed the load time by nothing measurable: the loading thread + waits for the parse threads, so the scan is not on the critical path while parsing is parallel. +- **What a large map's build still spends, by step (`-measure`), and why.** Measured on a Jass map of 1,625 files + with thousands of compile-time functions: + - *MPQ, on every run and build:* `Applying Map Config` (1.2 s) and `Injecting map data` (1.8 s) are mostly two + lookups in JMPQ3 which are linear per call and called once per file: `MpqArchive.localesOf` scans every hash + table mapping (from `MpqArchiveWriter.from`) and `MpqArchiveWriter.contains` streams over every pending key + (from `Jmpq3BasedEditor.hasFile` per imported file). `Finalize map` (7.8 s, release builds only) recompresses every + file on one thread. Fix in JMPQ3 and bump the pin. + - *Type check (12 s):* Guava collections are about 11% of its samples (name tables about 5%, the dataflow states + about 4%). The dataflow anomaly analysis (`VarStates`, `VState`) copies its immutable maps and multimaps on every + read and write, about 7% with its merges; persistent or mutable states would remove most of that. The rest is + attribute evaluation, see the next item. + - *Translation to IM (2.4 s):* `ImTranslator.getClassesWithImplementation` is 20% of it: for each method a member + lookup in every subclass, so a hierarchy where thousands of subclasses override one method pays methods times + subclasses lookups (the overrides are distinct, not duplicates). A subclass which declares no member of that + name could take its superclass's answer, but modules, interfaces and the binding of type arguments + (`matchDefLinkReceiver`) must come out the same: compare with the current answer on every call before switching. - **Parallel type check and per-function IM passes.** Parsing is parallel (above); the rest of a build is not, and JFR of fresh master says where the time is (shares of the compiler thread's samples, castle fight without optimisations / optimised / zombie defense): type check light and heavy phases (attribute evaluation, per unit and per From bf3da7e618c9749fcc9eedfc6b5f277cdaa2bc32 Mon Sep 17 00:00:00 2001 From: Frotty Date: Tue, 13 Oct 2026 11:56:59 +0200 Subject: [PATCH 2/2] Backlog: the superclass shortcut for implementations was tried; the slow subclasses override the method themselves --- BACKLOG.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 74df8b6db..4f2f05584 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -131,9 +131,12 @@ Notes; finished-work narrative does not. attribute evaluation, see the next item. - *Translation to IM (2.4 s):* `ImTranslator.getClassesWithImplementation` is 20% of it: for each method a member lookup in every subclass, so a hierarchy where thousands of subclasses override one method pays methods times - subclasses lookups (the overrides are distinct, not duplicates). A subclass which declares no member of that - name could take its superclass's answer, but modules, interfaces and the binding of type arguments - (`matchDefLinkReceiver`) must come out the same: compare with the current answer on every call before switching. + subclasses lookups (the overrides are distinct, not duplicates). Tried and dropped: letting a subclass which + brings no function of that name take its superclass's answer agreed with the current answer on every call + (11,727 on that map, and the class and generics suites) but changed nothing measurable, because the subclasses + which make it slow override the method themselves. What would help is a cheaper lookup per overriding subclass + (its own declaration of the name is the candidate), with the binding of type arguments (`matchDefLinkReceiver`) + coming out the same. - **Parallel type check and per-function IM passes.** Parsing is parallel (above); the rest of a build is not, and JFR of fresh master says where the time is (shares of the compiler thread's samples, castle fight without optimisations / optimised / zombie defense): type check light and heavy phases (attribute evaluation, per unit and per