-
-
Notifications
You must be signed in to change notification settings - Fork 34.5k
src: cache missing package.json files in the C++ package config cache #60425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
src: cache missing package.json files in the C++ package config cache #60425
Conversation
|
Review requested:
|
cd265c8 to
9c5c99f
Compare
9c5c99f to
003f911
Compare
003f911 to
ede35c9
Compare
src/node_modules.cc
Outdated
| return nullptr; | ||
| } | ||
|
|
||
| void BindingData::GetNearestParentPackageJSON( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Can we use existing methods rather then adding a new method?
- I believe
THROW_IF_INSUFFICIENT_PERMISSIONScall is missing in this method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was re-added as a straight revert of the code I deleted in #59888, but yeah it is 99% the same as GetNearestParentPackageJSONType. I can refactor this to remove that redundancy if you'd prefer!
I think TraverseParent handles the permissions checking you're referring to here, but it doesn't throw if there are insufficient permissions, it just returns nullptr. I'm not sure I have enough overall context to judge if that's right or wrong, but it's what this has always done!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pulled the common path-handling stuff into a method to reduce the redundancy here a bit. There were also changes in this area rebasing onto main pulled in, so it looks a tiny bit different now
|
After these 2 review comments I've left, I think we should merge this. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #60425 +/- ##
==========================================
+ Coverage 88.54% 88.56% +0.01%
==========================================
Files 703 703
Lines 208259 208267 +8
Branches 40165 40165
==========================================
+ Hits 184410 184444 +34
+ Misses 15861 15813 -48
- Partials 7988 8010 +22
🚀 New features to boost your workflow:
|
ede35c9 to
86c48ea
Compare
|
Lint issue is unrelated: #60636 |
|
@mcollina, @jasnell, @joyeecheung, @dario-piotrowicz I'm really sorry for pestering you guys as you may already have plenty of other stuff to do, but would please take some time to review this PR? The regression (in all current NodeJS main versions) is highly impacting my use cases and I believe the fix is quite safe. Thanks for your time! |
mcollina
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
|
@michaelsmithxyz can you fix linting? |
86c48ea to
dd093fe
Compare
|
@mcollina Should be good, just needed a rebase! |
|
How can we get this change integrated in Node 22, 24 and 25? |
Hi! This PR will be added to Node.js release lines following its merge. |
|
Is there a way to contribute in getting this PR merged? |
Commit Queue failed- Loading data for nodejs/node/pull/60425 ✔ Done loading data for nodejs/node/pull/60425 ----------------------------------- PR info ------------------------------------ Title src: cache missing package.json files in the C++ package config cache (#60425) Author Michael Smith <me@michaelsmith.xyz> (@michaelsmithxyz) Branch michaelsmithxyz:fix_module_loading_perf_regression -> nodejs:main Labels c++, module, needs-ci, typings Commits 2 - src: cache missing package.json files in the C++ package config cache - feedback and incorporate changes to path handling from rebase Committers 1 - Michael Smith <mikesm@zillowgroup.com> PR-URL: https://github.com/nodejs/node/pull/60425 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/60425 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> -------------------------------------------------------------------------------- ℹ This PR was created on Mon, 27 Oct 2025 05:02:49 GMT ✔ Approvals: 2 ✔ - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/60425#pullrequestreview-3425521599 ✔ - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/60425#pullrequestreview-3492031355 ✔ Last GitHub CI successful ℹ Last Full PR CI on 2026-01-13T21:17:59Z: https://ci.nodejs.org/job/node-test-pull-request/70792/ - Querying data for job/node-test-pull-request/70792/ ✔ Build data downloaded ✔ Last Jenkins CI successful -------------------------------------------------------------------------------- ✔ No git cherry-pick in progress ✔ No git am in progress ✔ No git rebase in progress -------------------------------------------------------------------------------- - Bringing origin/main up to date... From https://github.com/nodejs/node * branch main -> FETCH_HEAD ✔ origin/main is now up-to-date - Downloading patch for 60425 From https://github.com/nodejs/node * branch refs/pull/60425/merge -> FETCH_HEAD ✔ Fetched commits as 805673a0c85d..dd093fe9592d -------------------------------------------------------------------------------- [main ce6b685534] src: cache missing package.json files in the C++ package config cache Author: Michael Smith <mikesm@zillowgroup.com> Date: Sun Oct 26 21:27:52 2025 -0400 4 files changed, 123 insertions(+), 77 deletions(-) [main 836c1ad949] feedback and incorporate changes to path handling from rebase Author: Michael Smith <mikesm@zillowgroup.com> Date: Sat Nov 8 10:44:52 2025 -0500 2 files changed, 21 insertions(+), 28 deletions(-) ✔ Patches applied There are 2 commits in the PR. Attempting autorebase. (node:2277) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated. (Use `node --trace-deprecation ...` to show where the warning was created) Rebasing (2/4) Executing: git node land --amend --yes --------------------------------- New Message ---------------------------------- src: cache missing package.json files in the C++ package config cachehttps://github.com/nodejs/node/actions/runs/21170892221 |
|
Landed in 91a9978 |
PR-URL: #60425 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
PR-URL: nodejs#60425 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [node](https://nodejs.org) ([source](https://github.com/nodejs/node)) | minor | `25.4.0` → `25.5.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>nodejs/node (node)</summary> ### [`v25.5.0`](https://github.com/nodejs/node/releases/tag/v25.5.0): 2026-01-26, Version 25.5.0 (Current), @​aduh95 [Compare Source](nodejs/node@v25.4.0...v25.5.0) ##### Notable Changes - \[[`99a4e51f93`](nodejs/node@99a4e51f93)] - **crypto**: update root certificates to NSS 3.119 (Node.js GitHub Bot) [#​61419](nodejs/node#61419) - \[[`fbe4da5725`](nodejs/node@fbe4da5725)] - **(SEMVER-MINOR)** **deps**: add LIEF as a dependency (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`0feab0f083`](nodejs/node@0feab0f083)] - **(SEMVER-MINOR)** **deps**: add tools and scripts to pull LIEF as a dependency (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`e91b296001`](nodejs/node@e91b296001)] - **(SEMVER-MINOR)** **fs**: add ignore option to fs.watch (Matteo Collina) [#​61433](nodejs/node#61433) - \[[`b351910af1`](nodejs/node@b351910af1)] - **(SEMVER-MINOR)** **sea**: add `--build-sea` to generate SEA directly with Node.js binary (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`957292e233`](nodejs/node@957292e233)] - **(SEMVER-MINOR)** **sea**: split sea binary manipulation code (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`f289817ff8`](nodejs/node@f289817ff8)] - **(SEMVER-MINOR)** **sqlite**: enable defensive mode by default (Bart Louwers) [#​61266](nodejs/node#61266) - \[[`069f3603e2`](nodejs/node@069f3603e2)] - **(SEMVER-MINOR)** **sqlite**: add sqlite prepare options args (Guilherme Araújo) [#​61311](nodejs/node#61311) - \[[`5a984b9a09`](nodejs/node@5a984b9a09)] - **src**: use node- prefix on thread names (Stewart X Addison) [#​61307](nodejs/node#61307) - \[[`75c06bc2a8`](nodejs/node@75c06bc2a8)] - **(SEMVER-MINOR)** **test**: migrate to `--build-sea` in existing SEA tests (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`cabd58f1cb`](nodejs/node@cabd58f1cb)] - **(SEMVER-MINOR)** **test**: use fixture directories for sea tests (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`ff1fcabfc9`](nodejs/node@ff1fcabfc9)] - **(SEMVER-MINOR)** **test\_runner**: support expecting a test-case to fail (Jacob Smith) [#​60669](nodejs/node#60669) ##### Commits - \[[`778a56f3c9`](nodejs/node@778a56f3c9)] - **assert,util**: fix deep comparison for sets and maps with mixed types (Ruben Bridgewater) [#​61388](nodejs/node#61388) - \[[`32cd18e37f`](nodejs/node@32cd18e37f)] - **async\_hooks**: enabledHooksExist shall return if hooks are enabled (Gerhard Stöbich) [#​61054](nodejs/node#61054) - \[[`482b2568bc`](nodejs/node@482b2568bc)] - **benchmark**: add SQLite benchmarks (Guilherme Araújo) [#​61401](nodejs/node#61401) - \[[`e9a34263bb`](nodejs/node@e9a34263bb)] - **buffer**: make methods work on Uint8Array instances (Neal Beeken) [#​56578](nodejs/node#56578) - \[[`8255cdefcf`](nodejs/node@8255cdefcf)] - **build**: add `--shared-nbytes` configure flag (Antoine du Hamel) [#​61341](nodejs/node#61341) - \[[`8dd379d110`](nodejs/node@8dd379d110)] - **build**: update android-patches/trap-handler.h.patch (Mo Luo) [#​60369](nodejs/node#60369) - \[[`1b4b5eb0e4`](nodejs/node@1b4b5eb0e4)] - **build**: update devcontainer.json to use paired nix env (Joyee Cheung) [#​61414](nodejs/node#61414) - \[[`86e2a763ad`](nodejs/node@86e2a763ad)] - **build**: infer cargo mode with gyp var build\_type directly (Chengzhong Wu) [#​61354](nodejs/node#61354) - \[[`7e211e6942`](nodejs/node@7e211e6942)] - **build**: add embedtest into native suite (Joyee Cheung) [#​61357](nodejs/node#61357) - \[[`637470e79f`](nodejs/node@637470e79f)] - **build**: fix misplaced comma in ldflags (hqzing) [#​61294](nodejs/node#61294) - \[[`a1a0f77a45`](nodejs/node@a1a0f77a45)] - **build**: fix crate vendor file checksums on windows (Chengzhong Wu) [#​61329](nodejs/node#61329) - \[[`d597b8e342`](nodejs/node@d597b8e342)] - **build,tools**: fix addon build deadlock on errors (Vladimir Morozov) [#​61321](nodejs/node#61321) - \[[`b5cdc27ba4`](nodejs/node@b5cdc27ba4)] - **build,win**: improve logs when ClangCL is missing (Mike McCready) [#​61438](nodejs/node#61438) - \[[`ef01f0c033`](nodejs/node@ef01f0c033)] - **build,win**: update WinGet configurations to Python 3.14 (Mike McCready) [#​61431](nodejs/node#61431) - \[[`d8a1cdeefe`](nodejs/node@d8a1cdeefe)] - **child\_process**: treat ipc length header as unsigned uint32 (Ryuhei Shima) [#​61344](nodejs/node#61344) - \[[`588b00fafa`](nodejs/node@588b00fafa)] - **cluster**: fix port reuse between cluster (Ryuhei Shima) [#​60141](nodejs/node#60141) - \[[`99a4e51f93`](nodejs/node@99a4e51f93)] - **crypto**: update root certificates to NSS 3.119 (Node.js GitHub Bot) [#​61419](nodejs/node#61419) - \[[`048f7a5c9c`](nodejs/node@048f7a5c9c)] - **deps**: upgrade npm to 11.8.0 (npm team) [#​61466](nodejs/node#61466) - \[[`fbe4da5725`](nodejs/node@fbe4da5725)] - **(SEMVER-MINOR)** **deps**: add LIEF as a dependency (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`0feab0f083`](nodejs/node@0feab0f083)] - **(SEMVER-MINOR)** **deps**: add tools and scripts to pull LIEF as a dependency (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`4bb00d7e3c`](nodejs/node@4bb00d7e3c)] - **deps**: update googletest to [`8508785`](nodejs/node@8508785) (Node.js GitHub Bot) [#​61417](nodejs/node#61417) - \[[`6a3c614f27`](nodejs/node@6a3c614f27)] - **deps**: update sqlite to 3.51.2 (Node.js GitHub Bot) [#​61339](nodejs/node#61339) - \[[`13c0397d6d`](nodejs/node@13c0397d6d)] - **deps**: update icu to 78.2 (Node.js GitHub Bot) [#​60523](nodejs/node#60523) - \[[`098ec6f196`](nodejs/node@098ec6f196)] - **deps**: update ada to v3.4.0 (Yagiz Nizipli) [#​61315](nodejs/node#61315) - \[[`320b576125`](nodejs/node@320b576125)] - **deps**: update zlib to 1.3.1-e00f703 (Node.js GitHub Bot) [#​61135](nodejs/node#61135) - \[[`98f5e7cf51`](nodejs/node@98f5e7cf51)] - **deps**: V8: cherry-pick [highway@`dcc0ca1`](https://github.com/highway/node/commit/dcc0ca1cd42) (Richard Lau) [#​61008](nodejs/node#61008) - \[[`e326df79c9`](nodejs/node@e326df79c9)] - **deps**: V8: backport [`209d2db`](nodejs/node@209d2db9e24a) (Zhijin Zeng) [#​61322](nodejs/node#61322) - \[[`ccfd9d9b30`](nodejs/node@ccfd9d9b30)] - **doc**: remove `v` prefix for version references (Mike McCready) [#​61488](nodejs/node#61488) - \[[`b6cc5d77a1`](nodejs/node@b6cc5d77a1)] - **doc**: mention constructor comparison in assert.deepStrictEqual (Hamza Kargin) [#​60253](nodejs/node#60253) - \[[`236d7ee635`](nodejs/node@236d7ee635)] - **doc**: add CVE delay mention (Rafael Gonzaga) [#​61465](nodejs/node#61465) - \[[`0729fb6ee7`](nodejs/node@0729fb6ee7)] - **doc**: update previous version links in BUILDING (Mike McCready) [#​61457](nodejs/node#61457) - \[[`0fb464252f`](nodejs/node@0fb464252f)] - **doc**: include OpenJSF handle for security stewards (Rafael Gonzaga) [#​61454](nodejs/node#61454) - \[[`3331bdca7c`](nodejs/node@3331bdca7c)] - **doc**: clarify process.argv\[1] behavior for -e/--eval (Jeevankumar S) [#​61366](nodejs/node#61366) - \[[`94b34c38e2`](nodejs/node@94b34c38e2)] - **doc**: remove Windows Dev Home instructions from BUILDING (Mike McCready) [#​61434](nodejs/node#61434) - \[[`a17016ee81`](nodejs/node@a17016ee81)] - **doc**: clarify TypedArray properties on Buffer (Roman Reiss) [#​61355](nodejs/node#61355) - \[[`214fac9d7e`](nodejs/node@214fac9d7e)] - **doc**: update Python 3.14 manual install instructions (Windows) (Mike McCready) [#​61428](nodejs/node#61428) - \[[`6a32a685a6`](nodejs/node@6a32a685a6)] - **doc**: note resume build should not be done on node-test-commit (Stewart X Addison) [#​61373](nodejs/node#61373) - \[[`2a8e8dfaf3`](nodejs/node@2a8e8dfaf3)] - **doc**: refine WebAssembly error documentation (sangwook) [#​61382](nodejs/node#61382) - \[[`f3caf27f8b`](nodejs/node@f3caf27f8b)] - **doc**: add deprecation history for url.parse (Eng Zer Jun) [#​61389](nodejs/node#61389) - \[[`5ab8057856`](nodejs/node@5ab8057856)] - **doc**: add marco and rafael in last sec release (Marco Ippolito) [#​61383](nodejs/node#61383) - \[[`f83cb1e785`](nodejs/node@f83cb1e785)] - **doc**: packages: example of private import switch to internal (coderaiser) [#​61343](nodejs/node#61343) - \[[`3d23bcd0e2`](nodejs/node@3d23bcd0e2)] - **doc**: add esm and cjs examples to node:v8 (Alfredo González) [#​61328](nodejs/node#61328) - \[[`1d159550e0`](nodejs/node@1d159550e0)] - **doc**: added 'secure' event to tls.TLSSocket (ikeyan) [#​61066](nodejs/node#61066) - \[[`90080d2892`](nodejs/node@90080d2892)] - **doc**: restore [@​watilde](https://github.com/watilde) to collaborators (Daijiro Wachi) [#​61350](nodejs/node#61350) - \[[`a87f7a50f8`](nodejs/node@a87f7a50f8)] - **doc**: run license-builder (github-actions\[bot]) [#​61348](nodejs/node#61348) - \[[`adf5c84701`](nodejs/node@adf5c84701)] - **doc**: clean up writing-and-running-benchmarks.md (Hardanish Singh) [#​61345](nodejs/node#61345) - \[[`2be98add0c`](nodejs/node@2be98add0c)] - **doc**: document ALPNCallback option for TLSSocket constructor (ikeyan) [#​61331](nodejs/node#61331) - \[[`2db4893c8d`](nodejs/node@2db4893c8d)] - **esm**: ensure watch mode restarts after syntax errors (Xavier Stouder) [#​61232](nodejs/node#61232) - \[[`828feb2e6b`](nodejs/node@828feb2e6b)] - **events**: remove redundant todo (Gürgün Dayıoğlu) [#​60595](nodejs/node#60595) - \[[`e91b296001`](nodejs/node@e91b296001)] - **(SEMVER-MINOR)** **fs**: add ignore option to fs.watch (Matteo Collina) [#​61433](nodejs/node#61433) - \[[`606184fae5`](nodejs/node@606184fae5)] - **fs**: remove duplicate getValidatedPath calls (Mert Can Altin) [#​61359](nodejs/node#61359) - \[[`434fcd7f8f`](nodejs/node@434fcd7f8f)] - **fs**: fix errorOnExist behavior for directory copy in fs.cp (Nicholas Paun) [#​60946](nodejs/node#60946) - \[[`bacba16f5e`](nodejs/node@bacba16f5e)] - **fs**: fix ENOTDIR in globSync when file is treated as dir (sangwook) [#​61259](nodejs/node#61259) - \[[`7697ce0310`](nodejs/node@7697ce0310)] - **fs**: remove duplicate fd validation in sync functions (Mert Can Altin) [#​61361](nodejs/node#61361) - \[[`8abd54f597`](nodejs/node@8abd54f597)] - **gyp**: aix: change gcc version detection so CXX="ccache g++" works (Stewart X Addison) [#​61464](nodejs/node#61464) - \[[`24033ee7ea`](nodejs/node@24033ee7ea)] - **http**: fix rawHeaders exceeding maxHeadersCount limit (Max Harari) [#​61285](nodejs/node#61285) - \[[`cf56327939`](nodejs/node@cf56327939)] - **http2**: validate initialWindowSize per HTTP/2 spec (Matteo Collina) [#​61402](nodejs/node#61402) - \[[`696935eeeb`](nodejs/node@696935eeeb)] - **inspector**: initial support storage inspection (Ryuhei Shima) [#​61139](nodejs/node#61139) - \[[`3d5e718e38`](nodejs/node@3d5e718e38)] - **lib**: fix typo in `util.js` comment (Taejin Kim) [#​61365](nodejs/node#61365) - \[[`f55a5fea00`](nodejs/node@f55a5fea00)] - **lib**: fix TypeScript support check in jitless mode (sangwook) [#​61382](nodejs/node#61382) - \[[`b3fbc3c375`](nodejs/node@b3fbc3c375)] - **meta**: do not fast-track npm updates (Antoine du Hamel) [#​61475](nodejs/node#61475) - \[[`2423ecdaef`](nodejs/node@2423ecdaef)] - **meta**: fix typos in issue template config (Daijiro Wachi) [#​61399](nodejs/node#61399) - \[[`e2df85a33a`](nodejs/node@e2df85a33a)] - **meta**: label v8 module MRs (René) [#​61325](nodejs/node#61325) - \[[`bc9e5f7d4d`](nodejs/node@bc9e5f7d4d)] - **node-api**: fix node\_api\_create\_object\_with\_properties name (Vladimir Morozov) [#​61319](nodejs/node#61319) - \[[`4f30c21c59`](nodejs/node@4f30c21c59)] - **node-api**: use Node-API in comments (Vladimir Morozov) [#​61320](nodejs/node#61320) - \[[`62d71eb28d`](nodejs/node@62d71eb28d)] - **quic**: copy options.certs buffer instead of detaching (Chengzhong Wu) [#​61403](nodejs/node#61403) - \[[`4bbbe75ba1`](nodejs/node@4bbbe75ba1)] - **quic**: move quic behind compile time flag (Matteo Collina) [#​61444](nodejs/node#61444) - \[[`b351910af1`](nodejs/node@b351910af1)] - **(SEMVER-MINOR)** **sea**: add --build-sea to generate SEA directly with Node.js binary (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`957292e233`](nodejs/node@957292e233)] - **(SEMVER-MINOR)** **sea**: split sea binary manipulation code (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`f289817ff8`](nodejs/node@f289817ff8)] - **(SEMVER-MINOR)** **sqlite**: enable defensive mode by default (Bart Louwers) [#​61266](nodejs/node#61266) - \[[`6442229880`](nodejs/node@6442229880)] - **sqlite**: add some tests (Guilherme Araújo) [#​61410](nodejs/node#61410) - \[[`069f3603e2`](nodejs/node@069f3603e2)] - **(SEMVER-MINOR)** **sqlite**: add sqlite prepare options args (Guilherme Araújo) [#​61311](nodejs/node#61311) - \[[`df02d00d61`](nodejs/node@df02d00d61)] - **src**: improve StringBytes::Encode perf on UTF8 (Сковорода Никита Андреевич) [#​61131](nodejs/node#61131) - \[[`e35814ba80`](nodejs/node@e35814ba80)] - **src**: add missing override specifier to Clean() (Tobias Nießen) [#​61429](nodejs/node#61429) - \[[`803ff7d3de`](nodejs/node@803ff7d3de)] - **src**: cache context lookup in vectored io loops (Mert Can Altin) [#​61387](nodejs/node#61387) - \[[`58abe99cbf`](nodejs/node@58abe99cbf)] - **src**: cache missing package.json files in the C++ package config cache (Michael Smith) [#​60425](nodejs/node#60425) - \[[`2a542094e4`](nodejs/node@2a542094e4)] - **src**: use starts\_with instead of rfind/find (Tobias Nießen) [#​61426](nodejs/node#61426) - \[[`77cacf6d9d`](nodejs/node@77cacf6d9d)] - **src**: use C++ nullptr in sqlite (Tobias Nießen) [#​61416](nodejs/node#61416) - \[[`344cc629d4`](nodejs/node@344cc629d4)] - **src**: use C++ nullptr in webstorage (Tobias Nießen) [#​61407](nodejs/node#61407) - \[[`9f25cad26c`](nodejs/node@9f25cad26c)] - **src**: fix pointer alignment (jhofstee) [#​61336](nodejs/node#61336) - \[[`5a984b9a09`](nodejs/node@5a984b9a09)] - **src**: use node- prefix on thread names (Stewart X Addison) [#​61307](nodejs/node#61307) - \[[`d4cf423a65`](nodejs/node@d4cf423a65)] - **stream**: export namespace object from internal end-of-stream module (René) [#​61455](nodejs/node#61455) - \[[`7d8232e34c`](nodejs/node@7d8232e34c)] - **test**: add some validation for JSON doc output (Antoine du Hamel) [#​61413](nodejs/node#61413) - \[[`75c06bc2a8`](nodejs/node@75c06bc2a8)] - **(SEMVER-MINOR)** **test**: migrate to --build-sea in existing SEA tests (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`cabd58f1cb`](nodejs/node@cabd58f1cb)] - **(SEMVER-MINOR)** **test**: use fixture directories for sea tests (Joyee Cheung) [#​61167](nodejs/node#61167) - \[[`bcffca8911`](nodejs/node@bcffca8911)] - **test**: aix: mark test\_threadsafe\_function/test flaky on AIX (Stewart X Addison) [#​61452](nodejs/node#61452) - \[[`29399501c1`](nodejs/node@29399501c1)] - **test**: add implicit test for fs dispose handling with using (Ilyas Shabi) [#​61140](nodejs/node#61140) - \[[`3bb481571a`](nodejs/node@3bb481571a)] - **test**: reveal wpt evaluation errors in status files (Chengzhong Wu) [#​61358](nodejs/node#61358) - \[[`a132be7f71`](nodejs/node@a132be7f71)] - **test**: check new WebCryptoAPI enum values (Filip Skokan) [#​61406](nodejs/node#61406) - \[[`72f1463735`](nodejs/node@72f1463735)] - **test**: split test-esm-loader-hooks (Joyee Cheung) [#​61374](nodejs/node#61374) - \[[`39105e4c5f`](nodejs/node@39105e4c5f)] - **test**: aix: mark test-emit-on-destroyed as flaky (Stewart X Addison) [#​61381](nodejs/node#61381) - \[[`3f17acfb1c`](nodejs/node@3f17acfb1c)] - **test**: add webidl web-platform tests (Yagiz Nizipli) [#​61316](nodejs/node#61316) - \[[`89983cf747`](nodejs/node@89983cf747)] - **test**: update url web-platform tests (Yagiz Nizipli) [#​61315](nodejs/node#61315) - \[[`73c0a242d7`](nodejs/node@73c0a242d7)] - **test**: forbid use of named imports for fixtures (Antoine du Hamel) [#​61228](nodejs/node#61228) - \[[`a49d54308e`](nodejs/node@a49d54308e)] - **test**: enforce better never-settling-promise detection (Antoine du Hamel) [#​60976](nodejs/node#60976) - \[[`335cb0b5cc`](nodejs/node@335cb0b5cc)] - **test**: ensure assertions are reached on all tests (Antoine du Hamel) [#​60845](nodejs/node#60845) - \[[`5ee02c789a`](nodejs/node@5ee02c789a)] - **test**: ensure assertions are reached on more tests (Antoine du Hamel) [#​60763](nodejs/node#60763) - \[[`141fb82ffb`](nodejs/node@141fb82ffb)] - **test**: ensure assertions are reached on more tests (Antoine du Hamel) [#​60760](nodejs/node#60760) - \[[`edf90ce457`](nodejs/node@edf90ce457)] - **test**: use `RegExp.escape` to improve test reliability (Antoine du Hamel) [#​60803](nodejs/node#60803) - \[[`f5f9b2dcf6`](nodejs/node@f5f9b2dcf6)] - **test**: ensure assertions are reached on more tests (Antoine du Hamel) [#​60728](nodejs/node#60728) - \[[`ec1cbbe0b6`](nodejs/node@ec1cbbe0b6)] - **test\_runner**: fix memory leaks in runner (Abhishek Kv. Savani) [#​60860](nodejs/node#60860) - \[[`399ac68427`](nodejs/node@399ac68427)] - **test\_runner**: fix coverage report when a directory is named file (Heath Dutton🕴️) [#​61169](nodejs/node#61169) - \[[`6e1beda333`](nodejs/node@6e1beda333)] - **test\_runner**: print info when test restarts (Xavier Stouder) [#​61160](nodejs/node#61160) - \[[`f5803ccb86`](nodejs/node@f5803ccb86)] - **test\_runner**: fix rerun ambiguous test failures (Moshe Atlow) [#​61392](nodejs/node#61392) - \[[`a5a4c3eb44`](nodejs/node@a5a4c3eb44)] - **test\_runner**: nix dead reporter code (Vas Sudanagunta) [#​59700](nodejs/node#59700) - \[[`ff1fcabfc9`](nodejs/node@ff1fcabfc9)] - **(SEMVER-MINOR)** **test\_runner**: support expecting a test-case to fail (Jacob Smith) [#​60669](nodejs/node#60669) - \[[`ade4fc2338`](nodejs/node@ade4fc2338)] - **tools**: copyedit Nix files (Antoine du Hamel) [#​61447](nodejs/node#61447) - \[[`7c2242beb9`](nodejs/node@7c2242beb9)] - **tools**: validate release commit diff as part of `lint-release-proposal` (Antoine du Hamel) [#​61440](nodejs/node#61440) - \[[`ca4ebed258`](nodejs/node@ca4ebed258)] - **tools**: use ad-hoc flag to lint Nix files (Antoine du Hamel) [#​61405](nodejs/node#61405) - \[[`05ce2c87f3`](nodejs/node@05ce2c87f3)] - **tools**: fix vcbuild lint-js-build (Vladimir Morozov) [#​61318](nodejs/node#61318) - \[[`41adb54a37`](nodejs/node@41adb54a37)] - **tools**: enforce trailing commas in `test/es-module` (Antoine du Hamel) [#​60891](nodejs/node#60891) - \[[`eebd732a52`](nodejs/node@eebd732a52)] - **tools**: enforce trailing commas in `test/sequential` (Antoine du Hamel) [#​60892](nodejs/node#60892) - \[[`8b73739e03`](nodejs/node@8b73739e03)] - **typings**: add typing for string\_decoder (Taejin Kim) [#​61368](nodejs/node#61368) - \[[`e88dd012ad`](nodejs/node@e88dd012ad)] - **v8**: changing total\_allocated\_bytes to avoid ABI changes (Caio Lima) [#​60800](nodejs/node#60800) - \[[`c75ad3d87d`](nodejs/node@c75ad3d87d)] - **v8**: add GCProfiler support for erm (Ilyas Shabi) [#​61191](nodejs/node#61191) - \[[`611c179663`](nodejs/node@611c179663)] - **zlib**: validate write\_result array length (Ryuhei Shima) [#​61342](nodejs/node#61342) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi45Mi4xMSIsInVwZGF0ZWRJblZlciI6IjQyLjkyLjExIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6bWlub3IiXX0=-->
PR-URL: #60425 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Fixes #60397
In #59888 the nearest parent package JSON cache
package_json_reader.jswas adjusted from a map from any given module path to a representation of its parentpackage.jsonfile to a map frompackage.jsonpaths to a deserialized representation of their content. This addressed excessive memory usage caused by repeatedly caching identical deserializedpackage.jsonobjects for modules that shared a parentpackage.json, but also reintroduced a filesystem traversal inpackage_json_reader.jswhich finds the nearest parentpackage.jsonfile for a given module. Thestatcalls in this traversal are not cached, so we end up potentially issuing them for a bunch of duplicate paths. In the reported issue, this leads to poor performance for users using potentially high-latency network filesystems. Similar poor performance is also observed in Node versions that lack #59086, which (re)introduced the JS-side cache initially.This PR addresses this by unwinding the changes in #59888 and instead making the C++-side
package.jsoncache a bit more expressive, caching both a deserialized representation of apackage.jsonfile at a given path, as well as an indicator if no such file exists (modeled as anstd::optional). This addresses the poor performance reported in #60397 by:statcalls inpackage_json_reader.jspackage.jsonpaths on the C++ side, which also perform poorly on high-latency filesystemsWhile analyzing the performance of these changes, I noticed a confounding factor which is that the lazy-parsing and caching of
importsandexportson deserialized package configuration objects indeserializePackageJSONwasn't working as expected and was also contributing to the varying performance we've been seeing across these changes:importsandexportsis cached on deserializedpackage.jsonobjects, it's important that a givenpackage.jsonfile map to the same deserialized object. If we don't do this, we repeatedly re-parse these fields redundantly across calls. This motivates the sort of strange two-level caching scheme ingetNearestParentPackageJSONthat these changes introduce. The downside here is that we potentially redundantly call intomodulesBinding.getNearestParentPackageJSONfor a given path just to resolve the path to apackage.jsonfile that we may already have cached, but I don't see any way to avoid this.Benchmarks
I benchmarked this change with the same scripts I used in #59888. The first is the reproduction script from #58126:
The second is this:
Each benchmark compares v22.19.0 (which does not include #59888), v25.1.0 (the latest current release, which does include #59888), and this change (which is just the
nodedirectory in the output).Fast disk
ddtrace+ CDKdate-fnsSlow disk
I emulated this by mounting an NFS volume from
localhostwithnoac(to disable most caching).ddtrace+ CDKdate-fns