Performance Idea

[stefanpenner]

This is a cool project!

I was curious how this project approached various scenarios, after doing so I believe I can share some past findings and alternative approaches that may be of use to you.

Specifically finding package.json's recursively across a large dep graph with duplicates can easily consume a large amount of time (in some of our applications this alone could take many minutes) (although our approach did traverse node_modules, which amplifies this problem)

The existing approach ->

backfill/packages/hasher/src/helpers.ts

Line 17 in 03b0e80

const packageRoot = await findUp("package.json", { cwd });

To mitigate this problem, some collaborators and I implemented a pretty fast approach that worked well for our use cases ultimately turning those minutes into seconds in the largest of our applications.

Our approach does the following:

• carefully avoids unnecessary IO
• added various levels of memoization, which by default mimics that of nodes require but is configurable to suit most use cases.

When profiling slow builds, we noticed many such problems and ended up extracting our approach for reuse. If you are interested, it can be found here -> https://github.com/stefanpenner/resolve-package-path