Package Manager choice (npm vs yarn vs pnpm)

Cristal
#1

Hello all,

While the default package manager for node is npm, two other package managers are also supported by node: yarn and pnpm.
While npm is a good option, it can be limited for projects with a lot of dependencies, or with a lot of sub modules. In particular regarding installation time and disk space usage.

I’ll base myself on this A story of how we migrated to pnpm - ‹div›RIOTS which I find is a good summary of other results I could gather.

Package Manager Without cache With cache
yarn 2 (without dedupe) 6min 31s 1min 11s
yarn 3 (without dedupe) 4min 50s 57s
yarn 3 4min 1s 50s
yarn 3 (optimized) 1min 10 45s
pnpm 58s 24s
  • pnpm dependency installation is faster than the different versions and configurations of yarn
  • pnpm is also less space consuming as each version is only stored once on the file system, and the actual node_modules structure is based on hard-links to the stored modules

In addition, pnpm also prevents phantom dependencies:

Phantom dependencies are dependencies that you end up being able to use without explicitly depending on them. are dependencies that you end up being able to use without explicitly depending on them.

This is interesting as phantom dependencies can lead to unexpected breakage on dependency upgrade, as transitive dependencies can be removed without notice.

In conclusion, pnpm is both more performance and safer.
Therefore, I suggest to make it our package manager.

1 Like
#2

Regarding alternative package managers, I recently read Bun hype. How we learned nothing from Yarn - DEV Community. I think it is an interesting read on the state of alternative package managers and how performance differences right now might be irrelevant in a couple of years.

1 Like
#3

Hi Manuel,

What about lerna for this ? Is it at another level ?

Also I’m not sure about the space aspect. In the prototype yarn is used, and I only see space in the main node_modules directory. There is nothing in each submodule.

Concerning the speed, I’m much more concerned about the actual build time, then the dependency installation… It would be very interesting to test an alternate build than lerna/yarn on the prototype to check if there is any time won…

The main problem I’ve seen in the prototype is that it always want to rebuild, even though there are no changes to the code. I believe that would be the most important, is to only rebuild a submodule if either it has changed, or a dependency of it has changed.

#4

Lerna is an additional tool to work with workspaces.

From lerna introduction:

It solves two of the biggest problems of JavaScript/TypeScript monorepos:

Lerna runs a command against any number of projects, and it does it in the most efficient way, in the right order, and with the possibility to distribute that on multiple machines.
Lerna manages your publishing process, from version management to publishing to NPM, and it provides a variety of options to make sure any workflow can be accommodated.

It can be used with pnpm: Using pnpm with Lerna | Lerna

That’s a good point. I tried to build the prototype with yarn an pnpm:

  • pnpm install: 4.10s
  • yarn install: 18s
  • pnpm build: 1m14s
  • yarn build: 59.68

That where the cache task (Cache Task Results | Lerna) is useful.
I quickly added it to the yarn and pnpm version. Seems like it work well, though I’m a bit worried regarding how it can behave when the projects complexity increases (i.e., debugging build cache is a pain).

Last point. On the CI is likely that the functional test will take most of the build time pretty quickly, and no build tool can improve that.
What’s important imo is to make sure that what we have does not slow down the day to day experience of developers.

#5

You may find this helpful (or maybe not, it was about Lerna 3.x, much must have changed): npm - Node.js multi package project? - Stack Overflow

#6

Cool… I was searching for it but could not find it. Using turbo+pnpm I was able to have the cache working also. So turbo and lerna would be at similar level. And yarn and pnpm at their level.

Good point. Functionnal tests are clearly much more time consuming. This is actually why an incremental build (even on CI) would be helpful, to only run the tests on new code and not on the full code. This could help win time knowing if a commit has created problems.

Now for devs, the normal approach is not even to use build… Running the dev server is the way to go.
The only thing is that you might want to run a build locally before committing, to verify that it passes and also run eslint. It seems that codium might not detect everything or only detect the typescript issues (or some plugins might be needed).

So for the choice yarn/pnpm. I’m +0 for the choice of pnpm. I don’t really have enough info to decide that pnpm would be much better than yarn. I think both allow to do what we need.

For lerna/turbo, it would be interesting to see if there are alternatives or if lerna is the good solution. It seems to do what we need, including caching build. But we may discover issues later.

One aspect that we need to check if the whole publication process of our first modules.

#7

I did some more tests with nx and turbo. Both support executing builds steps based on the dependencies between modules, and provide caching mechanism to avoid re-building modules without changes.
They also both look equality well maintained, and both integrate nicely with pnpm/
But, nx is slightly older and, as of now, offers more features than turbo.
Therefore, I propose to adopt nx in addition to pnpm.