While the default package manager for node is npm, two other package managers are also supported by node: yarn and pnpm.
While npm is a good option, it can be limited for projects with a lot of dependencies, or with a lot of sub modules. In particular regarding installation time and disk space usage.
pnpm dependency installation is faster than the different versions and configurations of yarn
pnpm is also less space consuming as each version is only stored once on the file system, and the actual node_modules structure is based on hard-links to the stored modules
In addition, pnpm also prevents phantom dependencies:
Phantom dependencies are dependencies that you end up being able to use without explicitly depending on them. are dependencies that you end up being able to use without explicitly depending on them.
This is interesting as phantom dependencies can lead to unexpected breakage on dependency upgrade, as transitive dependencies can be removed without notice.
In conclusion, pnpm is both more performance and safer.
Therefore, I suggest to make it our package manager.
Regarding alternative package managers, I recently read Bun hype. How we learned nothing from Yarn - DEV Community. I think it is an interesting read on the state of alternative package managers and how performance differences right now might be irrelevant in a couple of years.
What about lerna for this ? Is it at another level ?
Also I’m not sure about the space aspect. In the prototype yarn is used, and I only see space in the main node_modules directory. There is nothing in each submodule.
Concerning the speed, I’m much more concerned about the actual build time, then the dependency installation… It would be very interesting to test an alternate build than lerna/yarn on the prototype to check if there is any time won…
The main problem I’ve seen in the prototype is that it always want to rebuild, even though there are no changes to the code. I believe that would be the most important, is to only rebuild a submodule if either it has changed, or a dependency of it has changed.
It solves two of the biggest problems of JavaScript/TypeScript monorepos:
Lerna runs a command against any number of projects, and it does it in the most efficient way, in the right order, and with the possibility to distribute that on multiple machines.
Lerna manages your publishing process, from version management to publishing to NPM, and it provides a variety of options to make sure any workflow can be accommodated.
That’s a good point. I tried to build the prototype with yarn an pnpm:
pnpm install: 4.10s
yarn install: 18s
pnpm build: 1m14s
yarn build: 59.68
That where the cache task (Cache Task Results | Lerna) is useful.
I quickly added it to the yarn and pnpm version. Seems like it work well, though I’m a bit worried regarding how it can behave when the projects complexity increases (i.e., debugging build cache is a pain).
Last point. On the CI is likely that the functional test will take most of the build time pretty quickly, and no build tool can improve that.
What’s important imo is to make sure that what we have does not slow down the day to day experience of developers.
Cool… I was searching for it but could not find it. Using turbo+pnpm I was able to have the cache working also. So turbo and lerna would be at similar level. And yarn and pnpm at their level.
Good point. Functionnal tests are clearly much more time consuming. This is actually why an incremental build (even on CI) would be helpful, to only run the tests on new code and not on the full code. This could help win time knowing if a commit has created problems.
Now for devs, the normal approach is not even to use build… Running the dev server is the way to go.
The only thing is that you might want to run a build locally before committing, to verify that it passes and also run eslint. It seems that codium might not detect everything or only detect the typescript issues (or some plugins might be needed).
So for the choice yarn/pnpm. I’m +0 for the choice of pnpm. I don’t really have enough info to decide that pnpm would be much better than yarn. I think both allow to do what we need.
For lerna/turbo, it would be interesting to see if there are alternatives or if lerna is the good solution. It seems to do what we need, including caching build. But we may discover issues later.
One aspect that we need to check if the whole publication process of our first modules.
I did some more tests with nx and turbo. Both support executing builds steps based on the dependencies between modules, and provide caching mechanism to avoid re-building modules without changes.
They also both look equality well maintained, and both integrate nicely with pnpm/
But, nx is slightly older and, as of now, offers more features than turbo.
Therefore, I propose to adopt nx in addition to pnpm.