A better npm

For the past several years I've worked on "embedded" devices (meaning things like Raspberry Pi, Beagle, Panda, etc) and doing such has given me empirical knowledge node / npm's handling of node_modules is REALLY bad.

It did solve a lot of the problems common to gem and bundler, but it introduces its own class due to the duplication of so many packages with so many files.

A Possible Solution

tree node_modules
├── awsome@5.67.10
│   ├── assets
│   ├── package.json
│   ├── index.dev.js
│   └── index.min.js
├── request@1.36.0
│   ├── package.json
│   ├── index.dev.js
│   └── index.min.js
└── new-thing@1.0.0-alpha
    ├── package.json
    ├── index.dev.js
    └── index.min.js

• No npm install. None. Ever. Automatic deps resolution.
• package.json.dependencies should be appended to automatically.
• When node boots up it should be able to look in node_modules and see the list of all 1,000 modules it has installed.
• When require runs it should be able to reference the current package.json

The API should be something like this:

npm deps ./               # statically analyze dependencies
npm dev ./                # get dev copies of dependencies

npm minify ./             # creates index.dev.js and index.min.js
npm public ./             # publishes dev.js, min.js, and assets

                          # use ferver instead of semver
npm bump feature ./       # bumps the bugfix / minor feature digit
npm bump break ./         # bumps the minor breaking / significant refactor digit
npm bump api ./           # bumps the fundamentally different api / rewrite digit

npm update-deps bugfix    # update deps
npm update-deps features

                          # update just specific packages
npm update-deps feature awesome
npm update-deps break new-thing

The difference between alpha, and beta or no suffix should be considered api-level breaking change. The difference between beta and no suffix should be considered minor breaking change

What we need to embrace as a community:

• No Singletons! (always require .create or new)
• No Dynamic requires - names.forEach(function (name) { require(name); }); is BAD!
  • it's a code smell that you should have a middleware system instead

The Problem(s)

• SUCH BAD versions. What does 0.0.5 mean? The module should be at least 1.0.0-alpha to publish.
• SO MUCH file access. Each of those fs.readFiles is a huge hit on embedded systems
• SO MUCH package.json.dependencies. It's not hard to statically determine packages required
• SO MUCH CRAP. Gigabytes of test data and framework and grunt, etc in the deps
• SO MUCH LOADING. As a design pattern, modules should be lazy loaded, not all declared at the top.
  • I've reduced app load times from 30s+ to 3s by lazy loading.
• SLOOOOOOOW. Try using npm modules on Raspberry Pi. Loading 8 copies of 10,000 individual files is NOT FAST and NOT MEMORY EFFICIENT.
• Singletons. Modules assume only one instance per process sometimes cause very bad behavior.
• COMMERCIAL packages. I love open source, but I respect capitalism. Not sure how this should work, but maybe:
  • still needs source code (so I can debug your crappy code)
  • require basic linting and security review?
  • standardized commercial licenses that protect ip, but encourage community development?

Credit where its Due

Don't get me wrong. There has been tons of love and hard work that has gone and continually goes into npm. I love it. I just have some gripes and some thoughts I wanted to jot down.

I'm looking forward to npm v3 with flat modules and hoping for further improvement in npm v4.

Buy me a coffee

(you can learn about the bigger picture I'm working towards on my patreon page )