This security vulnerability was reported in October (https://hackerone.com/reports/712065) and there have been 2 PRs open for 2 months that fix this issue, but lodash hasn't had any releases for a year. There is essentially one (unpaid) person who has power to release lodash, a library that a huge majority of reasonably-sized javascript projects now depend on.
How should the open source community address essentially abandoned projects that have become critical in the ecosystem?
Other than that: I find the microdependency / mass dependency trend in NPM very irritating. As a rule of thumb I think you should be able to know at least roughly all your dependencies + transitive dependencies and take some care that they're healthy projects. If you can't name them all - you probably have too many of them.
Does npm have a facility to say a package provides another? Would npm resolve that? Or at least accept that if I have package lojack-mine and it "provides:lojack" then this dependency is resolved?
One tricky aspect of the 'metapackage' approach (where, for example, hypothetical 'lodash-legacy' or 'lodash-ng' packages could both fulfil a 'lodash' dependency slot) is that someone needs to be responsible for determining which packages can fill which slots.
In cases where the package alternatives are close forks and API-compatible, that's easier to agree on - but in a large ecosystem, the solution needs to scale in terms of human and machine review time, while also providing a good level of trust to developers when they are selecting or migrating dependencies.
There are likely to be some tricks to reduce the assessment workload required in each case -- running test suites from the 'original' package and applications which rely on it against each candidate, for example -- but ultimately either the package maintainers, the community, the NPM organization themselves -- or some mix of all of the above -- have to determine the packages that could fulfil each slot.
Good packaging is hard, and it's maintenance work rather than green-field development. We (collectively) really should incentivize and fund it more, and ideally learn from and credit existing solutions.
I'm sure there are people who enjoy reviewing (and automating review of) packages, knowing that they're improving code quality and performance for huge numbers of developers. And similarly, there are plenty of developers who provide useful feedback on package quality by reporting bugs, running benchmarks and writing qualitative comparisons.
To guess at where we may be heading in the near-term: a pure Microsoft-subsidiary ecosystem is developing in this area -- GitHub and NPM provide all the data and infrastructure to host, scan and gather user feedback for the processes mentioned above.
I think they'll collectively do a good job, although any approach does take time to build and deploy. I imagine it'll also raise longer-term questions around the governance of open source software development and what role Microsoft would like to take in that.
On a more abstract level the problem space relates to network effects and influence goals. Microsoft believes that the open source developer community is either here to stay, or is here at least for a sufficient duration of time for them to be important -- and it would like to provide influence and guide rails to that community.
Well, I want a unicorn, too. That is simply not feasible for anyone using any common javascript/node frameworks - many people's builds started failing with literally thousands of `npm audit` failures, even when their direct dependencies were relatively small.
Github/Microsoft just bought NPM, I really hope they are able to add some features that make it easier to specify different resolution algorithms when pulling dependencies so it's easy for any individual to apply a patch up-and-down the dependency chain, even if the main project hasn't been updated.
The problems that can be caught by knowing transitive dependencies but can't be caught by knowing direct dependencies, are rare.
Perhaps open projects could make a pledge of some sort that guarantees important properties as long as all dependencies also commit to the same pledge.
> The npm ecosystem needs something like the distinction between Ubuntu's "main" and "universe" repositories, so that you have a smaller subset of known-good packages with harmonized transitive dependencies, stronger centralized curation, a lot more direct scrutiny, tighter control over versioning, and some party that is responsible for addressing vulnerabilities and ensuring appropriate maintainership.
https://news.ycombinator.com/item?id=23680734
> Ruby Together is a grassroots initiative committed to supporting the critical Ruby infrastructure you rely on.
> Your financial backing allows Ruby Together to pay expert, professional developers to work on improving critical infrastructure projects.
> The Ruby Together board of directors, elected by members, selects the most impactful projects to fund.
> Ruby Together publishes monthly updates showing progress and our proven results.
It brings together companies and individuals concerned about stability and health of the ecosystem, provides a aggregated place to do high impact sponsorship of the ecosystem and the elected board then sets about making a detailed workplan and working out the best distribution of funds.
One dev on lodash? Discuss its importance; offer the dev funds to keep things up-to-date and if not them, have the resources to work out another way.
Check out their early reports to get a better idea of the initial refactoring work they did within the ecosystem.
https://rubytogether.org/
The approach used by distributions is much, much better. Having a separate team of people responsible for packaging up software is a much better approach. This would allow the security patches to get in, help enforce a sane baseline of packaging standards, prevent the introduction of malware, and have someone accountable whose interests align more with downstream than upstream.
There is a docopt-ng package here: https://github.com/bazaar-projects/docopt-ng But users haven't found it yet. And sadly, there is no guarantee that the new maintainer won't also lose interest and abandon the project.
Definitely not an ideal situation. Yes, you can fork but how do you get visibility? How can the developer announce to the community that he or she wants to maintain the software?
I suspect if a human resource audit of open source were done, there would be surprisingly few people involved in the maintenance of these things.
Stripe & Segment have their open source grants, WhatsApp Co-Founder (Jan Koum) donated $1MM USD to FreeBSD, DuckDuckGo donates to companies since 2011 (https://duckduckgo.com/donations) etc.
At the company I have founded, I am putting my money where my mouth is -- we have a budget to allocate up to $25,000 per year to open source products that we find value in. As we hit certain revenue milestones, I have every intention of donating an even larger % to funding open source initiatives as part of our "giving back" corporate social responsibility.
It is a shame most companies do not do the same. I cannot change them, but I can control change within what I do. So, thank you to all the makers and builders out there, I will make it my personal mission to figure out a way to give back to you!
(Not to mention try to get it as a taxable donation... which complicates things further)
I really appreciate you attempting to do this though, thank you!
I was in contact with these guys but nothing came of it so far ... I think they’re still working on it but I don’t know. Would be great to find out options. Thank you for your help.
Side note: lack of funding for open source is a common complaint it seems, but I was personally surprised it’s actually not trivial at all to give money open source ...
Or work with a paid version of npm if you’re app is business critical.
IMHO coordination based methods will work poorly here.
I always smile when there's npm threads like this. :)