Measured false-positive rates
A dependency check that flags popular, legitimate packages gets ignored. So we measure how often ours do, on the most-used packages, and publish the result, the method and the blind spots. Last runs: 8 October 2026 (all measurements).
Want these checks on every pull request and for your AI coding agents? Presend for teams is a waitlist while we test demand.
Typosquat check (npm and PyPI)
- Top 15,000 PyPI packages: 11 flagged. Each was reviewed by hand and left flagged deliberately.
- 17,338 high-impact npm packages (the npm-high-impact list): 1 flagged, reviewed and left flagged deliberately.
- PyPI packages ranked 15,001–30,000 by September 2026 downloads, never used to tune the check: 41 of 15,000 flagged (0.3%), not reviewed one by one. The two lists above are the ones the reviewed exceptions were drawn from.
- Typosquat test set (24 look-alikes of popular packages, several from real incidents such as
crossenvandpython3-dateutil): 24 of 24 detected. Corrected 6 October 2026:expres(npm) andnumpi(PyPI) are legitimate projects and were removed from the test set; all detection figures were recomputed.
The popular-package lists move. On 1 October, four packages that had recently entered the top 15,000 PyPI (djongo, tmol, orgparse, webp) were flagged; we checked each repository and added them to the reviewed list the same day.
Blind spot: only names close to a list of popular targets are checked. A name that an AI model invents and that resembles no real package (slopsquatting) is not caught by name similarity.
Against real malicious packages
Name similarity only catches malware that imitates a popular name. To see how much that is, we ran the typosquat check on the malicious packages recorded in OSV (OpenSSF malicious-packages advisories, withdrawn ones excluded), on 3 October 2026:
- PyPI: 628 of 11,776 flagged (5.3%).
- npm: 93 of 30,554 flagged (0.3%), and 302 of 191,307 (0.2%) for packages published in 2025, most with random-looking names.
Most malicious packages do not imitate a popular name. On npm, many are internal company names published for dependency confusion, or spam. On PyPI, names that add a word to a target (requests-beta) are not caught by edit distance. The flagged ones aim at the expected targets: requests, beautifulsoup4 and colorama on PyPI, electron on npm. This is why the check is a triage signal to use alongside a malware scanner, not instead of one. Details and scripts: research/malicious-packages.
Publisher-change check (npm only)
- 200 most depended-upon npm packages, evaluated at 9 dates over the last two years: the raw rule (new publisher after a dormant period) flags 13 packages; after reclassifying handovers to publishers who already maintain another package with 100,000+ weekly downloads, 1 remains (source-map). Flagged today: 1 of 200.
- The event-stream compromise, replayed at 26 November 2018: detected (last checked 26 September 2026).
Blind spots: it cannot see a hijacked account that keeps the same publisher name (as with ua-parser-js) or a malicious release by the original maintainer (as with colors.js). An attacker who has already taken over a popular package would pass the established-publisher rule. When the npm search API rate-limits us, the publisher check is reported as unavailable and the event stays flagged.
New-package signal (npm and PyPI)
A package first published less than 30 days ago is flagged new_package and the overall verdict becomes at least review_recommended: a name registered a few days ago deserves a look before it is installed. Measured on 8 October 2026, on the same lists:
- Top 15,000 PyPI packages: 9 first published less than 30 days ago (1 under 7 days, 141 under 90 days, 890 under a year).
- 17,338 high-impact npm packages: none under 30 days (236 under a year). This list favours long-established packages, so it says little about how often the signal fires on the rest of npm.
- Reference packages: tmol (first published 20 May 2026), requests (2011) and express (2010) get their exact dates.
Blind spots: on PyPI the age is that of the oldest file still on PyPI, so a project that deleted its earlier releases looks new. In our run, pikobs, an older project whose only remaining release had been uploaded the day before, was flagged for this reason. And age is no guarantee: an old package can still change hands or ship a malicious release.
Vulnerability check
Not a heuristic: it reports the OSV.dev advisories for the exact version you give. Without a version it reports advisories across all versions, which overstates the risk, so give the version you use.
Reproduce it
The code and test sets are in the public presend-source repository (Node.js, no dependencies):
node tests/typosquat/run.mjs # offline test set
node tests/typosquat/osv-malicious.mjs # 1,023 real malicious packages, offline
node tests/typosquat/top-pypi.mjs # top 15,000 PyPI (network)
node tests/typosquat/top-npm.mjs # npm-high-impact (network)
node tests/maintainer-change/top-npm.mjs 200 # about 3 minutes (network)
node tests/package-age/measure.mjs 20000 # whole lists, about 14 minutes (network)
python3 research/malicious-packages/extract.py /tmp && node research/malicious-packages/measure.mjs /tmp # about 240 MB
node research/malicious-packages/legit.mjs # PyPI ranks 15,001-30,000 (network)
The typosquat and publisher-change scripts fail if a popular package is flagged without having been reviewed; the new-package script reports the counts by age. A regression test also fails if any of the 1,023 real malicious packages the typosquat check flagged on 3 October 2026 stops being flagged.
Install commands that point to unclaimed names
A separate measurement, of documentation rather than of our checks: how many npm READMEs of MCP and AI-agent packages run npx with a command name that nobody has registered. 80 of 4,830 on 8 October 2026.
Another one, of example apps: how many public examples and templates of funded developer-tool companies resolve a dependency with a critical advisory. 168 of 517, across 12 companies, in October 2026.
What both measurements mean for teams that publish install commands and examples, with a checklist: the code your users copy.
Contributions to other tools
- NVIDIA SkillSpector, typosquat rule SC6 (PR #647, merged 28 September 2026). Measured with the project's own code: 97 high-severity false positives on the top 15,000 PyPI packages and 26 on high-impact npm, 10 of the 24 names of our typosquat test set detected. After the fix (native, no dependency on Presend): 6 and 0, 24 of 24.
- twyn (PR #540, in review). Its trusted-package lists had not been refreshed since March (PyPI) and December (npm) because the weekly update workflow failed silently. With fresh lists, false positives on top PyPI packages went from 407 to 0; on npm they went from 218 to 284 (a different source list, not investigated).
Want the same measurement for your tool?
If you maintain a dependency-security tool and want its false-positive and detection rates measured the same way, write to presendapp@gmail.com.