Measured false-positive rates

A dependency check that flags popular, legitimate packages gets ignored. So we measure how often ours do, on the most-used packages, and publish the result, the method and the blind spots. Last run: 1 October 2026.

Want these checks on every pull request and for your AI coding agents? Presend for teams is a waitlist while we test demand.

Typosquat check (npm and PyPI)

• Top 15,000 PyPI packages: 11 flagged. Each was reviewed by hand and left flagged deliberately.
• 17,338 high-impact npm packages (the npm-high-impact list): 1 flagged, reviewed and left flagged deliberately.
• Known typosquats in our test set: 27 of 27 detected.

The popular-package lists move. On 1 October, four packages that had recently entered the top 15,000 PyPI (djongo, tmol, orgparse, webp) were flagged; we checked each repository and added them to the reviewed list the same day.

Blind spot: only names close to a list of popular targets are checked. A name that an AI model invents and that resembles no real package (slopsquatting) is not caught by name similarity.

Publisher-change check (npm only)

• 200 most depended-upon npm packages, evaluated at 9 dates over the last two years: the raw rule (new publisher after a dormant period) flags 13 packages; after reclassifying handovers to publishers who already maintain another package with 100,000+ weekly downloads, 1 remains (source-map). Flagged today: 1 of 200.
• The event-stream compromise, replayed at 26 November 2018: detected (last checked 26 September 2026).

Blind spots: it cannot see a hijacked account that keeps the same publisher name (as with ua-parser-js) or a malicious release by the original maintainer (as with colors.js). An attacker who has already taken over a popular package would pass the established-publisher rule. When the npm search API rate-limits us, the publisher check is reported as unavailable and the event stays flagged.

Vulnerability check

Not a heuristic: it reports the OSV.dev advisories for the exact version you give. Without a version it reports advisories across all versions, which overstates the risk, so give the version you use.

Reproduce it

The code and test sets are in the public presend-source repository (Node.js, no dependencies):

node tests/typosquat/run.mjs                 # offline test set
node tests/typosquat/top-pypi.mjs            # top 15,000 PyPI (network)
node tests/typosquat/top-npm.mjs             # npm-high-impact (network)
node tests/maintainer-change/top-npm.mjs 200 # about 3 minutes (network)

Each script fails if a popular package is flagged without having been reviewed.

Contributions to other tools

• NVIDIA SkillSpector, typosquat rule SC6 (PR #647, merged 28 September 2026). Measured with the project's own code: 97 high-severity false positives on the top 15,000 PyPI packages and 26 on high-impact npm, 13 of 27 known typosquats detected. After the fix (native, no dependency on Presend): 6 and 0, 27 of 27.
• twyn (PR #540, in review). Its trusted-package lists had not been refreshed since March (PyPI) and December (npm) because the weekly update workflow failed silently. With fresh lists, false positives on top PyPI packages went from 407 to 0; on npm they went from 218 to 284 (a different source list, not investigated).

Want the same measurement for your tool?

If you maintain a dependency-security tool and want its false-positive and detection rates measured the same way, write to presendapp@gmail.com.