npm packages can import and still fail: a 24-package behavior study

By CompatLab ·

Four packages imported on Bun 1.4.2, then failed our named behavior probes. Node.js 26.10.0 and Deno 2.9.7 passed all 19 named probes; Bun passed 15. The four differences reproduced on Bun 1.4.4. These are selected API cases, not an npm-wide compatibility score.

Our first study of 100 npm versions found that 97 passed root import on all four tested profiles. Loading was the boundary of that measurement. This follow-up calls package APIs and runs upstream tests against 24 deliberately selected versions.

These cases target native addons and Node-specific APIs. They do not estimate the failure rate of npm, Bun, or Deno as a whole.

What differed after loading

An identical native binary worked after a filename change

Bun PR #47648, opened November 1, 2026, already documented this filename check and proposed a fix before our study. Our matched-binary checks independently reproduced it: better-sqlite3@13.2.0 imported successfully, but opening a database through our from-source binding failed on Bun 1.4.2 and 1.4.4. The shipped prebuild passed. A byte-identical copy of the rebuilt binary also passed when selected under a different filename. Node and Deno passed all three variants. As checked November 16, 2026, the earlier PR remained open and unmerged.

bun run --no-install /pilot/probe-entry.cjs /pilot/better-sqlite3.cjs

Evidence and upstream context

A worker could start without receiving its task

Workerpool@10.0.3 loaded on all three primary profiles. Its thread task completed on Node and Deno but timed out at 75 seconds on Bun 1.4.2 and 1.4.4, including fresh replays. A package-free reduction selected the web message-listener branch because those globals exist, then received no echo within eight seconds on both Bun versions. This matches the message-channel mismatch already reported upstream; Piscina and Thread-stream passed their separate named cases.

bun run --no-install /pilot/probe-entry.cjs /pilot/worker-channel.cjs

Evidence and upstream context

Importing a loader utility did not exercise its loader hook

Import-fresh@4.0.1 imported on Bun, then threw when the named case created its fresh-import loader: module.registerHooks was unavailable. The same cache-reset case passed on Node and Deno. Both tested Bun versions reproduced the gap. This is a known Node API omission, not a newly discovered import-fresh defect.

bun run --no-install /pilot/probe-entry.cjs /pilot/remaining.cjs import-fresh

Evidence and upstream context

A timer lost the namespace value after a successful import

CLS-hooked@4.2.2 loaded on all primary profiles. In two concurrent namespace runs, Bun 1.4.2 and 1.4.4 returned undefined where the timer callback should have received its stored value. Node and Deno passed this narrow case. Bun documents async_hooks limitations. The broader CLS suite also failed on Node and Deno, so this passing timer probe cannot certify HTTP or TCP context propagation.

bun run --no-install /pilot/probe-entry.cjs /pilot/remaining.cjs cls-hooked

Evidence and upstream context

Every frozen package is accounted for

All 24 frozen versions are included: 19 received named behavior probes and nine received upstream suite or subset attempts, with four packages in both groups. The final run contains 189 runtime/case observations, including separate root imports, supplementary runtime versions and matched controls. Preparation succeeded for the shared tree and all four recorded offline hook/build steps.

The table shows each package’s primary behavior case, or its upstream suite when no separate named case was used. A pass covers that case only. Baseline failure means the matching Node test did not pass, so that row cannot establish an alternative-runtime regression.

Linux amd64, Node.js 26.10.0, Bun 1.4.2, Deno 2.9.7
PackageCaseNode.jsBunDeno
sharp@0.35.5Resize a red pixel; verify dimensions and bytesPassPassPass
better-sqlite3@13.2.0Insert/select through a from-source native bindingPassDifference: native binding rejectedPass
bcrypt@6.0.0Hash and verify, including a wrong passwordPassPassPass
canvas@3.2.3Render red and transparent pixels; read them backPassPassPass
esbuild@0.28.2Transform TypeScript and evaluate the resultPassPassPass
piscina@5.3.2Worker arithmetic and child runtime identityPassPassPass
thread-stream@4.2.1Worker writes a file; await close and read backPassPassPass
workerpool@10.0.3Thread task, runtime identity and pool teardownPassTimeout: thread task (75 s)Pass
http2-wrapper@2.2.4Local TLS HTTP/2 request and responsePassPassPass
spdy@4.0.2Upstream client and server testsBaseline failure: stops before testsBaseline failure: stops before testsNo Node baseline; 0/11 tests pass
undici@8.12.4Local HTTP and Agent.close using the pinned npm entryPassPassPass
require-from-string@2.0.2Upstream seven-test suiteBaseline failure: 5/7 tests passBaseline failure: 5/7 tests passBaseline failure: 5/7 tests pass
pirates@4.0.7Install a require hook, load, remove and reloadPassPassPass
import-fresh@4.0.1Repeated ESM imports reset module-local statePassDifference: loader hook unavailablePass
cls-hooked@4.2.2Concurrent timer namespace isolationPassDifference: timer context lostPass
async-hook-domain@4.0.1Catch a timer exception in its domainPassPassPass
cross-spawn@7.0.6Sync/async child output, exit and runtime identityPassPassPass
execa@10.2.1Successful child and nonzero exit rejectionPassPassPass
foreground-child@4.0.3Child exit callback, output and runtime identityPassPassPass
signal-exit@4.1.0Normal child-exit callback (not OS signal delivery)PassPassPass
readable-stream@4.8.0Object stream backpressure and error propagationPassPassPass
pump@3.0.4Both shipped JavaScript test scriptsPassPassPass
split2@4.2.0Shipped unit assertions (no lint or coverage)Pass (59; 0 skipped)Pass (59; 0 skipped)Pass (59; 0 skipped)
end-of-stream@1.4.5Pinned upstream test assertionsPass (18; 0 skipped)Pass (18; 0 skipped)Pass (18; 0 skipped)

Separate root imports passed for 23 of 24 versions on Node, 23 of 24 on Bun, and 24 of 24 on Deno. SPDY failed to import on Node and Bun. Bun's ordinary undici specifier resolves to its built-in shim, so that root-import result describes the consumer resolution, not the installed npm artifact.

What the upstream suites actually covered

We attempted JavaScript suites for nine versions. Eight use the listed JavaScript unit suites; CLS-hooked uses an explicitly bounded subset. Lint, type checks, in-browser execution, coverage instrumentation and upstream release builds are outside this tier. Counts are test cases for Jest/Mocha/node:test and TAP assertions for Split2/End-of-stream; they are not comparable between packages. Mocha uses --exit after its assertions, so suite completion does not establish a leak-free process lifetime.

Suite completion and coverage, separate from the named probes
PackageNode.jsBunDenoScope or limitation
pumpPassPassPassBoth test-browser.js and test-node.js under each runtime; no real browser. Two completion markers, not an assertion count.
split2Pass (59; 0 skipped)Pass (59; 0 skipped)Pass (59; 0 skipped)59 TAP assertions from the shipped unit file; lint and coverage excluded.
end-of-streamPass (18; 0 skipped)Pass (18; 0 skipped)Pass (18; 0 skipped)18 TAP assertions from the pinned source test, using Tape 4.
bcryptPass (75; 0 skipped)Pass (75; 0 skipped)Pass (75; 0 skipped)75 Jest tests, run in-band against a copy of the published implementation outside node_modules.
require-from-stringBaseline failure: 5/7 tests passBaseline failure: 5/7 tests passBaseline failure: 5/7 tests passTwo old error-message/stack-format expectations fail on all three profiles; no passing Node baseline.
spdyBaseline failure: stops before testsBaseline failure: stops before testsNo Node baseline; 0/11 tests passNode 24/26 and Bun stop before tests. Deno reaches the suite but fails; no passing Node baseline.
workerpoolBaseline failure: 169/172 pass; 2 skippedBaseline failure: 84/101 pass; 1 skippedBaseline failure: 99/128 pass; 2 skippedAll top-level JavaScript test files from the pinned source. Node baseline fails; other profiles also differ in test coverage and child-process assumptions.
import-freshPass (69; 0 skipped)Difference: loader hook unavailableTimeout: 2 tests completed (180 s)Both pinned unit files, copied outside node_modules. Node and Deno use native test runners; Bun lacks module.registerHooks.
cls-hookedBaseline failure: 25/27 tests passBaseline failure: 10/23 tests passBaseline failure: 19/27 tests passEight Mocha files. Public-Google-dependent http-agent-break.test.js and the separately configured TAP suite excluded before execution. Node baseline also fails.

A failed upstream suite is not automatically a runtime defect. Require-from-string relies on historical error formatting; SPDY depends on private bindings removed from current Node. Workerpool and CLS-hooked also have failing Node baselines and different failures or coverage on other runtimes. Their complete structured errors and fresh replays remain in the raw data. Compare individual cases only when their prerequisites and test coverage match. Import-fresh completes 69 tests on Node, stops before registration on Bun, and stalls twice on Deno after two completed tests, before its top-level-await case completes. Those are the original unmodified-suite observations. The dated Deno follow-up below isolates the stall and reports an adapted suite separately. Mocha failure totals can include hooks in addition to test cases, and Workerpool's Deno failure count changes between replay attempts.

Deno follow-up: completing the Import-fresh suite

November 16, 2026. A package-free reproduction isolates a cross-test async-context leak after a dynamic import on Deno 2.9.7 and 2.9.8. Binding test callbacks to their registration context removes the Import-fresh deadlock: all 69 tests finish in two fresh attempts on each profile. Node passes 69; each Deno version passes 65 and fails four. The four failures also reproduce individually without the adapter. This is an adapted-suite result, not a passing unmodified suite or a change to the original 189 observations.

Read the follow-up protocol, controls and raw receipts or the upstream context-leak report.

What the four failures mean

The four retained assertions reduce to three causes. Independent checks distinguish a spy that cannot observe calls, lost error codes, and an attribute that is discarded before validation. The 65-pass/four-failure suite result is unchanged.

  • Registration spy: export synchronization. Deno leaves the named export unchanged after syncBuiltinESMExports(), so the spy cannot observe registration. A separately instrumented copy records exactly one registration; the original spy assertion still fails. Upstream reproduction.
  • Two type attributes: error-code loss. Both calls reject before the CommonJS target executes. Deno replaces the thrown hook error code with ERR_MODULE_NOT_FOUND, so these are error-code mismatches, not missing files or observed target execution. Upstream reproduction.
  • Unknown attribute: validation bypassed. With hooks active, Deno drops the unknown dynamic-import attribute before the package can validate it, and the target executes once. Without hooks, Deno correctly rejects it. Earlier Deno work explicitly covered static imports only. Upstream reproduction.

The assertion-triage protocol and raw receipts retain the controls, prior-work references and instrumented-copy limitation. These reports are awaiting upstream fixes; no Deno runtime fix is claimed here.

Method and limits

  • Selection. All 24 artifacts were frozen before the pilot. The list includes exact versions, npm integrity and source commits where npm supplies them. No unsuccessful package was removed.
  • Preparation. One locked npm install with scripts disabled prepared the shared dependency tree. Separate offline gVisor containers ran the recorded bcrypt, esbuild, canvas and better-sqlite3 steps. This measures shared Node/npm preparation, not three independent package-manager installs.
  • Environment. Runs used pinned Linux amd64 Debian Bookworm images, gVisor, non-root users, resource limits and no external network during execution. Local loopback servers were allowed. Deno used full permissions with offline resolution. The raw receipts contain run dates, versions, image IDs, compiler details and exact commands.
  • Completion. Named probes require zero exit and the expected completion file after their assertions. Asynchronous probes await their work. Children in the named subprocess and worker cases report their runtime identity. Suite completion and assertion/skip counts are checked separately.
  • Failure checks. Every non-pass was replayed in a fresh container. Reduced cases and matched controls separate runtime API gaps, baseline failures, test-harness assumptions, timeouts and missing prerequisites. Known upstream issues are linked; an open fix is not a released fix.
  • Scope. These results describe the listed cases, exact versions and this OS sandbox. They do not certify whole applications, all APIs, all platforms, security or performance. Some suites assume Node-specific child arguments or executable names. Their failures need interpretation before attributing them to a runtime.

The audit corrected measurement mistakes before publication: Bun's bare undici shim lacked Agent.close while the explicitly pinned npm package passed; Import-fresh fixtures had to move outside node_modules; Deno needed deno test to execute node:test; asynchronous probes needed awaited completion records; and Mocha failures needed structured evidence beyond bounded stdout. Superseded receipts are retained with explanations. A failed first command was injected into Pump's shell sequence and propagated exit 23; no false Pump pass was observed. The separate Yargs diagnostic returned the same ERR_MODULE_NOT_FOUND code in all six fresh/order controls, so the earlier inconsistent-code suspicion did not reproduce.

Public CompatLab reports continue to describe loading observations. The research does not turn a loading badge into a behavior certificate. For an application migration, use the dependency checklist and test the calls your code actually makes.

Raw results and reproduction

Commands above run inside the prepared study container, from /study. Follow the pinned runner and protocol to prepare the same inputs on a disposable Linux amd64 host with gVisor. The public hosted scanner does not run these custom tests.