Skip to content

wasm: JSPI stacks are decommitted but never unmapped, VmSize/VmPTE grow without bound #67016

Open

Description

@yar2001

Version

v26.12.1 (also reproduced on v27.0.0-nightly202610263983092f63)

Platform

Linux 7.2.9+deb14-amd64 #1 SMP PREEMPT_DYNAMIC Debian 7.2.9-1 (2026-10-03) x86_64 GNU/Linux

Subsystem

wasm (deps/v8)

What steps will reproduce the bug?

Each JSPI call (WebAssembly.promising + WebAssembly.Suspending) runs on its own secondary stack. When V8 destroys such a stack instead of reusing it, the stack's address range stays mapped. VmSize and VmPTE grow with every destroyed stack and never come back. RSS stays flat.

repro.mjs, no dependencies:

import { readFileSync } from 'node:fs';

const CONCURRENCY = Number(process.env.CONCURRENCY ?? 80);
const ROUNDS = Number(process.env.ROUNDS ?? 1000);

// (module (import "env" "host" (func $host)) (func (export "run") call $host))
const bytes = new Uint8Array([
  0, 97, 115, 109, 1, 0, 0, 0, 1, 4, 1, 96, 0, 0, 2, 12, 1, 3, 101, 110, 118,
  4, 104, 111, 115, 116, 0, 0, 3, 2, 1, 0, 7, 7, 1, 3, 114, 117, 110, 0, 1, 10,
  6, 1, 4, 0, 16, 0, 11,
]);
const host = new WebAssembly.Suspending(async () => {});
const { exports } = new WebAssembly.Instance(new WebAssembly.Module(bytes), { env: { host } });
const run = WebAssembly.promising(exports.run);

function sample(label) {
  const status = readFileSync('/proc/self/status', 'utf8');
  const kb = (key) => status.match(new RegExp(`^${key}:\\s+(\\d+) kB`, 'm'))[1].padStart(9) + ' kB';
  const maps = readFileSync('/proc/self/maps', 'utf8').trimEnd().split('\n').length;
  console.log(`${label.padEnd(12)} VmSize=${kb('VmSize')}  VmPTE=${kb('VmPTE')}  VmRSS=${kb('VmRSS')}  maps=${maps}`);
}

sample('start');
for (let i = 1; i <= ROUNDS; i++) {
  await Promise.all(Array.from({ length: CONCURRENCY }, () => run()));
  if (i % (ROUNDS / 5) === 0) sample(`round ${i}`);
}
globalThis.gc?.();
sample('after gc');
node --expose-gc repro.mjs

On the nightly, the stack pool is larger since v8/v8@39ccace02f, so this single-isolate script no longer destroys stacks. The leak still shows up when an isolate goes away. repro-worker.mjs starts 200 workers one after another. Each worker runs 80 concurrent JSPI calls and then exits:

import { readFileSync } from 'node:fs';
import { Worker } from 'node:worker_threads';

const WORKERS = Number(process.env.WORKERS ?? 200);
const CONCURRENCY = Number(process.env.CONCURRENCY ?? 80);

const workerSource = `
const bytes = new Uint8Array([
  0, 97, 115, 109, 1, 0, 0, 0, 1, 4, 1, 96, 0, 0, 2, 12, 1, 3, 101, 110, 118,
  4, 104, 111, 115, 116, 0, 0, 3, 2, 1, 0, 7, 7, 1, 3, 114, 117, 110, 0, 1, 10,
  6, 1, 4, 0, 16, 0, 11,
]);
const host = new WebAssembly.Suspending(async () => {});
const { exports } = new WebAssembly.Instance(new WebAssembly.Module(bytes), { env: { host } });
const run = WebAssembly.promising(exports.run);
Promise.all(Array.from({ length: ${CONCURRENCY} }, () => run()));
`;

function sample(label) {
  const status = readFileSync('/proc/self/status', 'utf8');
  const kb = (key) => status.match(new RegExp(`^${key}:\\s+(\\d+) kB`, 'm'))[1].padStart(9) + ' kB';
  const maps = readFileSync('/proc/self/maps', 'utf8').trimEnd().split('\n').length;
  console.log(`${label.padEnd(12)} VmSize=${kb('VmSize')}  VmPTE=${kb('VmPTE')}  VmRSS=${kb('VmRSS')}  maps=${maps}`);
}

sample('start');
for (let i = 1; i <= WORKERS; i++) {
  await new Promise((resolve, reject) => {
    const worker = new Worker(workerSource, { eval: true });
    worker.on('error', reject);
    worker.on('exit', resolve);
  });
  if (i % (WORKERS / 5) === 0) sample(`worker ${i}`);
}
globalThis.gc?.();
sample('after gc');
node --expose-gc repro-worker.mjs

How often does it reproduce? Is there a required condition?

It reproduces every time on Linux x64. The only condition is that V8 destroys JSPI stacks instead of returning them to the stack pool:

  • v26.x: the pool keeps at most 4 MiB of stacks. Any workload with more than about four JSPI calls in flight destroys stacks on every round.
  • main / nightly: the pool limit is 500 MiB (--wasm-stack-pool-capacity-mb). Stacks are still destroyed when an isolate is disposed (for example, a worker exits) and when the pool goes over its limit.

What is the expected behavior? Why is that the expected behavior?

A destroyed stack should give its address range back to the OS. After warm-up, VmSize and VmPTE should stay flat, just like RSS does. Nothing references the stack memory after the stack is destroyed.

What do you see instead?

repro.mjs on v26.12.1. After 80,000 calls, VmSize grows by about 75 GB and VmPTE by about 148 MB, and GC does not reclaim either:

start        VmSize=  1418464 kB  VmPTE=      592 kB  VmRSS=    53668 kB  maps=127
round 200    VmSize= 16511480 kB  VmPTE=    30544 kB  VmRSS=    67120 kB  maps=333
round 400    VmSize= 31594232 kB  VmPTE=    60228 kB  VmRSS=    74092 kB  maps=352
round 600    VmSize= 46672632 kB  VmPTE=    89736 kB  VmRSS=    74632 kB  maps=352
round 800    VmSize= 61759868 kB  VmPTE=   119580 kB  VmRSS=    84304 kB  maps=386
round 1000   VmSize= 76839168 kB  VmPTE=   149120 kB  VmRSS=    84680 kB  maps=389
after gc     VmSize= 76838144 kB  VmPTE=   149084 kB  VmRSS=    83300 kB  maps=386

repro.mjs on v27.0.0-nightly202610263983092f63. The larger pool hides the growth here:

start        VmSize=  1424552 kB  VmPTE=      612 kB  VmRSS=    56444 kB  maps=123
round 1000   VmSize=  1539552 kB  VmPTE=     2128 kB  VmRSS=    98580 kB  maps=425
after gc     VmSize=  1538528 kB  VmPTE=     2092 kB  VmRSS=    96772 kB  maps=419

repro-worker.mjs on v27.0.0-nightly202610263983092f63. Each exited worker leaves about 79 MB of VmSize and 157 kB of VmPTE behind. v26.12.1 shows the same numbers:

start        VmSize=  1425068 kB  VmPTE=      588 kB  VmRSS=    52668 kB  maps=120
worker 40    VmSize=  4675164 kB  VmPTE=     7124 kB  VmRSS=    69016 kB  maps=167
worker 80    VmSize=  7850356 kB  VmPTE=    13376 kB  VmRSS=    70596 kB  maps=182
worker 120   VmSize= 11025816 kB  VmPTE=    19644 kB  VmRSS=    71928 kB  maps=203
worker 160   VmSize= 14200752 kB  VmPTE=    25880 kB  VmRSS=    72280 kB  maps=216
worker 200   VmSize= 17375184 kB  VmPTE=    32096 kB  VmRSS=    72740 kB  maps=227
after gc     VmSize= 17375052 kB  VmPTE=    32096 kB  VmRSS=    72620 kB  maps=226

VmPTE is kernel memory and counts against the container's memory limit. In a long-running service, this shows up as slow, unbounded memory growth that no heap snapshot explains.

Additional information

The cause looks like StackMemory::StackSegment in deps/v8/src/wasm/stacks.cc. The constructor reserves the segment plus two guard pages with AllocatePages. The destructor only calls DecommitPages(limit_ - page_size, size_ + 2 * page_size) and never calls FreePages, so the reservation stays in the address space and keeps its page tables. V8 main has the same code today.

v8/v8@39ccace02f raised the default pool limit from 4 MiB to 500 MiB and added --wasm-stack-pool-capacity-mb. That change hides the single-isolate case, but it does not fix the destructor.

Possible fixes:

  • Release the reservation in the destructor.
  • If keeping the range reserved is intentional, reuse the decommitted ranges for new stacks instead of reserving new ones.

Separately, would a backport of 39ccace02f to v26 be possible? It would remove the steady-state growth for single-isolate workloads on the upcoming LTS.

Mitigations on v26:

  • --wasm-stack-switching-stack-size=40 slows the growth to about 2.6 GB VmSize and 6 MB VmPTE per 80,000 calls, but it lowers the recursion limit.
  • Our workaround is to keep a fixed set of long-lived JSPI calls and reuse them for work items, instead of starting a new WebAssembly.promising call per item. This way stacks are never destroyed.

For comparison, a small C program that does the same mmap + mprotect + mmap(MAP_FIXED, PROT_NONE) decommit on ~1 MiB regions 80,000 times grows VmPTE from 52 kB to 155 MB. Using munmap instead keeps it at 40 kB.

Activity

  1. pratyush61 commented on Oct 26, 2026

    @pratyush61

    Hi, I'd like to work on fixing this issue by investigating StackMemory::StackSegment in deps/v8/src/wasm/stacks.cc. I'll look into safely releasing stack memory reservations and adding regression tests for repeated JSPI stack destruction and worker termination. I'd also like to explore a possible v26 backport as a separate mitigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions