Skip to content

Create a bundle for browsers that includes the Wasm code? #408

Open

Description

@rauschma

Suggestion: source-map would be easier to use if it completely hid the fact that it internally uses Wasm.

  • For browsers, you can do that by distributing a bundle, as shown below.
  • For Node.js, this should be even simpler.

webpack.config.js:

const path = require('path');

module.exports = {
  // ···
  module: {
    rules: [
      {
        test: /\.wasm$/,
        type: 'javascript/auto',
        loader: 'arraybuffer-loader', // installed via npm
      },    
    ],
  },
};

Setup code (when used outside the library):

import arrayBuffer from 'source-map/lib/mappings.wasm';
SourceMapConsumer.initialize({
  'lib/mappings.wasm': arrayBuffer,
});

AFAICT, .initialize() is not in the .d.ts file. Maybe add it?

Activity

  1. loganfsmyth commented on Oct 9, 2019

    @loganfsmyth
    Contributor

    Hey there!

    source-map would be easier to use if it completely hid the fact that it internally uses Wasm

    I 100% agree with you there.

    Before I potentially go the wrong direction, would the module also be "completely hide" the bundle's existance? When you say "For browsers" and "For Node.js", is what you have in mind

    1. two separate entrypoints, e.g. require("source-map") and require("source-map/browser")
    2. one entrypoint with
      main: "lib/index.js",
      browser: "browser-bundle.js"
      
      so that require("source-map") would load one or the other based on context, and the bundle would be an implementation detail?
  2. rauschma commented on Oct 9, 2019

    @rauschma
    Author

    Ah, good point! Tricky to get this right. Ideas:

    • Separate entry points may make things simpler, because you don’t have to dynamically check for the current environment(?) Another possibility:
      • source-map: same as now – for people using Node.js and bundlers other than webpack
      • source-map/webpack: uses arraybuffer-loader
    • It may make sense to do (then the entry in module.exports.module.rules may not be necessary):
      import arrayBuffer from 'arraybuffer-loader!!source-map/lib/mappings.wasm';
      
  3. loganfsmyth commented on Oct 10, 2019

    @loganfsmyth
    Contributor

    I think for me, since this is published on npm primarily, it should be targeted at users consuming the library via the most common tooling first. To me, that means that it would ideally work in Node and Webpack without users needing to know that the library is in use, which would preclude needing to install arraybuffer-loader. That seems extra important for file extensions like .wasm that have potentially conflicting definitions of what it means to import them.

    That all makes me lean toward the default being a base64-encoded WASM included in the npm package by default to avoid the loader.

    We could certainly try to expose ways to configure it manually, but I think that gets complicated quickly, especially once it is used as a subdependency instead of a direct dep.

  4. rauschma commented on Oct 10, 2019

    @rauschma
    Author

    Base64, stored in a CommonJS module? Sounds good! The more transparent, the better, IMO. If it can be overridden, then it doesn’t matter if that’s a little slower. Must people will just appreciate that it’s easier to use. Bonus: it’s easier to migrate to something else in the future (to a first-class Wasm module, away from Wasm, etc.).

  5. kwonoj commented on Dec 10, 2019

    @kwonoj

    it's somewhat coincidence but #369 takes same approach for b64 encoded binary, and makes isomorphic module publishing easier.

  6. adjenks commented on Jul 28, 2020

    @adjenks
  7. puzrin commented on Sep 2, 2020

    @puzrin

    From my experience, if 25% size increase is not deadly critical, loading "small" wasm from base64 string is optimal (simple & fast enough).

  8. gruns commented on Jul 24, 2023

    @gruns

    @rauschma i just accidentally, independently reinvented your

                    test: /\.wasm$/,
                    type: 'javascript/auto',
                    use: [
                        {
                            loader: 'arraybuffer-loader',
                        },
                    ],

    technique to bundle and include mappings.wasm into a bundled .js output file. but i then hit the roadblock where the latest npm version, v0.7.4, does not support providing an ArrayBuffer to SourceMapConsumer.Initialize(), while the latest trunk does. then i Googled and found this issue

    so, in short: as a first step to support people -- like myself and @rauschma -- bundling mappings.wasm in output .js bundles, please push trunk to npm as a new version, e.g. v0.7.5. the last release, v0.7.4, was published 4 Jun 2022, over a year ago

    then, thereafter, i too wholly support bundling mappings.wasm into source-map itself, hiding its internals that wasm is involved at all. that's an implementation detail. but that's more effort than just publishing trunk to npm, so starting with the latter immediately is best 🙂

    zooming out, I'm curious: why is mappings.wasm wasm instead of js to begin with? speed? compiled from other, native code?

    cc @loganfsmyth

  9. JasonGore commented on Feb 13, 2025

    @JasonGore

    I'm hitting this now with a Node application that's using webpack and it's introducing an incredible amount of friction. This is very painful for what should be encapsulated internally. I'm still not sure how to best tackle this with the most recent version of source-map on NPM which is version 0.7.4. CopyWebpackPlugin doesn't work since source-map is using fs.readFile with an absolute path and completely sidestepping wepback. Not having any apparent access to initialize() to try and control this is adding significantly to the pain.

  10. gruns commented on Feb 14, 2025

    @gruns

    @JasonGore +1

    i just emailed some of the devs a gentle nudge to see who at mozilla is in charge of npm releases nowadays 🙂. hopefully we can get a new release soon! 👏

  11. bomsy commented on Feb 14, 2025

    @bomsy
    Contributor

    Hi, i work on devtools at mozilla. Apologies this has been a blocker. I’ll look into getting a new npm release out. thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featNew feature

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions