Skip to content

Commit 5db40fd

Browse files
aheejinaheejin
authored
[wasm-split] Make placeholder namespace contain module name (#7975)
For now we use a single namespace for all placeholder imports. This namespace can be set by `--placeholder-namespace` and if not set it is by default `placeholder`. This changes it so that the namespace is in the format of `placeholder.[secondaryName]`. That `placeholder`, or a user-specified string via `--placeholder-namespace`, is a prefix of the namespace and not the namespace itself. In `secondaryName` part we put the name of secondary modules. To be consistent with that change, I'd like to rename `--placeholder-namespace` to `--placeholder-namespace-prefix`. But in order not to break people who are using the old option, this just adds `--placeholder-namespace-prefix` and keeps `--placeholder-namespace` but make its behavior the same as `--placeholder-namespace-prefix`. This is done to make multi-split module loading work in emscripten. Currently, if the primary module name is `test.wasm` and when a placeholder function is called, the emscripten JS runtime loads `test.deferred.wasm`, which is hardcoded. But as we want emscripten to support multi-split, the runtime has to know which secondary module to load. The companion PR in emscripten (emscripten-core/emscripten#25571) makes emscripten runtime use that `secondaryName` part to figure out which secondary module file to load in case a placeholder function is called. For example, if the import module is `placeholder.2` and the import base is `3` and the primary wasm file is `test.wasm`, ```wast (import "placeholder.2" "3" (func $placeholder_3 (result i32))) ``` when `placeholder_3` function is loaded, the runtime will parse `placeholder.2` and correctly figure out that it should load `test.2.wasm`. Companion PR: emscripten-core/emscripten#25571, which has to land first.
1 parent dcc704c commit 5db40fd

20 files changed

+88
-65
lines changed

‎src/ir/module-splitting.cpp‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -772,7 +772,9 @@ void ModuleSplitter::setupTablePatching() {
772772
assert(table == tableManager.activeTable->name);
773773

774774
placeholderMap[table][index] = ref->func;
775-
Module& secondary = *secondaries.at(funcToSecondaryIndex.at(ref->func));
775+
Index secondaryIndex = funcToSecondaryIndex.at(ref->func);
776+
Module& secondary = *secondaries.at(secondaryIndex);
777+
Name secondaryName = config.secondaryNames.at(secondaryIndex);
776778
auto* secondaryFunc = secondary.getFunction(ref->func);
777779
moduleToReplacedElems[&secondary][index] = secondaryFunc;
778780
if (!config.usePlaceholders) {
@@ -782,7 +784,8 @@ void ModuleSplitter::setupTablePatching() {
782784
return;
783785
}
784786
auto placeholder = std::make_unique<Function>();
785-
placeholder->module = config.placeholderNamespace;
787+
placeholder->module = config.placeholderNamespacePrefix.toString() + "." +
788+
secondaryName.toString();
786789
placeholder->base = std::to_string(index);
787790
placeholder->name = Names::getValidFunctionName(
788791
primary, std::string("placeholder_") + placeholder->base.toString());

‎src/ir/module-splitting.h‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,16 +53,18 @@ struct Config {
5353
// may not include imported functions, which are always kept in the primary
5454
// module regardless.
5555
std::vector<std::set<Name>> secondaryFuncs;
56+
// A vector of names of the secondary modules.
57+
std::vector<Name> secondaryNames;
5658
// Whether to import placeholder functions into the primary module that will
5759
// be called when a secondary function is called before the secondary module
5860
// has been loaded.
5961
bool usePlaceholders = true;
6062
// The namespace from which to import primary functions into the secondary
6163
// module.
6264
Name importNamespace = "primary";
63-
// The namespace from which to import placeholder functions into the primary
64-
// module. Ignored if `usePlaceholders` is false.
65-
Name placeholderNamespace = "placeholder";
65+
// The prefix of the namespaces from which to import placeholder functions
66+
// into the primary module. Ignored if `usePlaceholders` is false.
67+
Name placeholderNamespacePrefix = "placeholder";
6668
// The prefix to attach to the name of any newly created exports. This can be
6769
// used to differentiate between "real" exports of the module and exports that
6870
// should only be consumed by the secondary module.

‎src/tools/wasm-split/split-options.cpp‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -236,15 +236,25 @@ WasmSplitOptions::WasmSplitOptions()
236236
[&](Options* o, const std::string& argument) {
237237
importNamespace = argument;
238238
})
239+
.add("--placeholder-namespace-prefix",
240+
"",
241+
"The prefix for the namespaces from which to import placeholder "
242+
"functions into the primary module. The namespaces will be "
243+
"concatenations of the prefix and the module name.",
244+
WasmSplitOption,
245+
{Mode::Split, Mode::MultiSplit},
246+
Options::Arguments::One,
247+
[&](Options* o, const std::string& argument) {
248+
placeholderNamespacePrefix = argument;
249+
})
239250
.add("--placeholder-namespace",
240251
"",
241-
"The namespace from which to import placeholder functions into "
242-
"the primary module.",
252+
"The same as --placeholder-namespace-prefix.",
243253
WasmSplitOption,
244254
{Mode::Split, Mode::MultiSplit},
245255
Options::Arguments::One,
246256
[&](Options* o, const std::string& argument) {
247-
placeholderNamespace = argument;
257+
placeholderNamespacePrefix = argument;
248258
})
249259
.add("--jspi",
250260
"",

‎src/tools/wasm-split/split-options.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ struct WasmSplitOptions : ToolOptions {
6767
std::string secondaryOutput;
6868

6969
std::string importNamespace;
70-
std::string placeholderNamespace;
70+
std::string placeholderNamespacePrefix;
7171
std::string secondaryMemoryName;
7272
std::string exportPrefix;
7373

‎src/tools/wasm-split/wasm-split.cpp‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -232,8 +232,8 @@ void setCommonSplitConfigs(ModuleSplitting::Config& config,
232232
if (options.exportPrefix.size()) {
233233
config.newExportPrefix = options.exportPrefix;
234234
}
235-
if (options.placeholderNamespace.size()) {
236-
config.placeholderNamespace = options.placeholderNamespace;
235+
if (options.placeholderNamespacePrefix.size()) {
236+
config.placeholderNamespacePrefix = options.placeholderNamespacePrefix;
237237
}
238238
}
239239

@@ -346,6 +346,7 @@ void splitModule(const WasmSplitOptions& options) {
346346
ModuleSplitting::Config config;
347347
setCommonSplitConfigs(config, options);
348348
config.secondaryFuncs.push_back(std::move(splitFuncs));
349+
config.secondaryNames.push_back("deferred");
349350
config.jspi = options.jspi;
350351
auto splitResults = ModuleSplitting::splitFunctions(wasm, config);
351352
auto& secondary = *splitResults.secondaries.begin();
@@ -407,7 +408,6 @@ void multiSplitModule(const WasmSplitOptions& options) {
407408

408409
std::string line;
409410
bool newSection = true;
410-
std::vector<Name> moduleNames;
411411
std::unordered_set<Name> moduleNameSet;
412412
while (std::getline(manifest, line)) {
413413
if (line.empty()) {
@@ -424,7 +424,7 @@ void multiSplitModule(const WasmSplitOptions& options) {
424424
}
425425
currModule = name;
426426
moduleNameSet.insert(currModule);
427-
moduleNames.push_back(currModule);
427+
config.secondaryNames.push_back(currModule);
428428
config.secondaryFuncs.emplace_back(std::set<Name>());
429429
currFuncs = &config.secondaryFuncs.back();
430430
newSection = false;
@@ -448,10 +448,10 @@ void multiSplitModule(const WasmSplitOptions& options) {
448448
}
449449

450450
auto splitResults = ModuleSplitting::splitFunctions(wasm, config);
451-
assert(moduleNames.size() == splitResults.secondaries.size());
452-
for (Index i = 0, n = moduleNames.size(); i < n; i++) {
451+
assert(config.secondaryNames.size() == splitResults.secondaries.size());
452+
for (Index i = 0, n = config.secondaryNames.size(); i < n; i++) {
453453
auto& secondary = *splitResults.secondaries[i];
454-
auto moduleName = options.outPrefix + moduleNames[i].toString() +
454+
auto moduleName = options.outPrefix + config.secondaryNames[i].toString() +
455455
(options.emitBinary ? ".wasm" : ".wast");
456456
if (options.symbolMap) {
457457
writeSymbolMap(secondary, moduleName + ".symbols");

‎test/example/module-splitting.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ void do_test(const std::set<Name>& keptFuncs, std::string&& module) {
5252

5353
ModuleSplitting::Config config;
5454
config.secondaryFuncs.push_back(std::move(splitFuncs));
55+
config.secondaryNames.push_back("deferred");
5556
config.newExportPrefix = "%";
5657
auto results = splitFunctions(*primary, config);
5758
auto& secondary = *results.secondaries.begin();
@@ -477,6 +478,7 @@ void test_minimized_exports() {
477478

478479
ModuleSplitting::Config config;
479480
config.secondaryFuncs.push_back({"call"});
481+
config.secondaryNames.push_back("deferred");
480482
config.newExportPrefix = "%";
481483
config.minimizeNewExportNames = true;
482484

‎test/example/module-splitting.txt‎

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -333,7 +333,7 @@ Keeping: <none>
333333
After:
334334
(module
335335
(type $0 (func (param i32) (result i32)))
336-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
336+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
337337
(table $0 1 funcref)
338338
(elem $0 (i32.const 0) $placeholder_0)
339339
(export "foo" (func $trampoline_foo))
@@ -369,7 +369,7 @@ Keeping: <none>
369369
After:
370370
(module
371371
(type $0 (func (param i32) (result i32)))
372-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
372+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
373373
(table $table 1 funcref)
374374
(table $0 1 funcref)
375375
(elem $0 (table $table) (i32.const 0) func $trampoline_foo)
@@ -407,7 +407,7 @@ Keeping: <none>
407407
After:
408408
(module
409409
(type $0 (func (param i32) (result i32)))
410-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
410+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
411411
(table $table 2 funcref)
412412
(table $0 1 funcref)
413413
(elem $0 (table $table) (i32.const 0) func $trampoline_foo $trampoline_foo)
@@ -446,7 +446,7 @@ Keeping: <none>
446446
After:
447447
(module
448448
(type $0 (func (param i32) (result i32)))
449-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
449+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
450450
(table $table 1000 funcref)
451451
(table $0 1 funcref)
452452
(elem $0 (table $table) (i32.const 42) func $trampoline_foo)
@@ -488,7 +488,7 @@ After:
488488
(module
489489
(type $0 (func (param i32) (result i32)))
490490
(import "env" "base" (global $base i32))
491-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
491+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
492492
(table $table 1000 funcref)
493493
(table $0 1 funcref)
494494
(elem $0 (table $table) (global.get $base) func $trampoline_foo)
@@ -531,7 +531,7 @@ After:
531531
(module
532532
(type $0 (func (param i32) (result i32)))
533533
(import "env" "base" (global $base i32))
534-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
534+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
535535
(table $table 1000 funcref)
536536
(table $0 1 funcref)
537537
(elem $0 (table $table) (global.get $base) func $trampoline_foo $trampoline_foo)
@@ -578,7 +578,7 @@ After:
578578
(type $0 (func))
579579
(type $1 (func (param i32) (result i32)))
580580
(import "env" "base" (global $base i32))
581-
(import "placeholder" "0" (func $placeholder_0 (type $1) (param i32) (result i32)))
581+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $1) (param i32) (result i32)))
582582
(table $table 1000 funcref)
583583
(table $0 1 funcref)
584584
(elem $0 (table $table) (global.get $base) func $null $trampoline_foo)
@@ -671,7 +671,7 @@ Keeping: foo
671671
After:
672672
(module
673673
(type $0 (func))
674-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
674+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
675675
(table $0 1 funcref)
676676
(elem $0 (i32.const 0) $placeholder_0)
677677
(export "%table" (table $0))
@@ -726,7 +726,7 @@ Keeping: foo
726726
After:
727727
(module
728728
(type $0 (func))
729-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
729+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
730730
(table $0 1 funcref)
731731
(elem $0 (i32.const 0) $placeholder_0)
732732
(export "%foo" (func $trampoline_bar))
@@ -775,8 +775,8 @@ Keeping: bar, quux
775775
After:
776776
(module
777777
(type $0 (func))
778-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
779-
(import "placeholder" "1" (func $placeholder_1 (type $0)))
778+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
779+
(import "placeholder.deferred" "1" (func $placeholder_1 (type $0)))
780780
(table $table 4 funcref)
781781
(table $0 2 funcref)
782782
(elem $0 (table $table) (i32.const 0) func $trampoline_foo $bar $trampoline_baz $quux)
@@ -838,8 +838,8 @@ After:
838838
(module
839839
(type $0 (func))
840840
(import "env" "base" (global $base i32))
841-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
842-
(import "placeholder" "1" (func $placeholder_1 (type $0)))
841+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
842+
(import "placeholder.deferred" "1" (func $placeholder_1 (type $0)))
843843
(table $table 4 funcref)
844844
(table $0 2 funcref)
845845
(elem $0 (table $table) (global.get $base) func $trampoline_foo $bar $trampoline_baz $quux)
@@ -900,9 +900,9 @@ Keeping: baz
900900
After:
901901
(module
902902
(type $0 (func))
903-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
904-
(import "placeholder" "1" (func $placeholder_1 (type $0)))
905-
(import "placeholder" "2" (func $placeholder_2 (type $0)))
903+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
904+
(import "placeholder.deferred" "1" (func $placeholder_1 (type $0)))
905+
(import "placeholder.deferred" "2" (func $placeholder_2 (type $0)))
906906
(table $table 4 funcref)
907907
(table $0 3 funcref)
908908
(elem $0 (table $table) (i32.const 0) func $trampoline_foo $trampoline_bar $baz $trampoline_quux)
@@ -969,9 +969,9 @@ After:
969969
(module
970970
(type $0 (func))
971971
(import "env" "base" (global $base i32))
972-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
973-
(import "placeholder" "1" (func $placeholder_1 (type $0)))
974-
(import "placeholder" "2" (func $placeholder_2 (type $0)))
972+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
973+
(import "placeholder.deferred" "1" (func $placeholder_1 (type $0)))
974+
(import "placeholder.deferred" "2" (func $placeholder_2 (type $0)))
975975
(table $table 4 funcref)
976976
(table $0 3 funcref)
977977
(elem $0 (table $table) (global.get $base) func $trampoline_foo $trampoline_bar $baz $trampoline_quux)
@@ -1033,7 +1033,7 @@ After:
10331033
(module
10341034
(type $0 (func))
10351035
(import "env" "base" (global $base i32))
1036-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
1036+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
10371037
(table $table 2 funcref)
10381038
(table $0 1 funcref)
10391039
(elem $0 (table $table) (global.get $base) func $foo $trampoline_bar)
@@ -1083,7 +1083,7 @@ Keeping: foo
10831083
After:
10841084
(module
10851085
(type $0 (func (param i32) (result i32)))
1086-
(import "placeholder" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
1086+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0) (param i32) (result i32)))
10871087
(table $table 1 1 funcref)
10881088
(table $0 1 funcref)
10891089
(elem $0 (table $table) (i32.const 0) func $foo)
@@ -1124,7 +1124,7 @@ Keeping: <none>
11241124
After:
11251125
(module
11261126
(type $0 (func))
1127-
(import "placeholder" "0" (func $placeholder_0 (type $0)))
1127+
(import "placeholder.deferred" "0" (func $placeholder_0 (type $0)))
11281128
(table $0 1 funcref)
11291129
(elem $0 (i32.const 0) $placeholder_0)
11301130
(export "foo1" (func $trampoline_foo))

‎test/lit/help/wasm-split.test‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,9 +84,15 @@
8484
;; CHECK-NEXT: mode, refers to the namespace from which
8585
;; CHECK-NEXT: to import the secondary memory, if any.
8686
;; CHECK-NEXT:
87-
;; CHECK-NEXT: --placeholder-namespace [split, multi-split] The namespace from
88-
;; CHECK-NEXT: which to import placeholder functions
89-
;; CHECK-NEXT: into the primary module.
87+
;; CHECK-NEXT: --placeholder-namespace-prefix [split, multi-split] The prefix for the
88+
;; CHECK-NEXT: namespaces from which to import
89+
;; CHECK-NEXT: placeholder functions into the primary
90+
;; CHECK-NEXT: module. The namespaces will be
91+
;; CHECK-NEXT: concatenations of the prefix and the
92+
;; CHECK-NEXT: module name.
93+
;; CHECK-NEXT:
94+
;; CHECK-NEXT: --placeholder-namespace [split, multi-split] The same as
95+
;; CHECK-NEXT: --placeholder-namespace-prefix.
9096
;; CHECK-NEXT:
9197
;; CHECK-NEXT: --jspi [split] Transform the module to support
9298
;; CHECK-NEXT: asynchronously loading the secondary

‎test/lit/wasm-split/basic.wast‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@
7070

7171
;; KEEP-NONE-PRIMARY: (module
7272
;; KEEP-NONE-PRIMARY-NEXT: (type $0 (func (param i32) (result i32)))
73-
;; KEEP-NONE-PRIMARY-NEXT: (import "placeholder" "0" (func $placeholder_0 (param i32) (result i32)))
73+
;; KEEP-NONE-PRIMARY-NEXT: (import "placeholder.deferred" "0" (func $placeholder_0 (param i32) (result i32)))
7474
;; KEEP-NONE-PRIMARY-NEXT: (table $table 1 1 funcref)
7575
;; KEEP-NONE-PRIMARY-NEXT: (elem $0 (i32.const 0) $placeholder_0)
7676
;; KEEP-NONE-PRIMARY-NEXT: (export "%table" (table $table))
@@ -97,7 +97,7 @@
9797

9898
;; KEEP-FOO-PRIMARY: (module
9999
;; KEEP-FOO-PRIMARY-NEXT: (type $0 (func (param i32) (result i32)))
100-
;; KEEP-FOO-PRIMARY-NEXT: (import "placeholder" "1" (func $placeholder_1 (param i32) (result i32)))
100+
;; KEEP-FOO-PRIMARY-NEXT: (import "placeholder.deferred" "1" (func $placeholder_1 (param i32) (result i32)))
101101
;; KEEP-FOO-PRIMARY-NEXT: (table $table 2 2 funcref)
102102
;; KEEP-FOO-PRIMARY-NEXT: (elem $0 (i32.const 0) $foo $placeholder_1)
103103
;; KEEP-FOO-PRIMARY-NEXT: (export "%foo" (func $foo))
@@ -127,7 +127,7 @@
127127

128128
;; KEEP-BAR-PRIMARY: (module
129129
;; KEEP-BAR-PRIMARY-NEXT: (type $0 (func (param i32) (result i32)))
130-
;; KEEP-BAR-PRIMARY-NEXT: (import "placeholder" "0" (func $placeholder_0 (param i32) (result i32)))
130+
;; KEEP-BAR-PRIMARY-NEXT: (import "placeholder.deferred" "0" (func $placeholder_0 (param i32) (result i32)))
131131
;; KEEP-BAR-PRIMARY-NEXT: (table $table 1 1 funcref)
132132
;; KEEP-BAR-PRIMARY-NEXT: (elem $0 (i32.const 0) $placeholder_0)
133133
;; KEEP-BAR-PRIMARY-NEXT: (export "%bar" (func $bar))

‎test/lit/wasm-split/jspi-secondary-export.wast‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414
;; PRIMARY: (import "env" "__load_secondary_module" (func $fimport$0))
1515

16-
;; PRIMARY: (import "placeholder" "0" (func $placeholder_0 (param i32) (result i32)))
16+
;; PRIMARY: (import "placeholder.deferred" "0" (func $placeholder_0 (param i32) (result i32)))
1717

1818
;; PRIMARY: (global $global$0 (mut i32) (i32.const 0))
1919

0 commit comments

Comments
 (0)