Commit 6cb9616
[utils] [perf] Performance of fullResolve
While looking at a larger code base https://gitlab.com/gitlab-org/gitlab,
I've came to realize that `fullResolve` takes a lot of CPU cycles,
particularly the `hashObject` calls inside it.
I applied the following patch locally to see how often this is called and
how many unique hashes were produced:
```diff
diff --git a/utils/resolve.js b/utils/resolve.js
index 4a35c6a..3c28324 100644
--- a/utils/resolve.js
+++ b/utils/resolve.js
@@ -83,13 +83,28 @@ function relative(modulePath, sourceFile, settings) {
return fullResolve(modulePath, sourceFile, settings).path;
}
+let prevSettings = null;
+let nonEqualSettings = 0;
+let totalCalls = 0;
+let uniqueHashes = new Set();
function fullResolve(modulePath, sourceFile, settings) {
// check if this is a bonus core module
const coreSet = new Set(settings['import/core-modules']);
if (coreSet.has(modulePath)) return { found: true, path: null };
const sourceDir = path.dirname(sourceFile);
- const cacheKey = sourceDir + hashObject(settings).digest('hex') + modulePath;
+
+ totalCalls+=1;
+ const hash = hashObject(settings).digest('hex');
+
+ if(prevSettings !== settings){
+ uniqueHashes.add(hash);
+ prevSettings = settings;
+ nonEqualSettings+=1;
+ console.log(`fullResolve | Total calls:${totalCalls} | Non-Equal settings:${nonEqualSettings} | Unique hashes:${uniqueHashes.size} | dir:${sourceDir}`)
+ }
+
+ const cacheKey = sourceDir + hash + modulePath;
const cacheSettings = ModuleCache.getSettings(settings);
```
For our code base, `fullResolve` is called more than 570 thousand times.
The simple in-equality `!==` code path is taken 1090 times.
Actually only _four_ unique hashes are produced, meaning we only have
four unique settings across our code base.
I assume that a full object equality comparison might not be needed,
and a simple object comparison with `!==` already would reduce the amount
of `hashObject` calls by 570x. This is what is implemented in this commit.
Time spend in `fullResolve` was reduced by ~38%:
- Before: 17% (19.10s) of our total execution time
- After: 11% (11.86s) of our total execution time
The effect might even be more pronounced on machines that are slower when
calculating `sha256` hashes or that have less memory, as the `hashObject`
method tends to create loads of small strings which need to be garbage
collected.1 parent d2b10ae commit 6cb9616
2 files changed
+14
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
8 | 11 | | |
9 | 12 | | |
10 | 13 | | |
| |||
123 | 126 | | |
124 | 127 | | |
125 | 128 | | |
| 129 | + | |
126 | 130 | | |
127 | 131 | | |
128 | 132 | | |
| |||
160 | 164 | | |
161 | 165 | | |
162 | 166 | | |
| 167 | + | |
163 | 168 | | |
164 | 169 | | |
165 | 170 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
| 86 | + | |
| 87 | + | |
86 | 88 | | |
87 | 89 | | |
88 | 90 | | |
89 | 91 | | |
90 | 92 | | |
91 | 93 | | |
92 | | - | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
93 | 101 | | |
94 | 102 | | |
95 | 103 | | |
| |||
0 commit comments