CLOUDP-352308 Enhanced CLI options for robust linking (#3311)

* feat: update CLI options for improved linking process

* docs: add changelog

* docs: apply review feedback

* apply: review comments + docs update

* apply: review comments (2)

Author: nima-taheri-mongodb

.changeset/giant-phones-jog.md

@@ -0,0 +1,9 @@
+---
+'@lg-tools/link': minor
+'@lg-tools/cli': minor
+---
+
+- Added new flags to `link` script to cover broader use cases:
+  - `--no-parallel`: Run the link command sequentially for each package. Useful for when the default parallel approach fails
+  - `--launch-env`: A string of environment variable lines as `KEY=VALUE`, separated by a newline. Only the specified environment variables will be used during npm link commands in the source and destination directories. This is useful to workaround environment variable pollution by tools such as version managers (e.g., asdf) or script runners (e.g., pnpm) that override the script's environment which impacts the launched `npm link` commands. We recommend using `--launch-env="$(env)"` to use your original shell environment.
+- Improve documentation for linking in DEVELOPER.md

DEVELOPER.md

@@ -68,22 +68,54 @@ When you run the scaffold script, a `README` file will appear, which is a templa
 
 ### Locally
 
-We use @testing-library/react for writing tests locally. This library helps mock out user interactions with components. You can run all tests by running `yarn test` or turn on watch mode with `yarn test --watch`.
+We use @testing-library/react for writing tests locally. This library helps mock out user interactions with components. You can run all tests by running `pnpm test` or turn on watch mode with `pnpm test --watch`.
 
 ### Linking
 
-We also have a link script, such that you can test components that are in development in environments beyond Storybook. To do so, run `yarn run link -- [path-to-application]`.
+We provide a `link` script to help you test in-development components within environments outside of Storybook such as in your application. To do this, run:
 
-Note: There are some known issues using `yarn link` from yarn workspaces. Using Verdaccio, while more involved, is the more reliable and recommended approach for testing in an external project.
+```
+pnpm run link --to=[path-to-application]
+```
+
+The script does the following in order:
+
+- It scans the destination application for any installed `leafygreen-ui` components in its `node_modules` folder.
+  **NOTE:** If the package is new and unpublished/not installed, you will need to create a directory for the new component within the destination application inside `node_modules` before running this command.
+- If any `leafygreen-ui` components are found then:
+  - The script runs `pnpm link` in the corresponding leafygreen-ui package directory to publish a link to the package in the pnpm global registry.
+  - The script then runs `pnpm link <package-name>` in the destination application directory to install the package from the published link in the pnpm global registry.
+
+After the script completes, you can make changes directly to the component in your local `leafygreen-ui` repository. Once you do this, make sure to rebuild the component and the changes will be visible on your running application.
+
+If you encounter issues while linking, try the following any of the following flags:
+
+- When linking multiple packages with `--scope` or multiple `--packages` options, link processes run in parallel by default. If you experience failures, add the `--no-parallel` flag to run the linking tasks sequentially, which can help avoid race conditions.
+
+- If you are using a Node version manager such as `asdf` or `nvm`, add the `--launch-env="$(env)"` flag. This ensures the link script spawns commands using your current shell’s environment, preventing environment pollutions that may happen through the tooling of the version manager.
+
+- In your destination application project, make sure your module resolver picks up `'react'` from your own `node_modules` (not from LeafyGreen’s). If using webpack, you can enforce this by adding an alias in your webpack configuration:
+
+  ```js
+  resolve: {
+    alias: {
+      react: path.dirname(require.resolve('react/package.json'))
+    },
+  },
+  ```
+
+Note: There are some known issues in linking packages with `pnpm link`, if you encounter issues, try using a local registry instead. `Verdaccio` for example is a more reliable and recommended approach for testing in an external project.
 
 ### Using a local registry (Verdaccio)
 
-Publishing test versions to a local registry can be helpful when you need to make changes and test in an external app (or other library). To do this, you can install and use [Verdaccio](https://verdaccio.org/)
+Publishing test versions to a local registry can be helpful when you need to make changes and test
+in an external app (or other library). To do this, you can install and
+use [Verdaccio](https://verdaccio.org/)
 
 #### 1. Install `verdaccio`
 
 ```bash
-yarn install --global verdaccio
+pnpm install --global verdaccio
 ```
 
 #### 2. Start `verdaccio`, and make note on the localhost port (should be `http://localhost:4873/` by default)
@@ -147,23 +179,23 @@ With your local version published, open up some external app. If the app uses a
 Next, install the newly published version of your package in the external project.
 
 ```bash
-yarn install @leafygreen-ui/<package-name>
+pnpm install @leafygreen-ui/<package-name>
 ```
 
 #### 6. Publishing additional versions
 
-To publish additional versions, manually the version number in `packages/<package-name>/package.json`, and re-run step 4. Then, either manually update the external project's `package.json`, or re-run `yarn install @leafygreen-ui/<package-name>`.
+To publish additional versions, manually update the version number in `packages/<package-name>/package.json`, and re-run step 4. Then, either manually update the external project's `package.json`, or re-run `pnpm install @leafygreen-ui/<package-name>`.
 
 #### 7. Publishing to NPM
 
 If you want to stop publishing to and/or reading from your local Verdaccio server, remove the reference to the server URL in `~/.npmrc` (and the external project's local `.npmrc`/`.yarnrc`)
 
 ## Creating a new component
 
-- Run `yarn create-package <package-name>` to create a new component directory with default configurations
+- Run `pnpm create-package <package-name>` to create a new component directory with default configurations
 - Add the new component to `build.tsconfig.json`
 - If you are using any `leafygreen-ui` dependencies in your new component, add the dependency to the component directory's `tsconfig.json`.
-- Run `yarn run init` to link all packages before starting development
+- Run `pnpm run init` to link all packages before starting development
 
 ## Marking a Storybook story to be imported in mongodb.design
 

README.md

@@ -240,26 +240,7 @@ pnpm build --filter="[package]"
 
 ### Development within an Application
 
-To actively develop `leafygreen-ui` components within an application, the following script will link all `leafygreen-ui` components within your application to the local `leafygreen-ui` repository.
-
-This will allow you to make changes to your local repository of `leafygreen-ui` and see those changes immediately reflected within your running application. This allows you to develop both in isolation (within `leafygreen-ui`) and in the context of your application.
-
-To do this, clone this repository and navigate to the root directory (where `package.json` is located), then run the following:
-
-```
-pnpm run link -- ${PATH_TO_APPLICATION}
-```
-
-The script does several things in order:
-
-1. This builds every `leafygreen-ui` component so they are ready to be linked
-
-2. It scans your application for any installed `leafygreen-ui` components in your `node_modules/@leafygreen-ui` folder.
-   **NOTE:** If the package is new and unpublished/not installed, you will need to create a directory for the new component within your application inside `node_modules/@leafygreen-ui` before running this command.
-
-3. If any `leafygreen-ui` components are found then the script uses `pnpm link` to link every `node_modules/@leafygreen-ui` module to your local `leafygreen-ui` repository.
-
-After the script completes, you can make changes directly to the component in your local `leafygreen-ui` repository. Once you do this, run `pnpm build` in the root of the `leafygreen-ui` repository and the changes will be visible on your running application.
+To actively develop `leafygreen-ui` components within an application, we have a `link` script that will link all `leafygreen-ui` components within your application to the local `leafygreen-ui` repository. See the [DEVELOPER.md](./DEVELOPER.md) file for more information.
 
 ## Creating New Component
 

tools/cli/src/index.ts

@@ -83,10 +83,22 @@ cli
     'When running from a consuming application, defines the source of linked packages',
   )
   .option('-v --verbose', 'Prints additional information to the console', false)
-  .option('--scope <name>', 'The NPM organization')
+  .option(
+    '--no-parallel',
+    'Run the link command sequentially for each package. Useful for debugging or when the parallel approach fails',
+  )
+  .option(
+    '--launch-env <launchEnv>',
+    'A string of environment variable lines as `KEY=VALUE`, separated by a newline. ' +
+      'Only the specified environment variables will be used during npm link commands in the source and destination directories. ' +
+      'This is useful to workaround environment variable pollution by tools such as version managers (e.g., asdf) or script runners (e.g., pnpm) that interfere with `npm link`. ' +
+      'We recommend using --launch-env="$(env)" to use your original shell environment.',
+    undefined,
+  )
+  .option('--scope <name>', 'The NPM organization, e.g. @lg-charts')
   .option(
     '--packages <names...>',
-    'Specific package names (requires `scope` option, or full package name)',
+    'Specific package names (requires `scope` option, or full package name) e.g. `@lg-charts/core` or `core` if @lg-charts is the scope',
   )
   .action(linkPackages);
 

tools/link/src/link.ts

@@ -13,13 +13,32 @@ interface LinkOptions {
   verbose: boolean;
   to?: string;
   from?: string;
+  parallel?: boolean;
+  launchEnv?: string;
 }
 
 export async function linkPackages(
   dest: string | undefined,
   opts: LinkOptions,
 ) {
-  const { verbose, scope: scopeFlag, packages, to, from } = opts;
+  const {
+    verbose,
+    scope: scopeFlag,
+    packages,
+    to,
+    from,
+    parallel,
+    launchEnv: launchEnvString,
+  } = opts;
+  let launchEnv: NodeJS.ProcessEnv | undefined;
+
+  if (launchEnvString) {
+    const keyValuePairs = launchEnvString
+      .split('\n')
+      .filter(line => line.trim() && line.includes('='))
+      .map(line => line.trim().split(/=(.*)/).slice(0, 2));
+    launchEnv = Object.fromEntries(keyValuePairs);
+  }
 
   const rootDir = process.cwd();
 
@@ -75,13 +94,16 @@ export async function linkPackages(
   for (const [scopeName, scopePath] of Object.entries(availableScopes)) {
     if (!scopeFlag || scopeFlag.includes(scopeName)) {
       linkPromises.push(
-        linkPackagesForScope(
-          { scopeName, scopePath },
+        linkPackagesForScope({
+          scopeName,
+          scopePath,
           source,
           destination,
           packages,
           verbose,
-        ),
+          parallel,
+          launchEnv,
+        }),
       );
     }
   }

tools/link/src/utils/createLinkFrom.ts

@@ -10,6 +10,7 @@ import { PackageDetails } from './types';
 interface CreateLinkOptions extends PackageDetails {
   verbose?: boolean;
   packageManager?: SupportedPackageManager;
+  env?: NodeJS.ProcessEnv;
 }
 
 /**
@@ -24,6 +25,7 @@ export async function createLinkFrom(
     packageName,
     packageManager,
     verbose,
+    env,
   }: CreateLinkOptions,
 ): Promise<void> {
   const scopeSrc = scopePath;
@@ -44,6 +46,7 @@ export async function createLinkFrom(
         name: `link_src:${packageName}`,
         cwd: path.join(packagesDirectory, packageName),
         verbose,
+        env,
       });
     } catch (_) {
       throw new Error(`Couldn't create link for package: ${packageName}`);

tools/link/src/utils/install.ts

@@ -8,6 +8,7 @@ export async function installPackages(
   options?: {
     packageManager?: SupportedPackageManager;
     verbose?: boolean;
+    env?: NodeJS.ProcessEnv;
   },
 ): Promise<void> {
   if (fsx.existsSync(path)) {
@@ -18,6 +19,7 @@ export async function installPackages(
         name: 'install',
         cwd: path,
         verbose: options?.verbose,
+        env: options?.env,
       });
     } catch (err) {
       throw new Error(`Error installing packages\n` + err);

tools/link/src/utils/linkPackageTo.ts

@@ -8,6 +8,7 @@ interface LinkPackagesToOptions
   extends Pick<PackageDetails, 'scopeName' | 'packageName'> {
   verbose?: boolean;
   packageManager?: SupportedPackageManager;
+  env?: NodeJS.ProcessEnv;
 }
 
 /**
@@ -16,7 +17,13 @@ interface LinkPackagesToOptions
  */
 export async function linkPackageTo(
   destination: string,
-  { scopeName, packageName, verbose, packageManager }: LinkPackagesToOptions,
+  {
+    scopeName,
+    packageName,
+    verbose,
+    packageManager,
+    env,
+  }: LinkPackagesToOptions,
 ): Promise<void> {
   const fullPackageName = `${scopeName}/${packageName}`;
   // eslint-disable-next-line no-console
@@ -30,6 +37,7 @@ export async function linkPackageTo(
       name: `link_dst:${packageName}`,
       cwd: destination,
       verbose,
+      env,
     });
   } catch (_) {
     throw new Error(`Couldn't link package: ${fullPackageName}`);

tools/link/src/utils/linkPackagesForScope.spec.ts

@@ -37,14 +37,12 @@ describe('tools/link/linkPackagesForScope', () => {
     fsx.mkdirSync('./tmp/app/node_modules/@example');
     fsx.mkdirSync('./tmp/app/node_modules/@example/test-package');
 
-    await linkPackagesForScope(
-      {
-        scopeName: '@example',
-        scopePath: 'scope',
-      },
-      path.resolve('./tmp/packages'),
-      path.resolve('./tmp/app'),
-    );
+    await linkPackagesForScope({
+      scopeName: '@example',
+      scopePath: 'scope',
+      source: path.resolve('./tmp/packages'),
+      destination: path.resolve('./tmp/app'),
+    });
 
     // Creates links
     expect(spawnLoggedSpy).toHaveBeenCalledWith(