Migrate npm detectors from Newtonsoft.Json to System.Text.Json

[JamieMagee]

Replace Newtonsoft.Json usage in npm detector classes with System.Text.Json, continuing the project-wide migration effort #231

Changes:

• Add PackageJson model and related types for deserializing package.json files
• Add custom JsonConverters for polymorphic fields (author, workspaces, engines)
• Refactor NpmComponentUtilities to use typed parameters instead of JProperty
• Refactor NpmLockfileDetectorBase to use JsonDocument for lockfile parsing
• Update NpmComponentDetector to use PackageJson model
• Update NpmComponentDetectorWithRoots to use PackageLockV1Dependency model
• Update NpmLockfile3Detector to use PackageLockV3Package model
• Update NpmUtilitiesTests to use new API signatures

The existing PackageLock contract models (V1, V2, V3) already used System.Text.Json, so this change leverages those models and adds the missing PackageJson model.

JsonSerializerOptions configured with AllowTrailingCommas=true to handle npm's JSON format which sometimes includes trailing commas.

[github-actions]

👋 Hi! It looks like you modified some files in the Detectors folder.
You may need to bump the detector versions if any of the following scenarios apply:

• The detector detects more or fewer components than before
• The detector generates different parent/child graph relationships than before
• The detector generates different devDependencies values than before

If none of the above scenarios apply, feel free to ignore this comment 🙂

[Copilot]

Pull request overview

This PR completes the migration of npm detectors from Newtonsoft.Json to System.Text.Json as part of project-wide migration effort #231. The changes introduce strongly-typed models for package.json deserialization and refactor all npm detector classes to use System.Text.Json APIs.

Key changes:

• Introduced PackageJson model with custom JSON converters for polymorphic fields (author, workspaces, engines)
• Refactored all npm detector base classes and implementations to use typed models instead of JToken/JProperty
• Updated utility methods to accept typed parameters instead of Newtonsoft.Json types
• Configured JsonSerializerOptions with AllowTrailingCommas=true for npm JSON compatibility

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
test/Microsoft.ComponentDetection.Orchestrator.Tests/Experiments/LinuxApplicationLayerExperimentTests.cs	Added #nullable disable directive for consistency with existing test patterns
test/Microsoft.ComponentDetection.Detectors.Tests/NpmUtilitiesTests.cs	Updated tests to use new typed API signatures; removed Newtonsoft.Json imports and JProperty-based test data
src/Microsoft.ComponentDetection.Detectors/npm/NpmLockfileDetectorBase.cs	Refactored to use JsonDocument for lockfile parsing; introduced ProcessLockfile abstract method replacing JToken-based processing
src/Microsoft.ComponentDetection.Detectors/npm/NpmLockfile3Detector.cs	Implemented new ProcessLockfile method using PackageLockV3Package models; replaced JProperty traversal with typed dictionary lookups
src/Microsoft.ComponentDetection.Detectors/npm/NpmComponentUtilities.cs	Updated GetTypedComponent to accept typed parameters; refactored TryGetAllPackageJsonDependencies to use PackageJson model
src/Microsoft.ComponentDetection.Detectors/npm/NpmComponentDetectorWithRoots.cs	Implemented new ProcessLockfile method using PackageLockV1Dependency models; replaced JProperty-based dependency traversal
src/Microsoft.ComponentDetection.Detectors/npm/NpmComponentDetector.cs	Refactored to deserialize PackageJson directly; simplified author and engines handling using typed properties
src/Microsoft.ComponentDetection.Detectors/npm/Contracts/PackageJsonWorkspacesConverter.cs	New custom converter handling workspaces as array or object with packages field
src/Microsoft.ComponentDetection.Detectors/npm/Contracts/PackageJsonEnginesConverter.cs	New custom converter handling engines as object or array (for malformed files)
src/Microsoft.ComponentDetection.Detectors/npm/Contracts/PackageJsonAuthorConverter.cs	New custom converter parsing author as string (with regex) or object
src/Microsoft.ComponentDetection.Detectors/npm/Contracts/PackageJsonAuthor.cs	New model representing package.json author field
src/Microsoft.ComponentDetection.Detectors/npm/Contracts/PackageJson.cs	New model representing package.json structure with all relevant fields

Comments suppressed due to low confidence (1)

src/Microsoft.ComponentDetection.Detectors/npm/NpmComponentUtilities.cs:76

• These 'if' statements can be combined.

        if (versionString is not null && Uri.TryCreate(versionString, UriKind.Absolute, out var parsedUri))
        {
            if (string.Equals(npmRegistryHost, parsedUri.Host, StringComparison.OrdinalIgnoreCase))
            {
                return TryParseNpmRegistryVersion(packageName, parsedUri, out version);
            }
        }

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/Microsoft.ComponentDetection.Detectors/npm/NpmComponentUtilities.cs:76

• These 'if' statements can be combined.

        if (versionString is not null && Uri.TryCreate(versionString, UriKind.Absolute, out var parsedUri))
        {
            if (string.Equals(npmRegistryHost, parsedUri.Host, StringComparison.OrdinalIgnoreCase))
            {
                return TryParseNpmRegistryVersion(packageName, parsedUri, out version);
            }
        }

[Copilot]

The URL is not being extracted from the author string format. The regex pattern captures the URL in a group, but the converter only extracts name and email (lines 45-56). When parsing strings like "John Doe (https://example.com)" or "John Doe <john@example.com> (https://example.com)", the URL should be extracted from the third capture group and assigned to the Url property.

Example fix:

var name = match.Groups["name"].Value.Trim();
var email = match.Groups["email"].Value.Trim();
var url = match.Groups[3].Value.Trim(); // Extract URL from third group

return new PackageJsonAuthor
{
    Name = name,
    Email = string.IsNullOrEmpty(email) ? null : email,
    Url = string.IsNullOrEmpty(url) ? null : url,
};

Note: The corresponding tests at lines 47-57 and 60-70 are also missing assertions for the URL, which is why this bug was not caught.

[Copilot]

This test is missing an assertion for the URL. When parsing "John Doe <john@example.com> (https://example.com)", the URL should be extracted and verified. Add:

result.Author.Url.Should().Be("https://example.com");

Note: This test failure would have caught the bug in PackageJsonAuthorConverter.cs where the URL is not being extracted from the string format.

[Copilot]

This test is missing an assertion for the URL. When parsing "John Doe (https://example.com)", the URL should be extracted and verified. Add:

result.Author.Url.Should().Be("https://example.com");