Add support for affected_by_commits, fixed_by_commits, and OSV code fix commits

[ziadhany]

• Introduce affected_by_commits and fixed_by_commits fields in our advisory
• Update from_dict and to_dict methods
• Update compute_checksum method
• Create a CodeCommitData importer class
• Update OSV to collect code fix commits

[TG1999]

@ziadhany add description in the PR please!

[TG1999]

@ziadhany mostly looks good! Please run the importer once and paste the logs here. Thanks!

I want to see if we are missing on any data in OSV format. And how does the AdvisoryData and ImpactedPackages looks with the new CommitData. Thanks!

[ziadhany]

@TG1999 This is the log output for the following importers:

• pysec_importer_v2
• pypa_importer_v2
• oss_fuzz_importer_v2

importers_logs.zip

the database query result :
vulnerabilities_advisoryv2 Total rows: 10274
vulnerabilities_impactedpackage_fixed_by_commits Total rows: 4013
vulnerabilities_impactedpackage_affecting_commits Total rows: 3623
vulnerabilities_codecommit Total rows: 3791

[TG1999]

@ziadhany

Invalid VersionRange  for affected_pkg: {'package': {'name': 'apache-commons-io', 'ecosystem': 'OSS-Fuzz', 'purl': 'pkg:generic/apache-commons-io'}, 'ranges': [{'type': 'GIT', 'repo': 'https://github.com/apache/commons-io.git', 'events': [{'introduced': '72b1f88fb722def136ce87c9b2bfdd3c9126bb3d'}, {'fixed': 'd3e5bd6de8bc96abbadccea8b934dc038a32e90c'}]}], 'versions': ['commons-io-2.14.0-RC1', 'rel/commons-io-2.14.0'], 'ecosystem_specific': {'severity': 'LOW'}, 'database_specific': {'introduced_range': 'c511d15294d1a406a177368804014313948e2601:06fde31494c279ad940149e1a3d4944040c73c0d', 'fixed_range': '247c8e7d85a8df293011c7e9c94fd50bb2986fb7:d3e5bd6de8bc96abbadccea8b934dc038a32e90c'}} for OSV id: 'OSV-2023-962': error:InvalidVersion("'commons-io-2.14.0-RC1' is not a valid <class 'univers.versions.SemverVersion'>")
Invalid VersionRange  for affected_pkg: {'package': {'name': 'apache-commons-io', 'ecosystem': 'OSS-Fuzz', 'purl': 'pkg:generic/apache-commons-io'}, 'ranges': [{'type': 'GIT', 'repo': 'https://github.com/apache/commons-io.git', 'events': [{'introduced': '72b1f88fb722def136ce87c9b2bfdd3c9126bb3d'}, {'fixed': 'd3e5bd6de8bc96abbadccea8b934dc038a32e90c'}]}], 'versions': ['commons-io-2.14.0-RC1', 'rel/commons-io-2.14.0'], 'ecosystem_specific': {'severity': 'LOW'}, 'database_specific': {'introduced_range': 'c511d15294d1a406a177368804014313948e2601:06fde31494c279ad940149e1a3d4944040c73c0d', 'fixed_range': '247c8e7d85a8df293011c7e9c94fd50bb2986fb7:d3e5bd6de8bc96abbadccea8b934dc038a32e90c'}} for OSV id: 'OSV-2023-618': error:InvalidVersion("'commons-io-2.14.0-RC1' is not a valid <class 'univers.versions.SemverVersion'>")

Why are we getting in this logs? The commit data should have been created for this

[TG1999]

See all Invalid VersionRange errors. Why these are coming?

{'package': {'name': 'apache-commons-codec', 'ecosystem': 'OSS-Fuzz', 'purl': 'pkg:generic/apache-commons-codec'}, 'ranges': [{'type': 'GIT', 'repo': 'https://gitbox.apache.org/repos/asf/commons-codec.git', 'events': [{'introduced': '44e4c4d778c3ab87db09c00e9d1c3260fd42dad5'}, {'fixed': '3bf874e2141dc08550c0b330c7a7006f358bb0f0'}]}], 'versions': ['commons-codec-1.16.1-RC1', 'rel/commons-codec-1.16.1'], 'ecosystem_specific': {'severity': 'LOW'}, 'database_specific': {'fixed_range': '72c40fe6f62410bcaa019dbf2cb570ee4e49b70e:3bf874e2141dc08550c0b330c7a7006f358bb0f0'}} for OSV id: 'OSV-2023-1195': error:InvalidVersion("'commons-codec-1.16.1-RC1' is not a valid <class 'univers.versions.SemverVersion'>")

when we have introduced and fixed events to create code commit data.

[ziadhany]

I Updated the script to handle unsupported packages (especially for OSS-Fuzz). CodeCommit is no longer ignored even if the package is unsupported, and logs are now more meaningful.

This is the updated logs:
importers_v2.zip