Merge pull request #20366 from michaelnebel/csharp/improvedatabasequalitydiagnostics

C#/Java/Rust: Improve database quality diagnostics query.

Author: michaelnebel

csharp/ql/integration-tests/all-platforms/standalone/DatabaseQualityDiagnostics.expected

@@ -1,6 +1,6 @@
 diagnosticAttributes
-| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityCliSummaryTable | true |
-| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityStatusPage | true |
-| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityTelemetry | true |
+| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 25 % (threshold 85 %). Percentage of expressions with known type: 58 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityCliSummaryTable | true |
+| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 25 % (threshold 85 %). Percentage of expressions with known type: 58 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityStatusPage | true |
+| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 25 % (threshold 85 %). Percentage of expressions with known type: 58 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityTelemetry | true |
 #select
-| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | 1 |
+| Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 25 % (threshold 85 %). Percentage of expressions with known type: 58 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | Scanning C# code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 25 % (threshold 85 %). Percentage of expressions with known type: 58 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning C# using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | 1 |

csharp/ql/src/Telemetry/DatabaseQualityDiagnostics.ql

@@ -8,26 +8,39 @@
 import csharp
 import DatabaseQuality
 
+private predicate diagnostic(string msg, float value, float threshold) {
+  CallTargetStatsReport::percentageOfOk(msg, value) and
+  threshold = 85
+  or
+  ExprTypeStatsReport::percentageOfOk(msg, value) and
+  threshold = 85
+}
+
 private newtype TDbQualityDiagnostic =
   TTheDbQualityDiagnostic() {
-    exists(float percentageGood |
-      CallTargetStatsReport::percentageOfOk(_, percentageGood)
-      or
-      ExprTypeStatsReport::percentageOfOk(_, percentageGood)
-    |
-      percentageGood < 95
+    exists(float percentageGood, float threshold |
+      diagnostic(_, percentageGood, threshold) and
+      percentageGood < threshold
     )
   }
 
+private string getDbHealth() {
+  result =
+    strictconcat(string msg, float value, float threshold |
+      diagnostic(msg, value, threshold)
+    |
+      msg + ": " + value.floor() + " % (threshold " + threshold.floor() + " %)", ". "
+    )
+}
+
 class DbQualityDiagnostic extends TDbQualityDiagnostic {
   string toString() {
     result =
       "Scanning C# code completed successfully, but the scan encountered issues. " +
-        "This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- "
-        +
-        "see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. "
-        +
-        "Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning C# "
+        "This may be caused by problems identifying dependencies or use of generated source code. " +
+        "Some metrics of the database quality are: " + getDbHealth() + ". " +
+        "Ideally these metrics should be above their thresholds. " +
+        "Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning C# "
         +
         "using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes)."
   }

csharp/ql/src/change-notes/2025-09-04-database-diagnostics.md

@@ -0,0 +1,4 @@
+---
+category: fix
+---
+* The message for `csharp/diagnostic/database-quality` has been updated to include detailed database health metrics. Additionally, the threshold for reporting database health issues has been lowered from 95% to 85% (if any metric falls below this percentage). These changes are visible on the tool status page.

java/ql/integration-tests/java/buildless-erroneous/DatabaseQualityDiagnostics.expected

@@ -1,6 +1,6 @@
 diagnosticAttributes
-| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityCliSummaryTable | true |
-| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityStatusPage | true |
-| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityTelemetry | true |
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 20 % (threshold 85 %). Percentage of expressions with known type: 14 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityCliSummaryTable | true |
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 20 % (threshold 85 %). Percentage of expressions with known type: 14 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityStatusPage | true |
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 20 % (threshold 85 %). Percentage of expressions with known type: 14 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | visibilityTelemetry | true |
 #select
-| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | 1 |
+| Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 20 % (threshold 85 %). Percentage of expressions with known type: 14 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | Scanning Java code completed successfully, but the scan encountered issues. This may be caused by problems identifying dependencies or use of generated source code. Some metrics of the database quality are: Percentage of calls with call target: 20 % (threshold 85 %). Percentage of expressions with known type: 14 % (threshold 85 %). Ideally these metrics should be above their thresholds. Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning Java using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes). | 1 |

java/ql/src/Telemetry/DatabaseQualityDiagnostics.ql

@@ -8,26 +8,39 @@
 import java
 import DatabaseQuality
 
+private predicate diagnostic(string msg, float value, float threshold) {
+  CallTargetStatsReport::percentageOfOk(msg, value) and
+  threshold = 85
+  or
+  ExprTypeStatsReport::percentageOfOk(msg, value) and
+  threshold = 85
+}
+
 private newtype TDbQualityDiagnostic =
   TTheDbQualityDiagnostic() {
-    exists(float percentageGood |
-      CallTargetStatsReport::percentageOfOk(_, percentageGood)
-      or
-      ExprTypeStatsReport::percentageOfOk(_, percentageGood)
-    |
-      percentageGood < 95
+    exists(float percentageGood, float threshold |
+      diagnostic(_, percentageGood, threshold) and
+      percentageGood < threshold
     )
   }
 
+private string getDbHealth() {
+  result =
+    strictconcat(string msg, float value, float threshold |
+      diagnostic(msg, value, threshold)
+    |
+      msg + ": " + value.floor() + " % (threshold " + threshold.floor() + " %)", ". "
+    )
+}
+
 class DbQualityDiagnostic extends TDbQualityDiagnostic {
   string toString() {
     result =
       "Scanning Java code completed successfully, but the scan encountered issues. " +
-        "This may be caused by problems identifying dependencies or use of generated source code, among other reasons -- "
-        +
-        "see other CodeQL diagnostics reported on the CodeQL status page for more details of possible causes. "
-        +
-        "Addressing these warnings is advisable to avoid false-positive or missing results. If they cannot be addressed, consider scanning Java "
+        "This may be caused by problems identifying dependencies or use of generated source code. " +
+        "Some metrics of the database quality are: " + getDbHealth() + ". " +
+        "Ideally these metrics should be above their thresholds. " +
+        "Addressing these issues is advisable to avoid false-positives or missing results. If they cannot be addressed, consider scanning Java "
         +
         "using either the `autobuild` or `manual` [build modes](https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#comparison-of-the-build-modes)."
   }

java/ql/src/change-notes/2025-09-04-database-diagnostics.md

@@ -0,0 +1,4 @@
+---
+category: fix
+---
+* The message for `java/diagnostic/database-quality` has been updated to include detailed database health metrics. Additionally, the threshold for reporting database health issues has been lowered from 95% to 85% (if any metric falls below this percentage). These changes are visible on the tool status page.

rust/ql/src/change-notes/2025-09-04-database-diagnostics.md

@@ -0,0 +1,4 @@
+---
+category: fix
+---
+* The message for `rust/diagnostic/database-quality` has been updated to include detailed database health metrics. These changes are visible on the tool status page.