Convenient handling of DBMS notifications (#1059)

This PR introduces support for a more convenient way of consuming notifications
received from the DBMS server.

This feature is still in **PREVIEW**.
It might be changed without following the deprecation policy.

## Logging
A new sub-logger `neo4j.notifications` is introduced.
Every notification received will be logged through this logger.
The log-level is determined by the notification's severity.

If logging is not configured explicitly, warnings are logged to stderr.
This means that, by default, warnings like deprecations received from the DBMS
will appear on stderr.

## Warnings
A new driver-level configuration `warn_notification_severity` is introduced.
It can be used to configure from which notification severity level upward the
driver should emit a warning (i.e., call `warnings.warn`).

By default (`None`), it will be set to `OFF` (never emitting warnings on
notifications), unless Python runs in development mode or the environment
variable `PYTHONNEO4JDEBUG` is set, in which case the driver will emit a
warning on every notification.

## Usage
This functionality if mainly meant for developing and debugging.
Therefore, no emphasis is put on efficiency.
However, it's impact is capt to a minimum when disabled.
It's assumed that in productions environments this feature is either disabled
explicitly

```python
neo4j.GraphDatabase.driver(
    ...,
    warn_notification_severity=neo4j.NotificationMinimumSeverity.OFF,
    ...,
)
```

or default behavior (see above) is used by not running Python in development mode.

## Example
```python
# example.py
import neo4j


URL = "neo4j://localhost:7687"
AUTH = ("neo4j", "pass")
DB = "neo4j"


with neo4j.GraphDatabase.driver(
    URL,
    auth=AUTH,
    warn_notification_severity=neo4j.NotificationMinimumSeverity.INFORMATION,
) as driver:
    driver.execute_query(
        "MERGE (:Test {name: 'foo'})",
        database_=DB,
    )
    driver.execute_query(
        "MATCH (n:Test {name: 'baz'}), (m:Test1 {name: 'bar'}) "
        "RETURN n, m",
        database_=DB,
        routing_="r",
    )
```

On an empty database, this leads to the following output:

```
$ python example.py
/path/to/example.py:10: PreviewWarning: notification warnings are a preview feature. It might be changed without following the deprecation policy. See also https://github.com/neo4j/neo4j-python-driver/wiki/preview-features.
  with neo4j.GraphDatabase.driver(
/path/to/example.py:19: Neo4jWarning: {severity: INFORMATION} {code: Neo.ClientNotification.Statement.CartesianProduct} {category: PERFORMANCE} {title: This query builds a cartesian product between disconnected patterns.} {description: If a part of a query contains multiple disconnected patterns, this will build a cartesian product between all those parts. This may produce a large amount of data and slow down query processing. While occasionally intended, it may often be possible to reformulate the query that avoids the use of this cross product, perhaps by adding a relationship between the different parts or by using OPTIONAL MATCH (identifier is: (m))} {position: line: 1, column: 1, offset: 0} for query:
MATCH (n:Test {name: 'baz'}), (m:Test1 {name: 'bar'}) RETURN n, m
^
  driver.execute_query(
Received notification from DBMS server: {severity: WARNING} {code: Neo.ClientNotification.Statement.UnknownLabelWarning} {category: UNRECOGNIZED} {title: The provided label is not in the database.} {description: One of the labels in your query is not available in the database, make sure you didn't misspell it or that the label is available when you run this statement in your application (the missing label name is: Test1)} {position: line: 1, column: 34, offset: 33} for query: "MATCH (n:Test {name: 'baz'}), (m:Test1 {name: 'bar'}) RETURN n, m"
/path/to/example.py:19: Neo4jWarning: {severity: WARNING} {code: Neo.ClientNotification.Statement.UnknownLabelWarning} {category: UNRECOGNIZED} {title: The provided label is not in the database.} {description: One of the labels in your query is not available in the database, make sure you didn't misspell it or that the label is available when you run this statement in your application (the missing label name is: Test1)} {position: line: 1, column: 34, offset: 33} for query:
MATCH (n:Test {name: 'baz'}), (m:Test1 {name: 'bar'}) RETURN n, m
                                 ^
  driver.execute_query(
```

Author: robsdedude

docs/source/api.rst

@@ -407,6 +407,7 @@ Additional configuration can be provided via the :class:`neo4j.Driver` construct
 + :ref:`user-agent-ref`
 + :ref:`driver-notifications-min-severity-ref`
 + :ref:`driver-notifications-disabled-categories-ref`
++ :ref:`driver-warn-notification-severity-ref`
 + :ref:`telemetry-disabled-ref`
 
 
@@ -725,6 +726,28 @@ Notifications are available via :attr:`.ResultSummary.notifications` and :attr:`
 .. seealso:: :class:`.NotificationDisabledCategory`, session config :ref:`session-notifications-disabled-categories-ref`
 
 
+.. _driver-warn-notification-severity-ref:
+
+``warn_notification_severity``
+------------------------------
+Set the minimum severity for server notifications that should cause the driver to emit a :class:`.Neo4jWarning`.
+
+Setting it to :attr:`.NotificationMinimumSeverity.OFF` disables these kind of warnings.
+Setting it to :data:`None` will be equivalent to ``OFF``, unless Python runs in development mode
+(e.g., with ``python -X dev ...``) or the environment variable ``PYTHONNEO4JDEBUG`` is set, in which case the driver
+defaults to emitting warnings on all notification (currently equivalent to :attr:`.NotificationMinimumSeverity.INFORMATION`).
+
+**This is experimental** (see :ref:`filter-warnings-ref`).
+It might be changed or removed any time even without prior notice.
+
+:Type: :data:`None`, :class:`.NotificationMinimumSeverity`, or :class:`str`
+:Default: :data:`None`
+
+.. versionadded:: 5.21
+
+.. seealso:: :ref:`development-environment-ref`
+
+
 .. _telemetry-disabled-ref:
 
 ``telemetry_disabled``
@@ -2085,6 +2108,14 @@ The Python Driver uses the built-in :class:`python:ResourceWarning` class to war
 
 .. autoclass:: neo4j.ExperimentalWarning
 
+.. autoclass:: neo4j.warnings.Neo4jWarning
+    :show-inheritance:
+    :members:
+
+.. autoclass:: neo4j.warnings.Neo4jDeprecationWarning
+    :show-inheritance:
+    :members:
+
 
 .. _filter-warnings-ref:
 
@@ -2156,6 +2187,9 @@ Currently available:
   * ``neo4j.pool``: Logs connection pool activity (including routing).
   * ``neo4j.auth_management``: Logger for provided :class:`.AuthManager`
     implementations.
+  * ``neo4j.notifications``: Logs notifications received from the server.
+    The notifications' :attr:`.SummaryNotification.severity_level` is used to
+    determine the log level.
 
 There are different ways of enabling logging as listed below.
 

docs/source/index.rst

@@ -99,6 +99,8 @@ To deactivate the current active virtual environment, use:
     deactivate
 
 
+.. _development-environment-ref:
+
 Development Environment
 =======================
 
@@ -112,9 +114,14 @@ Specifically for this driver, this will:
    * **This is experimental**.
      It might be changed or removed any time even without prior notice.
    * the driver will raise an exception if non-concurrency-safe methods are used concurrently.
+   * the driver will emit warnings if the server sends back notification
+     (see also :ref:`driver-warn-notification-severity-ref`).
 
      .. versionadded:: 5.15
 
+     .. versionchanged:: 5.21
+         Added functionality to automatically emit warnings on server notifications.
+
 .. _development mode: https://docs.python.org/3/library/devmode.html
 
 

src/neo4j/_async/_debug/__init__.py

@@ -17,4 +17,6 @@
 from ._concurrency_check import AsyncNonConcurrentMethodChecker
 
 
-__all__ = ["AsyncNonConcurrentMethodChecker"]
+__all__ = [
+    "AsyncNonConcurrentMethodChecker",
+]

src/neo4j/_async/_debug/_concurrency_check.py

@@ -17,8 +17,6 @@
 from __future__ import annotations
 
 import inspect
-import os
-import sys
 import traceback
 import typing as t
 from copy import deepcopy
@@ -29,6 +27,7 @@
     AsyncRLock,
 )
 from ..._async_compat.util import AsyncUtil
+from ..._debug import ENABLED
 from ..._meta import copy_signature
 
 
@@ -37,9 +36,6 @@
                           bound=t.Callable[..., t.AsyncIterator])
 
 
-ENABLED = sys.flags.dev_mode or bool(os.getenv("PYTHONNEO4JDEBUG"))
-
-
 class NonConcurrentMethodError(RuntimeError):
     pass
 

src/neo4j/_async/driver.py

@@ -30,6 +30,7 @@
     )
 
 from .._api import (
+    NotificationMinimumSeverity,
     RoutingControl,
     TelemetryAPI,
 )
@@ -42,6 +43,7 @@
     TrustStore,
     WorkspaceConfig,
 )
+from .._debug import ENABLED as DEBUG_ENABLED
 from .._meta import (
     deprecation_warn,
     experimental_warn,
@@ -157,6 +159,9 @@ def driver(
             notifications_disabled_categories: t.Optional[
                 t.Iterable[T_NotificationDisabledCategory]
             ] = ...,
+            warn_notification_severity: t.Optional[
+                T_NotificationMinimumSeverity
+            ] = ...,
             telemetry_disabled: bool = ...,
 
             # undocumented/unsupported options
@@ -270,11 +275,14 @@ def driver(
             elif security_type == SECURITY_TYPE_SELF_SIGNED_CERTIFICATE:
                 config["encrypted"] = True
                 config["trusted_certificates"] = TrustAll()
-            _normalize_notifications_config(config)
+            if "warn_notification_severity" in config:
+                preview_warn("notification warnings are a preview feature.",
+                             stack_level=2)
+            _normalize_notifications_config(config, driver_level=True)
             liveness_check_timeout = config.get("liveness_check_timeout")
             if (
-                    liveness_check_timeout is not None
-                    and liveness_check_timeout < 0
+                liveness_check_timeout is not None
+                and liveness_check_timeout < 0
             ):
                 raise ConfigurationError(
                     'The config setting "liveness_check_timeout" must be '
@@ -566,7 +574,7 @@ def session(
             # they may be change or removed any time without prior notice
             initial_retry_delay: float = ...,
             retry_delay_multiplier: float = ...,
-            retry_delay_jitter_factor: float = ...
+            retry_delay_jitter_factor: float = ...,
         ) -> AsyncSession:
             ...
 
@@ -581,6 +589,10 @@ def session(self, **config) -> AsyncSession:
 
             :returns: new :class:`neo4j.AsyncSession` object
             """
+            if "warn_notification_severity" in config:
+                # Would work just fine, but we don't want to introduce yet
+                # another undocumented/unsupported config option.
+                del config["warn_notification_severity"]
             self._check_state()
             session_config = self._read_session_config(config)
             return self._session(session_config)
@@ -1312,14 +1324,33 @@ def __init__(self, pool, default_workspace_config):
         AsyncDriver.__init__(self, pool, default_workspace_config)
 
 
-def _normalize_notifications_config(config_kwargs):
-    if config_kwargs.get("notifications_disabled_categories") is not None:
-        config_kwargs["notifications_disabled_categories"] = [
-            getattr(e, "value", e)
-            for e in config_kwargs["notifications_disabled_categories"]
-        ]
-    if config_kwargs.get("notifications_min_severity") is not None:
-        config_kwargs["notifications_min_severity"] = getattr(
-            config_kwargs["notifications_min_severity"], "value",
-            config_kwargs["notifications_min_severity"]
+
+def _normalize_notifications_config(config_kwargs, *, driver_level=False):
+    list_config_keys = ("notifications_disabled_categories",)
+    for key in list_config_keys:
+        value = config_kwargs.get(key)
+        if value is not None:
+            config_kwargs[key] = [getattr(e, "value", e) for e in value]
+    single_config_keys = (
+        "notifications_min_severity",
+        "warn_notification_severity",
+    )
+    for key in single_config_keys:
+        value = config_kwargs.get(key)
+        if value is not None:
+            config_kwargs[key] = getattr(value, "value", value)
+    value = config_kwargs.get("warn_notification_severity")
+    if value not in (*NotificationMinimumSeverity, None):
+        raise ValueError(
+            f"Invalid value for configuration "
+            f"warn_notification_severity: {value}. Should be None, a "
+            f"NotificationMinimumSeverity, or a string representing a "
+            f"NotificationMinimumSeverity."
         )
+    if driver_level:
+        if value is None:
+            if DEBUG_ENABLED:
+                config_kwargs["warn_notification_severity"] = \
+                    NotificationMinimumSeverity.INFORMATION
+        elif value == NotificationMinimumSeverity.OFF:
+            config_kwargs["warn_notification_severity"] = None