Merge branch 'features/refactoring'

Author: Paweł Kędzia

.version

@@ -0,0 +1 @@
+0.0.1

README.md

@@ -0,0 +1,141 @@
+## Overview
+
+The **LLM‑Router** project ships with a modular plugin system that lets you plug‑in **anonymizers**
+(also called *maskers*) and **guardrails** into request‑processing pipelines.  
+Each plugin implements a tiny, well‑defined interface (`apply`) and can be composed
+in an ordered list to form a **pipeline**. The pipelines are instantiated by the
+`MaskerPipeline` and `GuardrailPipeline` classes and are driven automatically by the
+endpoint logic in `endpoint_i.py`.
+
+---
+
+## 1. Anonymizers (Maskers)
+
+### 1.1 What they do
+
+* **Goal** – Remove or replace personally‑identifiable information (PII) from a payload before it reaches the LLM or
+  external service.
+* **Typical strategy** – Run a pipeline of maskers, to locate spans that correspond to IDs, etc., and replace each span
+  with a placeholder such as `{{MASKED_ITEM}}`.
+
+### 1.2 Built‑in anonymizer plugins
+
+Full list of `FastMaskerPlugin` masking strategies is located in [README.md](llm_router_plugins/maskers/fast_masker/README.md) file.
+
+| Plugin                                         | Description                                                                                                                                            | Technical notes                                                                                                                                                            |
+|------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **FastMaskerPlugin** (`fast_masker_plugin.py`) | A thin wrapper around the `FastMasker` utility class. It receives a JSON‑compatible payload and returns the same payload with all detected PII masked. | Implements `PluginInterface`. The heavy lifting is delegated to `FastMasker.mask_payload(payload)`. No extra I/O; the `FastMasker` instance is created once in `__init__`. |
+
+### 1.3 How a masker is used
+
+1. The endpoint (e.g. `EndpointI._do_masking_if_needed`) checks the global flag `FORCE_MASKING`.
+2. If enabled, it creates a `MaskerPipeline` with the list of masker plugin identifiers (e.g. `["fast_masker"]`).
+3. The pipeline calls each plugin’s `apply` method sequentially, feeding the output of one as the input to the next.
+4. The final payload – now stripped of PII – proceeds to the rest of the request flow (guardrails, model dispatch,
+   etc.).
+
+---
+
+## 2. Guardrails
+
+### 2.1 What they do
+
+* **Goal** – Verify that a request (or its response) complies with policy rules (e.g. no hateful, illegal, or unsafe
+  content).
+* **Typical strategy** – Split the payload into manageable text chunks, run a pipeline of guardrails,
+  aggregate per‑chunk scores, and decide whether the overall request is safe.
+
+### 2.2 Built‑in guardrail plugins
+
+| Plugin                                             | Description                                                                                                                                                                                                                                                  | Technical notes                                                                                                                                                                                                                       |
+|----------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **NASKGuardPlugin** (`nask_guard_plugin.py`)       | An HTTP‑based guardrail that forwards the payload to the external NASK guardrail service (`/nask_guard` endpoint) and returns a boolean *safe* flag together with the raw response.                                                                          | Inherits from `HttpPluginInterface`. The `apply` method calls `_request(payload)` (provided by the base class) and extracts `results["safe"]`. Errors are caught and logged; on failure the plugin returns `(False, {})`.             |
+| **(Implicit) GuardrailProcessor** (`processor.py`) | Not a plugin per‑se, but the core logic used by the internal NASK guardrail Flask route (`nask_guardrail`). It tokenises the payload, creates overlapping chunks, runs a Hugging‑Face `text‑classification` pipeline, and produces a detailed safety report. | Handles model loading (`AutoTokenizer`, `pipeline("text‑classification")`), chunking (`_chunk_text`), and scoring thresholds (`MIN_SCORE_FOR_SAFE`, `MIN_SCORE_FOR_NOT_SAFE`). Returns a dict: `{"safe": <bool>, "detailed": [...]}`. |
+
+### 2.3 How a guardrail is used
+
+1. The endpoint calls `_is_request_guardrail_safe(payload)` (or the analogous response guardrail).
+2. If `FORCE_GUARDRAIL_REQUEST` is true, a `GuardrailPipeline` is built from the configured plugin IDs (e.g.
+   `["nask_guard"]`).
+3. The pipeline iterates over each guardrail plugin; each `apply` returns `(is_safe, message)`.
+4. The first plugin that reports `is_safe=False` short‑circuits the pipeline and the request is rejected with a 400/500
+   error payload.
+
+---
+
+## 3. Pipelines
+
+Both masker and guardrail pipelines share the same design pattern:
+
+| Class                                                     | Purpose                                                                            |
+|-----------------------------------------------------------|------------------------------------------------------------------------------------|
+| **MaskerPipeline** (`pipeline.py` – masker version)       | Executes a list of masker plugins in order, transforming the payload step‑by‑step. |                                                                                                                                                                      
+| **GuardrailPipeline** (`pipeline.py` – guardrail version) | Executes guardrail plugins sequentially, stopping on the first failure.            |
+
+### 3.1 Registration
+
+* Plugins are registered lazily via `MaskerRegistry.register(name, logger)` or
+  `GuardrailRegistry.register(name, logger)`.
+* The registry maps a string identifier (e.g. `"fast_masker"`) to a concrete plugin class, allowing pipelines to resolve
+  the classes at runtime.
+
+### 3.2 Configuration
+
+All plugin identifiers are stored in environment variables or constants such as:
+
+```python
+MASKING_STRATEGY_PIPELINE = ["fast_masker"]
+GUARDRAIL_STRATEGY_PIPELINE_REQUEST = ["nask_guard"]
+```
+
+These lists are consumed by the endpoint initialization (`EndpointI._prepare_masker_pipeline`,
+`EndpointI._prepare_guardrails_pipeline`).
+
+---
+
+## 4. Adding a New Plugin
+
+1. **Create a subclass** of either `PluginInterface` (for maskers) or `HttpPluginInterface` / a custom guardrail base.
+2. **Define a `name` class attribute** – this is the identifier used in pipeline configuration.
+3. **Implement `apply(self, payload: Dict) -> Dict`** (masker) **or `apply(self, payload: Dict) -> Tuple[bool, Dict]`
+   ** (guardrail).
+4. **Register the plugin** – either automatically via the registry’s `register` call in the pipeline constructor, or
+   manually by calling `MaskerRegistry.register(name=MyPlugin.name, logger=logger)`.
+
+*Example stub for a new masker:*
+
+```python
+# my_custom_masker.py
+from llm_router_plugins.maskers.plugin_interface import PluginInterface
+import logging
+from typing import Dict, Optional
+
+
+class MyCustomMasker(PluginInterface):
+    name = "my_custom_masker"
+
+    def __init__(self, logger: Optional[logging.Logger] = None):
+        super().__init__(logger=logger)
+        # Load any heavy resources here (e.g., a spaCy model)
+
+    def apply(self, payload: Dict) -> Dict:
+        # Perform your masking logic and return the modified payload
+        return payload
+```
+
+After placing the file in `llm_router_plugins/maskers/plugins/`, you can enable it by adding `"my_custom_masker"` to
+`MASKING_STRATEGY_PIPELINE`.
+
+---
+
+## 5. Summary
+
+* **Anonymizers** (`FastMaskerPlugin`, `BANonymizer`) scrub PII from requests.
+* **Guardrails** (`NASKGuardPlugin`, internal `GuardrailProcessor`) enforce safety policies.
+* **Pipelines** (`MaskerPipeline`, `GuardrailPipeline`) orchestrate the sequential execution of these plugins,
+  short‑circuiting on failure for guardrails.
+* The system is **extensible**: new plugins are just classes that obey the tiny interface contract and can be referenced
+  by name in the configuration.
+
+These components together give the LLM‑Router a flexible, policy‑driven request‑processing stack that can be tailored to
+any deployment scenario.

llm_router_plugins/__init__.py

@@ -0,0 +1,113 @@
+"""
+NASK Guardrail Plugin
+
+This plugin sends the incoming ``payload`` to the NASK guardrail service and
+parses the JSON response.  The service URL can be configured through the
+environment variable ``LLM_ROUTER_GUARDRAIL_NASK_GUARD_HOST_EP``;
+
+The expected response format is:
+
+{
+    "results": {
+        "detailed": [
+            {
+                "chunk_index": 0,
+                "chunk_text": "...",
+                "label": "safe",
+                "safe": true,
+                "score": 0.9834
+            }
+        ],
+        "safe": true
+    }
+}
+
+If the request succeeds, ``apply`` returns a dictionary containing the
+extracted fields.  If any error occurs (network error, unexpected payload,
+missing keys, etc.) the method returns ``{'success': False}``.
+
+---
+
+**Model License:** The model used by this plugin is licensed under **CC BY‑NC‑SA 4.0**.
+**Router License:** The LLM router component is licensed under **Apache 2.0**.
+Before using the plugin, ensure that your intended use complies with these licenses.
+
+**Authors:** Aleksandra Krasnodębska, Karolina Seweryn, Szymon Łukasik, Wojciech Kusa
+(see *PL‑Guard: Benchmarking Language Model Safety for Polish*, 2025).
+
+"""
+
+import json
+import logging
+from typing import Dict, Optional, Tuple
+
+from llm_router_api.base.constants import GUARDRAIL_NASK_GUARD_HOST_EP
+from llm_router_plugins.plugin_interface import HttpPluginInterface
+
+
+class NASKGuardPlugin(HttpPluginInterface):
+    """
+    Concrete implementation of :class:`HttpPluginInterface` that
+    talks to the NASK guardrail HTTP endpoint.
+    """
+
+    name = "nask_guard"
+
+    def __init__(self, logger: Optional[logging.Logger] = None):
+        if not len(GUARDRAIL_NASK_GUARD_HOST_EP):
+            raise RuntimeError(
+                f"When you are using `nask_guard` plugin, you must provide a "
+                f"host with model, GUARDRAIL_NASK_GUARD_HOST_EP must be set "
+                f"to valid host."
+            )
+
+        super().__init__(logger=logger)
+
+    @property
+    def base_url(self) -> str:
+        """
+        Resolve the endpoint URL from the environment variable or fall back to
+        the default value.
+        """
+        return GUARDRAIL_NASK_GUARD_HOST_EP
+
+    def apply(self, payload: Dict) -> Tuple[bool, Dict]:
+        """
+        Send ``payload`` to the guardrail service, parse the JSON response and
+        expose the most relevant fields.
+
+        Parameters
+        ----------
+        payload: Dict
+            The data that should be evaluated by the guardrail.
+
+        Returns
+        -------
+        Dict
+            ``{'success': True, 'safe': <bool>, 'chunk_index': <int>,
+            'chunk_text': <str>, 'label': <str>, 'score': <float>}``
+            on success, or ``{'success': False}`` on any error.
+        """
+        try:
+            response = self._request(payload)
+            results = response.get("results", {})
+            safe_overall: bool = bool(results.get("safe", False))
+
+            # detailed = results.get("detailed", [])
+            # if not detailed:
+            #     # No detailed information – treat as failure
+            #     raise ValueError("Missing 'detailed' entries in response")
+            # first_chunk = detailed[0]
+            # chunk_index: int = first_chunk.get("chunk_index", -1)
+            # chunk_text: str = first_chunk.get("chunk_text", "")
+            # label: str = first_chunk.get("label", "")
+            # safe_chunk: bool = first_chunk.get("safe", False)
+            # score: float = first_chunk.get("score", 0.0)
+            # Build a concise result dictionary
+            return safe_overall, response
+        except Exception as exc:
+            if self._logger:
+                self._logger.error(
+                    "NASKGuardPlugin failed to process payload: %s", exc
+                )
+            return False, {}

llm_router_plugins/guardrails/__init__.py

@@ -0,0 +1,53 @@
+"""
+Executable pipeline for guardrail plugins.
+
+It works the same way as the masker pipeline: an ordered list of plugin
+identifiers is supplied, each plugin is registered (if not already), and then
+their ``apply`` methods are called sequentially on the payload.
+"""
+
+import logging
+from typing import Tuple, Dict
+
+from llm_router_plugins.guardrails.plugin_registrator import GuardrailRegistry
+
+
+class GuardrailPipeline:
+    """
+    Represents an executable pipeline of guardrail plugins.
+
+    The pipeline is built from an ordered list of plugin identifiers.
+    Calling ``apply(payload, *args, **kwargs)`` will invoke each plugin's
+    ``apply`` method sequentially, passing the result of one as the input
+    to the next.
+    """
+
+    def __init__(self, plugin_names: list[str], logger: logging.Logger):
+        self._logger = logger
+
+        # Ensure every requested plugin is instantiated and cached.
+        for p_name in plugin_names:
+            GuardrailRegistry.register(name=p_name, logger=logger)
+
+        # Resolve the concrete plugin instances.
+        self._plugin_instances = [
+            GuardrailRegistry.get(name) for name in plugin_names
+        ]
+
+    def apply(self, payload: Dict) -> Tuple[bool, Dict]:
+        """
+        Execute the pipeline.
+
+        Args:
+            payload: Initial data passed to the first guardrail plugin.
+            *args, **kwargs: Additional arguments forwarded to each plugin's
+                ``apply`` method.
+
+        Returns:
+            True when payload is satisfied, False otherwise.
+        """
+        for plugin in self._plugin_instances:
+            is_safe, message = plugin.apply(payload)
+            if not is_safe:
+                return False, message
+        return True, {}

llm_router_plugins/guardrails/nask/__init__.py

@@ -0,0 +1,70 @@
+"""
+Helper that registers guardrail plugins on demand, similar to the masker
+registrator.
+
+Usage:
+    GuardrailRegistry.register(name="name", logger=my_logger)
+    plugin = GuardrailRegistry.get("name")
+"""
+
+import logging
+from typing import Optional
+
+from llm_router_plugins.guardrails.registry import (
+    MAIN_GUARDRAILS_REGISTRY,
+    GUARDRAILS_REGISTRY_SESSION,
+)
+
+
+class GuardrailRegistry:
+    """Central registry for guardrail plugins."""
+
+    @staticmethod
+    def register(name: str, logger: Optional[logging.Logger] = None) -> None:
+        """
+        Register a guardrail plugin for the current session.
+
+        Args:
+            name: Identifier of the guardrail plugin (must exist in
+                  ``MAIN_GUARDRAILS_REGISTRY``).
+            logger: Optional logger that will be passed to the plugin's
+                    constructor.
+        """
+        if name not in MAIN_GUARDRAILS_REGISTRY:
+            raise KeyError(
+                f"Guardrail '{name}' not found in registry: {MAIN_GUARDRAILS_REGISTRY}"
+            )
+
+        # Already registered – nothing to do.
+        if name in GUARDRAILS_REGISTRY_SESSION:
+            return
+
+        # Instantiate the plugin and store it in the session cache.
+        _cls = MAIN_GUARDRAILS_REGISTRY[name](logger=logger)
+        GUARDRAILS_REGISTRY_SESSION[name] = _cls
+
+        if logger:
+            logger.info(
+                f"[guardrail] Registering guardrail '{name}' for plugin '{_cls}'"
+            )
+
+    @staticmethod
+    def get(name: str):
+        """
+        Retrieve a registered guardrail plugin instance by name.
+
+        Raises:
+            KeyError: If the plugin has not been registered yet.
+        """
+        try:
+            return GUARDRAILS_REGISTRY_SESSION[name]
+        except KeyError as exc:
+            raise KeyError(
+                f"Guardrail '{name}' not found in registry. "
+                f"Available plugins: {list(GUARDRAILS_REGISTRY_SESSION.keys())}"
+            ) from exc
+
+    @staticmethod
+    def list_plugins() -> list[str]:
+        """Return the list of guardrail names currently registered in the session."""
+        return list(GUARDRAILS_REGISTRY_SESSION.keys())