Merge pull request #20477 from netbox-community/20210-new-token-auth

Closes #20210: Implement new version of API token

Author: bctiemann

contrib/openapi.json

@@ -23,6 +23,31 @@ ALLOWED_HOSTS = ['*']
 
 ---
 
+## API_TOKEN_PEPPERS
+
+!!! info "This parameter was introduced in NetBox v4.5."
+
+[Cryptographic peppers](https://en.wikipedia.org/wiki/Pepper_(cryptography)) are employed to generate hashes of sensitive values on the server. This parameter defines the peppers used to hash v2 API tokens in NetBox. You must define at least one pepper before creating a v2 API token. See the [API documentation](../integrations/rest-api.md#authentication) for further information about how peppers are used.
+
+```python
+API_TOKEN_PEPPERS = {
+    # DO NOT USE THIS EXAMPLE PEPPER IN PRODUCTION
+    1: 'kp7ht*76fiQAhUi5dHfASLlYUE_S^gI^(7J^K5M!LfoH@vl&b_',
+}
+```
+
+!!! warning "Peppers are sensitive"
+    Treat pepper values as extremely sensitive. Consider populating peppers from environment variables at initialization time rather than defining them in the configuration file, if feasible. 
+
+Peppers must be at least 50 characters in length and should comprise a random string with a diverse character set. Consider using the Python script at `$INSTALL_ROOT/netbox/generate_secret_key.py` to generate a pepper value.
+
+It is recommended to start with a pepper ID of `1`. Additional peppers can be introduced later as needed to begin rotating token hashes.
+
+!!! tip
+    Although NetBox will run without `API_TOKEN_PEPPERS` defined, the use of v2 API tokens will be unavailable.
+
+---
+
 ## DATABASE
 
 !!! warning "Legacy Configuration Parameter"

docs/configuration/required-parameters.md

@@ -8,7 +8,7 @@ NetBox's REST API, powered by the [Django REST Framework](https://www.django-res
 
 ```no-highlight
 curl -s -X POST \
--H "Authorization: Token $TOKEN" \
+-H "Authorization: Bearer $TOKEN" \
 -H "Content-Type: application/json" \
 http://netbox/api/ipam/prefixes/ \
 --data '{"prefix": "192.0.2.0/24", "site": {"name": "Branch 12"}}'

docs/features/api-integration.md

@@ -120,6 +120,23 @@ If you are not yet sure what the domain name and/or IP address of the NetBox ins
 ALLOWED_HOSTS = ['*']
 ```
 
+### API_TOKEN_PEPPERS
+
+Define at least one random cryptographic pepper, identified by a numeric ID starting at 1. This will be used to generate SHA256 checksums for API tokens.
+
+```python
+API_TOKEN_PEPPERS = {
+    # DO NOT USE THIS EXAMPLE PEPPER IN PRODUCTION
+    1: 'kp7ht*76fiQAhUi5dHfASLlYUE_S^gI^(7J^K5M!LfoH@vl&b_',
+}
+```
+
+!!! tip
+    As with [`SECRET_KEY`](#secret_key) below, you can use the `generate_secret_key.py` script to generate a random pepper:
+    ```no-highlight
+    python3 ../generate_secret_key.py
+    ```
+
 ### DATABASES
 
 This parameter holds the PostgreSQL database configuration details. The default database must be defined; additional databases may be defined as needed e.g. by plugins.

docs/installation/3-netbox.md

@@ -653,18 +653,19 @@ The NetBox REST API primarily employs token-based authentication. For convenienc
 
 ### Tokens
 
-A token is a unique identifier mapped to a NetBox user account. Each user may have one or more tokens which he or she can use for authentication when making REST API requests. To create a token, navigate to the API tokens page under your user profile.
+A token is a secret, unique identifier mapped to a NetBox user account. Each user may have one or more tokens which he or she can use for authentication when making REST API requests. To create a token, navigate to the API tokens page under your user profile. When creating a token, NetBox will automatically populate a randomly-generated token value.
 
 By default, all users can create and manage their own REST API tokens under the user control panel in the UI or via the REST API. This ability can be disabled by overriding the [`DEFAULT_PERMISSIONS`](../configuration/security.md#default_permissions) configuration parameter.
 
-Each token contains a 160-bit key represented as 40 hexadecimal characters. When creating a token, you'll typically leave the key field blank so that a random key will be automatically generated. However, NetBox allows you to specify a key in case you need to restore a previously deleted token to operation.
-
 Additionally, a token can be set to expire at a specific time. This can be useful if an external client needs to be granted temporary access to NetBox.
 
-!!! info "Restricting Token Retrieval"
-    The ability to retrieve the key value of a previously-created API token can be restricted by disabling the [`ALLOW_TOKEN_RETRIEVAL`](../configuration/security.md#allow_token_retrieval) configuration parameter.
+#### v1 and v2 Tokens
+
+Beginning with NetBox v4.5, two versions of API token are supported, denoted as v1 and v2. Users are strongly encouraged to create only v2 tokens and to discontinue the use of v1 tokens. Support for v1 tokens will be removed in a future NetBox release.
+
+v2 API tokens offer much stronger security. The token plaintext given at creation time is hashed together with a configured [cryptographic pepper](../configuration/required-parameters.md#api_token_peppers) to generate a unique checksum. This checksum is irreversible; the token plaintext is never stored on the server and thus cannot be retrieved.
 
-### Restricting Write Operations
+#### Restricting Write Operations
 
 By default, a token can be used to perform all actions via the API that a user would be permitted to do via the web UI. Deselecting the "write enabled" option will restrict API requests made with the token to read operations (e.g. GET) only.
 
@@ -681,10 +682,22 @@ It is possible to provision authentication tokens for other users via the REST A
 
 ### Authenticating to the API
 
-An authentication token is attached to a request by setting the `Authorization` header to the string `Token` followed by a space and the user's token:
+An authentication token is included with a request in its `Authorization` header. The format of the header value depends on the version of token in use. v2 tokens use the following form, concatenating the token's prefix (`nbt_`) and key with its plaintext value, separated by a period:
+
+```
+Authorization: Bearer nbt_<key>.<token>
+```
+
+Legacy v1 tokens use the prefix `Token` rather than `Bearer`, and include only the token plaintext. (v1 tokens do not have a key.)
+
+```
+Authorization: Token <token>
+```
+
+Below is an example REST API request utilizing a v2 token.
 
 ```
-$ curl -H "Authorization: Token $TOKEN" \
+$ curl -H "Authorization: Bearer nbt_4F9DAouzURLb.zjebxBPzICiPbWz0Wtx0fTL7bCKXKGTYhNzkgC2S" \
 -H "Accept: application/json; indent=4" \
 https://netbox/api/dcim/sites/
 {

docs/integrations/rest-api.md

@@ -26,8 +26,9 @@
 from netbox.authentication import get_auth_backend_display, get_saml_idps
 from netbox.config import get_config
 from netbox.views import generic
-from users import forms, tables
+from users import forms
 from users.models import UserConfig
+from users.tables import TokenTable
 from utilities.request import safe_for_redirect
 from utilities.string import remove_linebreaks
 from utilities.views import register_model_view
@@ -328,7 +329,8 @@ class UserTokenListView(LoginRequiredMixin, View):
 
     def get(self, request):
         tokens = UserToken.objects.filter(user=request.user)
-        table = tables.UserTokenTable(tokens)
+        table = TokenTable(tokens)
+        table.columns.hide('user')
         table.configure(request)
 
         return render(request, 'account/token_list.html', {
@@ -343,11 +345,9 @@ class UserTokenView(LoginRequiredMixin, View):
 
     def get(self, request, pk):
         token = get_object_or_404(UserToken.objects.filter(user=request.user), pk=pk)
-        key = token.key if settings.ALLOW_TOKEN_RETRIEVAL else None
 
         return render(request, 'account/token.html', {
             'object': token,
-            'key': key,
         })
 
 

netbox/account/tables.py

@@ -8,6 +8,7 @@
 from rq.registry import FailedJobRegistry, StartedJobRegistry
 
 from rest_framework import status
+from users.constants import TOKEN_PREFIX
 from users.models import Token, User
 from utilities.testing import APITestCase, APIViewTestCases, TestCase
 from utilities.testing.utils import disable_logging
@@ -136,7 +137,7 @@ def setUp(self):
         # Create the test user and assign permissions
         self.user = User.objects.create_user(username='testuser', is_active=True)
         self.token = Token.objects.create(user=self.user)
-        self.header = {'HTTP_AUTHORIZATION': f'Token {self.token.key}'}
+        self.header = {'HTTP_AUTHORIZATION': f'Bearer {TOKEN_PREFIX}{self.token.key}.{self.token.token}'}
 
         # Clear all queues prior to running each test
         get_queue('default').connection.flushall()

netbox/account/views.py

@@ -2,47 +2,90 @@
 
 from django.conf import settings
 from django.utils import timezone
-from rest_framework import authentication, exceptions
+from drf_spectacular.extensions import OpenApiAuthenticationExtension
+from rest_framework import exceptions
+from rest_framework.authentication import BaseAuthentication, get_authorization_header
 from rest_framework.permissions import BasePermission, DjangoObjectPermissions, SAFE_METHODS
 
 from netbox.config import get_config
+from users.constants import TOKEN_PREFIX
 from users.models import Token
 from utilities.request import get_client_ip
 
+V1_KEYWORD = 'Token'
+V2_KEYWORD = 'Bearer'
 
-class TokenAuthentication(authentication.TokenAuthentication):
+
+class TokenAuthentication(BaseAuthentication):
     """
     A custom authentication scheme which enforces Token expiration times and source IP restrictions.
     """
     model = Token
 
     def authenticate(self, request):
-        result = super().authenticate(request)
-
-        if result:
-            token = result[1]
-
-            # Enforce source IP restrictions (if any) set on the token
-            if token.allowed_ips:
-                client_ip = get_client_ip(request)
-                if client_ip is None:
-                    raise exceptions.AuthenticationFailed(
-                        "Client IP address could not be determined for validation. Check that the HTTP server is "
-                        "correctly configured to pass the required header(s)."
-                    )
-                if not token.validate_client_ip(client_ip):
-                    raise exceptions.AuthenticationFailed(
-                        f"Source IP {client_ip} is not permitted to authenticate using this token."
-                    )
-
-        return result
-
-    def authenticate_credentials(self, key):
-        model = self.get_model()
+        # Authorization header is not present; ignore
+        if not (auth := get_authorization_header(request).split()):
+            return
+        # Unrecognized header; ignore
+        if auth[0].lower() not in (V1_KEYWORD.lower().encode(), V2_KEYWORD.lower().encode()):
+            return
+        # Check for extraneous token content
+        if len(auth) != 2:
+            raise exceptions.AuthenticationFailed(
+                'Invalid authorization header: Must be in the form "Bearer <key>.<token>" or "Token <token>"'
+            )
+        # Extract the key (if v2) & token plaintext from the auth header
+        try:
+            auth_value = auth[1].decode()
+        except UnicodeError:
+            raise exceptions.AuthenticationFailed("Invalid authorization header: Token contains invalid characters")
+
+        # Infer token version from presence or absence of prefix
+        version = 2 if auth_value.startswith(TOKEN_PREFIX) else 1
+
+        if version == 1:
+            key, plaintext = None, auth_value
+        else:
+            auth_value = auth_value.removeprefix(TOKEN_PREFIX)
+            try:
+                key, plaintext = auth_value.split('.', 1)
+            except ValueError:
+                raise exceptions.AuthenticationFailed(
+                    "Invalid authorization header: Could not parse key from v2 token. Did you mean to use 'Token' "
+                    "instead of 'Bearer'?"
+                )
+
+        # Look for a matching token in the database
         try:
-            token = model.objects.prefetch_related('user').get(key=key)
-        except model.DoesNotExist:
-            raise exceptions.AuthenticationFailed("Invalid token")
+            qs = Token.objects.prefetch_related('user')
+            if version == 1:
+                # Fetch v1 token by querying plaintext value directly
+                token = qs.get(version=version, plaintext=plaintext)
+            else:
+                # Fetch v2 token by key, then validate the plaintext
+                token = qs.get(version=version, key=key)
+                if not token.validate(plaintext):
+                    # Key is valid but plaintext is not. Raise DoesNotExist to guard against key enumeration.
+                    raise Token.DoesNotExist()
+        except Token.DoesNotExist:
+            raise exceptions.AuthenticationFailed(f"Invalid v{version} token")
+
+        # Enforce source IP restrictions (if any) set on the token
+        if token.allowed_ips:
+            client_ip = get_client_ip(request)
+            if client_ip is None:
+                raise exceptions.AuthenticationFailed(
+                    "Client IP address could not be determined for validation. Check that the HTTP server is "
+                    "correctly configured to pass the required header(s)."
+                )
+            if not token.validate_client_ip(client_ip):
+                raise exceptions.AuthenticationFailed(
+                    f"Source IP {client_ip} is not permitted to authenticate using this token."
+                )
+
+        # Enforce the Token's expiration time, if one has been set.
+        if token.is_expired:
+            raise exceptions.AuthenticationFailed("Token expired")
 
         # Update last used, but only once per minute at most. This reduces write load on the database
         if not token.last_used or (timezone.now() - token.last_used).total_seconds() > 60:
@@ -54,11 +97,8 @@ def authenticate_credentials(self, key):
             else:
                 Token.objects.filter(pk=token.pk).update(last_used=timezone.now())
 
-        # Enforce the Token's expiration time, if one has been set.
-        if token.is_expired:
-            raise exceptions.AuthenticationFailed("Token expired")
-
         user = token.user
+
         # When LDAP authentication is active try to load user data from LDAP directory
         if 'netbox.authentication.LDAPBackend' in settings.REMOTE_AUTH_BACKEND:
             from netbox.authentication import LDAPBackend
@@ -132,3 +172,17 @@ def has_permission(self, request, view):
         if not settings.LOGIN_REQUIRED:
             return True
         return request.user.is_authenticated
+
+
+class TokenScheme(OpenApiAuthenticationExtension):
+    target_class = 'netbox.api.authentication.TokenAuthentication'
+    name = 'tokenAuth'
+    match_subclasses = True
+
+    def get_security_definition(self, auto_schema):
+        return {
+            'type': 'apiKey',
+            'in': 'header',
+            'name': 'Authorization',
+            'description': '`Token <token>` (v1) or `Bearer <key>.<token>` (v2)',
+        }

netbox/core/tests/test_api.py

@@ -68,6 +68,16 @@
 # https://docs.djangoproject.com/en/stable/ref/settings/#std:setting-SECRET_KEY
 SECRET_KEY = ''
 
+# Define a mapping of cryptographic peppers to use when hashing API tokens. A minimum of one pepper is required to
+# enable v2 API tokens (NetBox v4.5+). Define peppers as a mapping of numeric ID to pepper value, as shown below. Each
+# pepper must be at least 50 characters in length.
+#
+#     API_TOKEN_PEPPERS = {
+#         1: "<random string>",
+#         2: "<random string>",
+#     }
+API_TOKEN_PEPPERS = {}
+
 
 #########################
 #                       #