Skip to content

Commit 53dbf08

Browse files
aclark4lifeaclark4life
committed
Code review fixes for tests and docs
1 parent 5e0818d commit 53dbf08

File tree

5 files changed

+195
-190
lines changed

5 files changed

+195
-190
lines changed

‎docs/source/howto/queryable-encryption.rst‎

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,12 @@ Configuring Queryable Encryption
33
================================
44

55
Configuring Queryable Encryption in Django is similar to
6-
`configuring Queryable Encryption in Python <https://www.mongodb.com/docs/manual/core/queryable-encryption/quick-start/>`_
7-
but with some additional steps required for Django.
6+
:doc:`manual:core/queryable-encryption/quick-start` but with some additional
7+
steps required for Django.
88

9-
.. note:: This section describes how to configure server side Queryable Encryption in Django.
10-
For configuration of client side Queryable Encryption, please refer to this :ref:`FAQ question <queryable-encryption>`.
9+
.. note:: This section describes how to configure server side Queryable
10+
Encryption in Django. For configuration of client side Queryable Encryption,
11+
please refer to this :ref:`FAQ question <queryable-encryption>`.
1112

1213
Prerequisites
1314
-------------
@@ -19,18 +20,18 @@ you will need to install PyMongo with Queryable Encryption support::
1920
pip install django-mongodb-backend[encryption]
2021

2122
.. note:: You can use Queryable Encryption on a MongoDB 7.0 or later replica
22-
set or sharded cluster, but not a standalone instance.
23-
`This table <https://www.mongodb.com/docs/manual/core/queryable-encryption/reference/compatibility/#std-label-qe-compatibility-reference>`_
24-
shows which MongoDB server products support which Queryable Encryption mechanisms.
23+
set or sharded cluster, but not a standalone instance.
24+
:ref:`This table <manual:qe-compatibility-reference>` shows which MongoDB
25+
server products support which Queryable Encryption mechanisms.
2526

2627
.. _server-side-queryable-encryption-settings:
2728

2829
Settings
2930
--------
3031

31-
Queryable Encryption in Django requires the use of an additional encrypted database
32-
and Key Management Service (KMS) credentials as well as an encrypted database
33-
router. Here's how to set it up in your Django settings.
32+
Queryable Encryption in Django requires the use of an additional encrypted
33+
database and Key Management Service (KMS) credentials as well as an encrypted
34+
database router. Here's how to set it up in your Django settings.
3435

3536
::
3637

@@ -73,4 +74,5 @@ router. Here's how to set it up in your Django settings.
7374

7475
DATABASE_ROUTERS = [EncryptedRouter()]
7576

76-
You are now ready to use server side :doc:`Queryable Encryption </topics/queryable-encryption>` in your Django project.
77+
You are now ready to use server side :doc:`Queryable Encryption
78+
</topics/queryable-encryption>` in your Django project.

‎docs/source/topics/known-issues.rst‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -102,8 +102,3 @@ Caching
102102
Secondly, you must use the :class:`django_mongodb_backend.cache.MongoDBCache`
103103
backend rather than Django's built-in database cache backend,
104104
``django.core.cache.backends.db.DatabaseCache``.
105-
106-
Queryable Encryption
107-
====================
108-
109-
.. TODO: Add Django core limitations that affect Queryable Encryption.

‎docs/source/topics/queryable-encryption.rst‎

Lines changed: 25 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
Queryable Encryption
22
====================
33

4-
Use :ref:`encrypted fields <encrypted-fields>` to store sensitive data in MongoDB
5-
your data using `Queryable Encryption <https://www.mongodb.com/docs/manual/core/queryable-encryption/>`_.
4+
Use :ref:`encrypted fields <encrypted-fields>` to store sensitive data in
5+
MongoDB with :doc:`manual:core/queryable-encryption`.
66

77
.. _encrypted-field-example:
88

@@ -22,25 +22,11 @@ Let's consider this example::
2222
def __str__(self):
2323
return self.ssn
2424

25-
The API is similar to that of Django's relational fields, with some
26-
security-related changes::
27-
28-
>>> bob = Patient(ssn="123-45-6789")
29-
>>> bob.ssn
30-
'123-45-6789'
31-
32-
Represented in BSON, from an encrypted client connection, the patient data looks like this:
33-
34-
.. code-block:: js
35-
36-
{
37-
_id: ObjectId('68825b066fac55353a8b2b41'),
38-
ssn: '123-45-6789',
39-
__safeContent__: [b'\xe0)NOFB\x9a,\x08\xd7\xdd\xb8\xa6\xba$…']
40-
}
25+
Querying encrypted fields
26+
-------------------------
4127

42-
The ``ssn`` field is only visible from an encrypted client connection. From an unencrypted client connection,
43-
the patient data looks like this:
28+
The ``ssn`` field is only visible from an encrypted client connection. From an
29+
unencrypted client connection, the patient data looks like this:
4430

4531
.. code-block:: js
4632
@@ -51,17 +37,27 @@ the patient data looks like this:
5137
5238
.. admonition:: List of encrypted fields
5339

54-
See the full list of :ref:`encrypted fields <encrypted-fields>` in the :doc:`Model field reference </ref/models/fields>`.
40+
See the full list of :ref:`encrypted fields <encrypted-fields>` in the
41+
:doc:`Model field reference </ref/models/fields>`.
5542

56-
Querying encrypted fields
57-
-------------------------
58-
59-
You can query encrypted fields using a `limited set of
60-
query operators <https://www.mongodb.com/docs/manual/core/queryable-encryption/reference/supported-operations/#std-label-qe-supported-query-operators>`_
61-
which must be specified in the field definition. For example, to query the ``ssn`` field for equality, you can use the
62-
``EqualityQuery`` operator as shown in the example above.
43+
You can query encrypted fields using a
44+
:ref:`manual:qe-supported-query-operators` which must be specified in the
45+
field definition. For example, to query the ``ssn`` field for equality, you
46+
can use the ``EqualityQuery`` operator as shown in the example above.
6347

6448
>>> Patient.objects.get(ssn="123-45-6789").ssn
6549
'123-45-6789'
6650

67-
If the ``ssn`` field provided in the query matches the encrypted value in the database, the query will succeed.
51+
If the ``ssn`` field provided in the query matches the encrypted value in the
52+
database, the query will succeed.
53+
54+
Represented in BSON, from an encrypted client connection, the patient data
55+
looks like this:
56+
57+
.. code-block:: js
58+
59+
{
60+
_id: ObjectId('68825b066fac55353a8b2b41'),
61+
ssn: '123-45-6789',
62+
__safeContent__: [b'\xe0)NOFB\x9a,\x08\xd7\xdd\xb8\xa6\xba$…']
63+
}

‎tests/encryption_/tests.py‎ renamed to ‎tests/encryption_/test_base.py‎

Lines changed: 13 additions & 145 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,10 @@
11
import importlib
22
from datetime import datetime, time
3-
from io import StringIO
43
from unittest.mock import patch
54

65
import pymongo
7-
from bson import json_util
86
from bson.binary import Binary
97
from django.conf import settings
10-
from django.core.management import call_command
118
from django.db import connections, models
129
from django.test import TransactionTestCase, modify_settings, override_settings
1310

@@ -23,115 +20,6 @@
2320
)
2421
from .routers import TestEncryptedRouter
2522

26-
EXPECTED_ENCRYPTED_FIELDS_MAP = {
27-
"encryption__billing": {
28-
"fields": [
29-
{
30-
"bsonType": "string",
31-
"path": "cc_type",
32-
"queries": {"queryType": "equality"},
33-
# "keyId": Binary(b" \x901\x89\x1f\xafAX\x9b*\xb1\xc7\xc5\xfdl\xa4", 4),
34-
},
35-
{
36-
"bsonType": "long",
37-
"path": "cc_number",
38-
"queries": {"queryType": "equality"},
39-
# "keyId": Binary(b"\x97\xb4\x9d\xb8\xd5\xa6Ay\x85\xfe\x00\xc0\xd4{\xa2\xff", 4),
40-
},
41-
{
42-
"bsonType": "decimal",
43-
"path": "account_balance",
44-
"queries": {"queryType": "range"},
45-
# "keyId": Binary(b"\xcc\x01-s\xea\xd9B\x8d\x80\xd7\xf8!n\xc6\xf5U", 4),
46-
},
47-
]
48-
},
49-
"encryption__patientrecord": {
50-
"fields": [
51-
{
52-
"bsonType": "string",
53-
"path": "ssn",
54-
"queries": {"queryType": "equality"},
55-
# "keyId": Binary(b"\x14F\x89\xde\x8d\x04K7\xa9\x9a\xaf_\xca\x8a\xfb&", 4),
56-
},
57-
{
58-
"bsonType": "date",
59-
"path": "birth_date",
60-
"queries": {"queryType": "range"},
61-
# "keyId": Binary(b"@\xdd\xb4\xd2%\xc2B\x94\xb5\x07\xbc(ER[s", 4),
62-
},
63-
{
64-
"bsonType": "binData",
65-
"path": "profile_picture",
66-
"queries": {"queryType": "equality"},
67-
# "keyId": Binary(b"Q\xa2\xebc!\xecD,\x8b\xe4$\xb6ul9\x9a", 4),
68-
},
69-
{
70-
"bsonType": "int",
71-
"path": "patient_age",
72-
"queries": {"queryType": "range"},
73-
# "keyId": Binary(b"\ro\x80\x1e\x8e1K\xde\xbc_\xc3bi\x95\xa6j", 4),
74-
},
75-
{
76-
"bsonType": "double",
77-
"path": "weight",
78-
"queries": {"queryType": "range"},
79-
# "keyId": Binary(b"\x9b\xfd:n\xe1\xd0N\xdd\xb3\xe7e)\x06\xea\x8a\x1d", 4),
80-
},
81-
]
82-
},
83-
"encryption__patient": {
84-
"fields": [
85-
{
86-
"bsonType": "int",
87-
"path": "patient_id",
88-
"queries": {"queryType": "equality"},
89-
# "keyId": Binary(b"\x8ft\x16:\x8a\x91D\xc7\x8a\xdf\xe5O\n[\xfd\\", 4),
90-
},
91-
{
92-
"bsonType": "string",
93-
"path": "patient_name",
94-
# "keyId": Binary(b"<\x9b\xba\xeb:\xa4@m\x93\x0e\x0c\xcaN\x03\xfb\x05", 4),
95-
},
96-
{
97-
"bsonType": "string",
98-
"path": "patient_notes",
99-
"queries": {"queryType": "equality"},
100-
# "keyId": Binary(b"\x01\xe7\xd1isnB$\xa9(gwO\xca\x10\xbd", 4),
101-
},
102-
{
103-
"bsonType": "date",
104-
"path": "registration_date",
105-
"queries": {"queryType": "equality"},
106-
# "keyId": Binary(b"F\xfb\xae\x82\xd5\x9a@\xee\xbfJ\xaf#\x9c:-I", 4),
107-
},
108-
{
109-
"bsonType": "bool",
110-
"path": "is_active",
111-
"queries": {"queryType": "equality"},
112-
# "keyId": Binary(b"\xb2\xb5\xc4K53A\xda\xb9V\xa6\xa9\x97\x94\xea;", 4),
113-
},
114-
{"bsonType": "string", "path": "email", "queries": {"queryType": "equality"}},
115-
]
116-
},
117-
"encryption__patientportaluser": {
118-
"fields": [
119-
{"bsonType": "string", "path": "ip_address", "queries": {"queryType": "equality"}},
120-
{"bsonType": "string", "path": "url", "queries": {"queryType": "equality"}},
121-
]
122-
},
123-
"encryption__encryptednumbers": {
124-
"fields": [
125-
{"bsonType": "int", "path": "pos_bigint", "queries": {"queryType": "equality"}},
126-
{"bsonType": "int", "path": "pos_smallint", "queries": {"queryType": "equality"}},
127-
{"bsonType": "int", "path": "smallint", "queries": {"queryType": "equality"}},
128-
]
129-
},
130-
"encryption__appointment": {
131-
"fields": [{"bsonType": "date", "path": "time", "queries": {"queryType": "equality"}}]
132-
},
133-
}
134-
13523

13624
class EncryptedDurationField(EncryptedFieldMixin, models.DurationField):
13725
"""
@@ -166,38 +54,36 @@ class EncryptedFieldTests(TransactionTestCase):
16654
available_apps = ["django_mongodb_backend", "encryption_"]
16755

16856
def setUp(self):
169-
self.appointment = Appointment(time="8:00")
170-
self.appointment.save()
57+
self.appointment = Appointment.objects.create(time="8:00")
17158

172-
self.billing = Billing(cc_type="Visa", cc_number=1234567890123456, account_balance=100.50)
173-
self.billing.save()
59+
self.billing = Billing.objects.create(
60+
cc_type="Visa", cc_number=1234567890123456, account_balance=100.50
61+
)
17462

175-
self.portal_user = PatientPortalUser(
63+
self.portal_user = PatientPortalUser.objects.create(
17664
ip_address="127.0.0.1",
17765
url="https://example.com",
17866
)
179-
self.portal_user.save()
18067

181-
self.patientrecord = PatientRecord(
68+
self.patientrecord = PatientRecord.objects.create(
18269
ssn="123-45-6789",
18370
birth_date="1970-01-01",
18471
profile_picture=b"image data",
18572
weight=175.5,
18673
patient_age=47,
18774
)
188-
self.patientrecord.save()
18975

190-
self.patient = Patient(
76+
self.patient = Patient.objects.create(
19177
patient_id=1,
19278
patient_name="John Doe",
19379
patient_notes="patient notes " * 25,
19480
registration_date=datetime(2023, 10, 1, 12, 0, 0),
19581
is_active=True,
19682
email="john.doe@example.com",
19783
)
198-
self.patient.save()
19984

200-
# TODO: Embed billing and patient_record models in patient model then add tests
85+
# TODO: Embed billing and patient_record models in patient model
86+
# then add tests
20187

20288
@classmethod
20389
def setUpClass(cls):
@@ -284,25 +170,6 @@ def test_get_encrypted_fields_map(self):
284170
expected_encrypted_fields_map[db_table],
285171
)
286172

287-
def test_show_schema_map(self):
288-
self.maxDiff = None
289-
out = StringIO()
290-
call_command(
291-
"showschemamap",
292-
"--database",
293-
"encrypted",
294-
verbosity=0,
295-
stdout=out,
296-
)
297-
# Remove keyIds since they are different for each run.
298-
output_json = json_util.loads(out.getvalue())
299-
for table in output_json:
300-
for field in output_json[table]["fields"]:
301-
del field["keyId"]
302-
# TODO: probably we don't need to test the entire mapping, otherwise it
303-
# requires updates every time a new model or field is added!
304-
self.assertEqual(EXPECTED_ENCRYPTED_FIELDS_MAP, output_json)
305-
306173
def test_set_encrypted_fields_map_in_client(self):
307174
# TODO: Create new client with and without schema map provided then
308175
# sync database to ensure encrypted collections are created in both
@@ -373,9 +240,10 @@ def test_patient(self):
373240
records = connections["encrypted"].database.encryption__patientrecord.find()
374241
self.assertTrue("__safeContent__" in records[0])
375242

376-
377-
class EncryptedNumberFieldTests(EncryptedFieldTests):
378-
def test_create_and_query(self):
243+
def test_numeric_fields(self):
244+
"""
245+
Fields that have not been tested elsewhere.
246+
"""
379247
EncryptedNumbers.objects.create(
380248
pos_bigint=1000000,
381249
# FIXME: pymongo.errors.EncryptionError: Cannot encrypt element of type int

0 commit comments

Comments
 (0)