Skip to content

Commit 252a0e1

Browse files
Merge pull request #3 from cheese-drawer/sync-client
Sync client
2 parents 2abb6bf + fcc080f commit 252a0e1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+3179
-436
lines changed

.pylintrc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ fail-under=10.0
1010

1111
# Add files or directories to the blacklist. They should be base names, not
1212
# paths.
13-
ignore=CVS
13+
ignore=CVS,stubs
1414

1515
# Add files or directories matching the regex patterns to the blacklist. The
1616
# regex matches against base names, not paths.
@@ -479,7 +479,7 @@ ignore-comments=yes
479479
ignore-docstrings=yes
480480

481481
# Ignore imports when computing similarities.
482-
ignore-imports=no
482+
ignore-imports=yes
483483

484484
# Minimum lines number of a similarity.
485485
min-similarity-lines=4

README.md

Lines changed: 63 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# DB Wrapper Lib
22

3-
A simple wrapper on [aio-libs/aiopg](https://github.com/aio-libs/aiopg).
3+
A simple wrapper on [aio-libs/aiopg](https://github.com/aio-libs/aiopg) or [psycopg/psycopg2](https://github.com/psycopg/psycopg2).
44
Encapsulates connection logic & execution logic into one Client class for convenience.
55

66
## Installation
@@ -9,32 +9,32 @@ Install with `pip` from releases on this repo.
99
For example, you can install version 0.1.0 with the following command:
1010

1111
```
12-
$ pip install https://github.com/cheese-drawer/lib-python-db-wrapper/releases/download/0.1.2/db-wrapper-0.1.2.tar.gz
12+
$ pip install https://github.com/cheese-drawer/lib-python-db-wrapper/releases/download/2.1.0/db-wrapper-2.1.0.tar.gz
1313
```
1414

15-
If looking for a different release version, just replace the two instances of `0.1.2` in the command with the version number you need.
15+
If looking for a different release version, just replace the two instances of `2.1.0` in the command with the version number you need.
1616

1717
## Usage
1818

19-
This library uses a fairly simple API to manage asynchronously connecting to, executing queries on, & disconnecting from a PostgresQL database.
20-
Additionally, it includes a very simple Model abstraction to help with defining queries for a given model & managing separation of concerns in your application.
19+
This library uses a fairly simple API to manage connecting to, executing queries on, & disconnecting from a PostgresQL database, in both synchronous & asynchronous APIs.
20+
Additionally, it includes a very simple Model abstraction to help with declaring data types, enforcing types at runtime, defining queries for a given model, & managing separation of concerns in your application.
2121

22-
### Example `Client`
22+
### Example: Clients
2323

2424
Intializing a database `Client` & executing a query begins with defining a connection & giving it to `Client` on intialization:
2525

2626
```python
27-
from db_wrapper import ConnectionParameters
27+
from db_wrapper import ConnectionParameters, AsyncClient
2828

2929
connection_parameters = ConnectionParameters(
3030
host='localhost',
3131
user='postgres',
3232
password='postgres',
3333
database='postgres')
34-
client = Client(connection_parameters)
34+
client = AsyncClient(connection_parameters)
3535
```
3636

37-
From there, you need to tell the client to connect using `Client.connect()` before you can execute any queries.
37+
From there, you need to tell the client to connect using `client.connect()` before you can execute any queries.
3838
This method is asynchronous though, so you need to supply an async/await runtime.
3939

4040
```python
@@ -45,7 +45,7 @@ import asyncio
4545

4646
async def a_query() -> None:
4747
# we'll come back to this part
48-
# just know that it usins async/await to call Client.execute_and_return
48+
# just know that it uses async/await to call Client.execute_and_return
4949
result = await client.execute_and_return(query)
5050

5151
# do something with the result...
@@ -78,14 +78,42 @@ async def a_query() -> None:
7878

7979
```
8080

81-
### Example: `Model`
81+
Alternatively, everything can also be done synchronously, using an API that is almost exactly the same.
82+
Simply drop the async/await keywords & skip the async event loop, then proceed in exactly the same fashion:
8283

83-
Using `Model` isn't necessary at all, you can just interact directly with the `Client` instance using it's `execute` & `execute_and_return` methods to execute SQL queries as needed.
84-
`Model` may be helpful in managing your separation of concerns by giving you a single place to define queries related to a given data model in your database.
85-
Additionally, `Model` will be helpful in defining types, if you're using mypy to check your types in development.
84+
```python
85+
from db_wrapper import ConnectionParameters, SyncClient
86+
87+
connection_parameters = ConnectionParameters(
88+
host='localhost',
89+
user='postgres',
90+
password='postgres',
91+
database='postgres')
92+
client = SyncClient(connection_parameters)
93+
94+
95+
def a_query() -> None:
96+
query = 'SELECT table_name' \
97+
'FROM information_schema.tables' \
98+
'WHERE table_schema = public'
99+
result = client.execute_and_return(query)
100+
101+
assert result[0] == 'postgres'
102+
103+
104+
client.connect()
105+
a_query()
106+
client.disconnect()
107+
```
108+
109+
### Example: Models
110+
111+
Using `AsyncModel` or `SyncModel` isn't necessary at all, you can just interact directly with the Client instance using it's `execute` & `execute_and_return` methods to execute SQL queries as needed.
112+
A Model may be helpful in managing your separation of concerns by giving you a single place to define queries related to a given data model in your database.
113+
Additionally, `Model` will be helpful in defining types, if you're using mypy to check your types in development, & in enforcing types at runtime using pydantic..
86114
It has no concept of types at runtime, however, & cannot be relied upon to constrain data types & shapes during runtime.
87115

88-
A `Model` instance has 4 properties, corresponding with each of the CRUD operations: `create`, `read`, `update`, & `delete`.
116+
A Model instance has 4 properties, corresponding with each of the CRUD operations: `create`, `read`, `update`, & `delete`.
89117
Each CRUD property has one built-in method to handle the simplest of queries for you already (create one record, read one record by id, update one record by id, & delete one record by id).
90118

91119
Using a model requires defining it's expected type (using `ModelData`), initializing a new instance, then calling the query methods as needed.
@@ -102,35 +130,36 @@ class AModel(ModelData):
102130
a_boolean_value: bool
103131
```
104132

105-
Subclassing `ModelData` is important because `Model` expects all records to be constrained to a dictionary containing at least one field labeled `_id` & constrained to the UUID type. This means the above `AModel` will contain records that look like the following dictionary in python:
133+
Subclassing `ModelData` is important because `Model` expects all records to be constrained to a Subclass of `ModelData`, containing least one property labeled `_id` constrained to the UUID type.
134+
This means the above `AModel` will contain records that look like the following dictionary in python:
106135

107136
```python
108-
a_model_result = {
137+
a_model_result.dict() == {
109138
_id: UUID(...),
110139
a_string_value: 'some string',
111140
a_number_value: 12345,
112141
a_boolean_value: True
113142
}
114143
```
115144

116-
Then to initialize your `Model` with your new expected type, simply initialize `Model` by passing `AModel` as a type parameter, a `Client` instance, & the name of the table this `Model` will be represented on:
145+
Then to initialize your Model with your new expected type, simply initialize `AsyncModel` or `SyncModel` by passing `AModel` as a type parameter, a matching Client instance, & the name of the table this Model will be represented on:
117146

118147
```python
119148
from db_wrapper import (
120149
ConnectionParameters,
121-
Client,
122-
Model,
150+
AsyncClient,
151+
AsyncModel,
123152
ModelData,
124153
)
125154

126155
connection_parameters = ConnectionParameters(...)
127-
client = Client(...)
156+
client = AsyncClient(...)
128157

129158

130159
class AModel(ModelData):
131160
# ...
132161

133-
a_model = Model[AModel](client, 'a_table_name')
162+
a_model = AsyncModel[AModel](client, 'a_table_name')
134163
```
135164

136165
From there, you can query your new `Model` by calling CRUD methods on the instance:
@@ -142,7 +171,8 @@ from typing import List
142171

143172

144173
async get_some_record() -> List[AModel]:
145-
return await a_model.read.one_by_id('some record id') # NOTE: in reality the id would be a UUID
174+
return await a_model.read.one_by_id('some record id')
175+
# NOTE: in reality the id would be a UUID
146176
```
147177

148178
Of course, just having methods for creating, reading, updating, or deleting a single record at a time often won't be enough.
@@ -152,8 +182,7 @@ For example, if you want to write an additional query for reading any record tha
152182

153183
```python
154184
from db_wrapper import ModelData
155-
from db_wrapper.model import Read
156-
from psycopg2 import sql
185+
from db_wrapper.model import AsyncRead, sql
157186

158187
# ...
159188

@@ -162,7 +191,7 @@ class AnotherModel(ModelData):
162191
a_field: str
163192

164193

165-
class ExtendedReader(Read[AnotherModel]):
194+
class ExtendedReader(AsyncRead[AnotherModel]):
166195
"""Add custom method to Model.read."""
167196

168197
async def all_with_some_string_value(self) -> List[AnotherModel]:
@@ -173,7 +202,9 @@ class ExtendedReader(Read[AnotherModel]):
173202
# sql module
174203
query = sql.SQL(
175204
'SELECT * '
176-
'FROM {table} ' # a Model knows it's own table name, no need to specify it manually here
205+
'FROM {table} '
206+
# a Model knows it's own table name,
207+
# no need to specify it manually here
177208
'WHERE a_field = 'some value';'
178209
).format(table=self._table)
179210

@@ -183,24 +214,24 @@ class ExtendedReader(Read[AnotherModel]):
183214
return result
184215
```
185216

186-
Then, you would subclass `Model` & redefine it's read property to be an instance of your new `ExtendedReader` class:
217+
Then, you would subclass `AsyncModel` & redefine it's read property to be an instance of your new `ExtendedReader` class:
187218

188219
```python
189-
from db_wrapper import Client, Model, ModelData
220+
from db_wrapper import AsyncClient, AsyncModel, ModelData
190221

191222
# ...
192223

193-
class ExtendedModel(Model[AnotherModel]):
224+
class ExtendedModel(AsyncModel[AnotherModel]):
194225
"""Build an AnotherModel instance."""
195226

196227
read: ExtendedReader
197228

198-
def __init__(self, client: Client) -> None:
229+
def __init__(self, client: AsyncClient) -> None:
199230
super().__init__(client, 'another_model_table') # you can supply your table name here
200231
self.read = ExtendedReader(self.client, self.table)
201232
```
202233

203-
Finally, using your `ExtendedModel` is simple, just initialize the class with a `Client` instance & use it just as you would your previous `Model` instance, `a_model`:
234+
Finally, using your `ExtendedModel` is simple, just initialize the class with a `AsyncClient` instance & use it just as you would your previous `AsyncModel` instance, `a_model`:
204235

205236
```python
206237
# ...

db_wrapper/__init__.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,5 +18,5 @@
1818
Model instance.
1919
"""
2020
from .connection import ConnectionParameters
21-
from .client import Client
22-
from .model import Model, ModelData
21+
from .client import AsyncClient, SyncClient
22+
from .model import AsyncModel, SyncModel, ModelData

db_wrapper/client/__init__.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""Create a database Client for managing connections & executing queries."""
2+
3+
from typing import Union
4+
5+
from .async_client import AsyncClient
6+
from .sync_client import SyncClient
7+
8+
Client = Union[AsyncClient, SyncClient]

db_wrapper/client.py renamed to db_wrapper/client/async_client.py

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,9 @@
1212

1313
import aiopg # type: ignore
1414
from psycopg2.extras import register_uuid
15-
# importing for the sole purpose of re-exporting
16-
# pylint: disable=unused-import
1715
from psycopg2 import sql
1816

19-
from .connection import ConnectionParameters, connect
17+
from db_wrapper.connection import ConnectionParameters, connect
2018

2119
# add uuid support to psycopg2 & Postgres
2220
register_uuid()
@@ -26,11 +24,10 @@
2624
# pylint: disable=invalid-name
2725
T = TypeVar('T')
2826

29-
# pylint: disable=unsubscriptable-object
3027
Query = Union[str, sql.Composed]
3128

3229

33-
class Client:
30+
class AsyncClient:
3431
"""Class to manage database connection & expose necessary methods to user.
3532
3633
Stores connection parameters on init, then exposes methods to

db_wrapper/client/sync_client.py

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
"""Wrapper on aiopg to simplify connecting to & interacting with db."""
2+
3+
from __future__ import annotations
4+
from typing import (
5+
Any,
6+
TypeVar,
7+
Union,
8+
Optional,
9+
Hashable,
10+
List,
11+
Dict)
12+
13+
from psycopg2.extras import register_uuid
14+
from psycopg2 import sql
15+
# pylint can't seem to find the items in psycopg2 despite being available
16+
from psycopg2._psycopg import cursor # pylint: disable=no-name-in-module
17+
18+
from db_wrapper.connection import (
19+
sync_connect,
20+
ConnectionParameters,
21+
)
22+
23+
# add uuid support to psycopg2 & Postgres
24+
register_uuid()
25+
26+
27+
# Generic doesn't need a more descriptive name
28+
# pylint: disable=invalid-name
29+
T = TypeVar('T')
30+
31+
Query = Union[str, sql.Composed]
32+
33+
34+
class SyncClient:
35+
"""Class to manage database connection & expose necessary methods to user.
36+
37+
Stores connection parameters on init, then exposes methods to
38+
asynchronously connect & disconnect the database, as well as execute SQL
39+
queries.
40+
"""
41+
42+
_connection_params: ConnectionParameters
43+
_connection: Any
44+
45+
def __init__(self, connection_params: ConnectionParameters) -> None:
46+
self._connection_params = connection_params
47+
48+
def connect(self) -> None:
49+
"""Connect to the database."""
50+
self._connection = sync_connect(self._connection_params)
51+
52+
def disconnect(self) -> None:
53+
"""Disconnect from the database."""
54+
self._connection.close()
55+
56+
@staticmethod
57+
def _execute_query(
58+
db_cursor: cursor,
59+
query: Query,
60+
params: Optional[Dict[Hashable, Any]] = None,
61+
) -> None:
62+
if params:
63+
db_cursor.execute(query, params) # type: ignore
64+
else:
65+
db_cursor.execute(query)
66+
67+
def execute(
68+
self,
69+
query: Query,
70+
params: Optional[Dict[Hashable, Any]] = None,
71+
) -> None:
72+
"""Execute the given SQL query.
73+
74+
Arguments:
75+
query (Query) -- the SQL query to execute
76+
params (dict) -- a dictionary of parameters to interpolate when
77+
executing the query
78+
79+
Returns:
80+
None
81+
"""
82+
with self._connection.cursor() as db_cursor:
83+
self._execute_query(db_cursor, query, params)
84+
85+
def execute_and_return(
86+
self,
87+
query: Query,
88+
params: Optional[Dict[Hashable, Any]] = None,
89+
) -> List[T]:
90+
"""Execute the given SQL query & return the result.
91+
92+
Arguments:
93+
query (Query) -- the SQL query to execute
94+
params (dict) -- a dictionary of parameters to interpolate when
95+
executing the query
96+
97+
Returns:
98+
List containing all the rows that matched the query.
99+
"""
100+
with self._connection.cursor() as db_cursor:
101+
self._execute_query(db_cursor, query, params)
102+
103+
result: List[T] = db_cursor.fetchall()
104+
return result

0 commit comments

Comments
 (0)