Skip to content

Commit ad4bd6e

Browse files
whackedwhacked
committed
fix cell leakage, add cli executable, bump to 0.0.6
v0.0.5 failed because due to a "mixed" cell we got a prepare-time warning: ``` nbdev/processors.py:186: UserWarning: Found cells containing imports and other code. See FAQ. ``` I didn't see this at first, and due to all the test errors popping up, merged all the test cells. This caused the test code to leak into the exported code, which hard-coded the default storage driver to MemoryFS, effectively breaking the library entirely while also printing the test case on every run. Fixing the mixed import cell restores everything. turns out this is configured in settings.ini. We also use a decorator hack and a importlib hack to fetch + print the version in the cli usage string
1 parent 4404a31 commit ad4bd6e

File tree

10 files changed

+214
-351
lines changed

10 files changed

+214
-351
lines changed

‎README.md‎

Lines changed: 51 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,26 +9,67 @@ schematized config
99
pip install python_schematized_config
1010
```
1111

12+
## usage
13+
14+
``` python
15+
from schematized_config.core import ConfigValidator
16+
```
17+
18+
``` python
19+
import os
20+
import os.path as _p
21+
22+
if _p.exists('.env') and 'CONFIG_VALIDATOR_JSON_SCHEMA' in os.environ:
23+
ConfigValidator.load_dotenv() # use defaults of .env and CONFIG_VALIDATOR_JSON_SCHEMA
24+
```
25+
1226
# development
1327

28+
<details>
29+
<summary>
30+
entering the development environment
31+
</summary>
32+
1433
assuming you have [nix](https://nixos.org/download.html) installed and
1534
ready, make sure `nix-command` and `flake` are enabled (oneliner: run
1635
`export NIX_CONFIG="experimental-features = nix-command flakes"` in the
1736
terminal), then enter the dev shell using `nix develop`
1837

1938
start the jupyter notebook using the provided alias or just
2039
`jupyter notebook`, and hack away
40+
</details>
2141

22-
## How to use
42+
## nbdev
2343

24-
``` python
25-
from schematized_config.core import ConfigValidator
26-
```
44+
this package is developed using [nbdev](https://nbdev.fast.ai/), so we
45+
use an nbdev-centric development flow. For a quick guide, we recommend
46+
checking out the [end-to-end
47+
walkthrough](https://nbdev.fast.ai/tutorials/tutorial.html). But in
48+
short: edit notebooks, then run the `nbdev_*` management commands. The
49+
most essential flow is as follows:
2750

28-
``` python
29-
import os
30-
import os.path as _p
51+
1. edit the notebook files (core fore core, and cli for the command
52+
line interface)
53+
2. `python setup.py install` \# note we don’t use `nbdev_install`
54+
because we manage `quarto` using `nix`
55+
3. `nbdev_prepare`
56+
4. run code using the package
57+
5. `nbdev_release`
58+
6. `nbdev_pypi`
3159

32-
if _p.exists('.env') and 'CONFIG_VALIDATOR_JSON_SCHEMA' in os.environ:
33-
ConfigValidator.load_dotenv() # use defaults of .env and CONFIG_VALIDATOR_JSON_SCHEMA
34-
```
60+
### running tests
61+
62+
if you share variables between cells in your test blocks, this causes
63+
trouble during `nbdev_prepare` as it tries to run cells in isolation,
64+
leading to e.g.
65+
66+
NameError: name 'example_properties_schema' is not defined
67+
68+
to deal with this, you can merge cells that use a common variable
69+
70+
### updating package dependencies
71+
72+
note that package dependencies are specified in
73+
[settings.ini](./settings.ini); you shouldn’t be editing `setup.py` by
74+
hand. To add a requirement, add it to the `requuirements` entry in
75+
`settings.ini`, then run `python setup.py install`

‎flake.nix‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@
2222
# (whacked-setup + /bash/node_shortcuts.sh)
2323
];
2424
} {
25+
nativeBuildInputs = [
26+
pkgs.quarto
27+
];
28+
2529
buildInputs = [
2630
pkgs.python3
2731
];

‎nbs/00_core.ipynb‎

Lines changed: 58 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -367,7 +367,17 @@
367367
"cell_type": "code",
368368
"execution_count": null,
369369
"metadata": {},
370-
"outputs": [],
370+
"outputs": [
371+
{
372+
"name": "stderr",
373+
"output_type": "stream",
374+
"text": [
375+
"$:\t'string_value_with_enum' is a required property\n",
376+
"$:\t'MY_INTEGER_VALUE' is a required property\n",
377+
"$:\t'A_NUMERIC_VALUE' is a required property\n"
378+
]
379+
}
380+
],
371381
"source": [
372382
"#| hide\n",
373383
"test_fail(ConfigValidator.load_validated_config, args=(example_properties_schema, {}))"
@@ -415,7 +425,15 @@
415425
"cell_type": "code",
416426
"execution_count": null,
417427
"metadata": {},
418-
"outputs": [],
428+
"outputs": [
429+
{
430+
"name": "stderr",
431+
"output_type": "stream",
432+
"text": [
433+
"$.string_value_with_enum:\t'blah-blah' is not one of ['it', 'can', 'only', 'be', 'one', 'of', 'these']\n"
434+
]
435+
}
436+
],
419437
"source": [
420438
"#| hide\n",
421439
"test_fail(ConfigValidator.load_validated_config,\n",
@@ -431,7 +449,15 @@
431449
"cell_type": "code",
432450
"execution_count": null,
433451
"metadata": {},
434-
"outputs": [],
452+
"outputs": [
453+
{
454+
"name": "stderr",
455+
"output_type": "stream",
456+
"text": [
457+
"$.MY_INTEGER_VALUE:\t'5555.999' is not of type 'integer'\n"
458+
]
459+
}
460+
],
435461
"source": [
436462
"#| hide\n",
437463
"test_fail(ConfigValidator.load_validated_config,\n",
@@ -447,7 +473,15 @@
447473
"cell_type": "code",
448474
"execution_count": null,
449475
"metadata": {},
450-
"outputs": [],
476+
"outputs": [
477+
{
478+
"name": "stderr",
479+
"output_type": "stream",
480+
"text": [
481+
"$.A_NUMERIC_VALUE:\t'WHAT???' is not of type 'number'\n"
482+
]
483+
}
484+
],
451485
"source": [
452486
"#| hide\n",
453487
"test_fail(ConfigValidator.load_validated_config,\n",
@@ -463,7 +497,15 @@
463497
"cell_type": "code",
464498
"execution_count": null,
465499
"metadata": {},
466-
"outputs": [],
500+
"outputs": [
501+
{
502+
"name": "stderr",
503+
"output_type": "stream",
504+
"text": [
505+
"$.A_NUMERIC_VALUE:\t13.0 is less than the minimum of 22\n"
506+
]
507+
}
508+
],
467509
"source": [
468510
"#| hide\n",
469511
"test_fail(ConfigValidator.load_validated_config,\n",
@@ -475,6 +517,16 @@
475517
"}))"
476518
]
477519
},
520+
{
521+
"cell_type": "code",
522+
"execution_count": null,
523+
"metadata": {},
524+
"outputs": [],
525+
"source": [
526+
"#| hide\n",
527+
"from fs.memoryfs import MemoryFS"
528+
]
529+
},
478530
{
479531
"cell_type": "code",
480532
"execution_count": null,
@@ -484,7 +536,6 @@
484536
"#| hide\n",
485537
"# test ability to override the storage driver (memoryfs here)\n",
486538
"\n",
487-
"from fs.memoryfs import MemoryFS\n",
488539
"memfs = MemoryFS()\n",
489540
"\n",
490541
"memfs.makedirs('extra-long-directory-place', recreate=True)\n",
@@ -637,21 +688,9 @@
637688
],
638689
"metadata": {
639690
"kernelspec": {
640-
"display_name": "Python 3 (ipykernel)",
691+
"display_name": "python3",
641692
"language": "python",
642693
"name": "python3"
643-
},
644-
"language_info": {
645-
"codemirror_mode": {
646-
"name": "ipython",
647-
"version": 3
648-
},
649-
"file_extension": ".py",
650-
"mimetype": "text/x-python",
651-
"name": "python",
652-
"nbconvert_exporter": "python",
653-
"pygments_lexer": "ipython3",
654-
"version": "3.10.8"
655694
}
656695
},
657696
"nbformat": 4,

‎nbs/01_cli.ipynb‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
"import argparse\n",
4040
"import dotenv\n",
4141
"from typing import Union\n",
42-
"from fastcore.script import *"
42+
"from fastcore.script import call_parse, anno_parser"
4343
]
4444
},
4545
{
@@ -133,8 +133,25 @@
133133
"outputs": [],
134134
"source": [
135135
"#| export\n",
136+
" \n",
137+
"import importlib\n",
138+
"\n",
139+
"_self_module = importlib.import_module(\n",
140+
" \".\",\n",
141+
" __name__.split('.')[0] # module_name\n",
142+
")\n",
143+
"VERSION = getattr(_self_module, '__version__', 'NOT-IN-MODULE') # fails in notebook, works in module\n",
144+
"EXECUTABLE_NAME = 'schematized-config'\n",
145+
"\n",
146+
"def _hack_docstring(func):\n",
147+
" # hack the docstring to inject the version\n",
148+
" # the docstring gets rendered as the second line in the CLI help,\n",
149+
" # but it doesn't simply take an f-string, so we hack it in\n",
150+
" func.__doc__ = f\"{_self_module.__name__}: {func.__doc__} (v{VERSION})\"\n",
151+
" return func\n",
136152
"\n",
137153
"@call_parse\n",
154+
"@_hack_docstring\n",
138155
"def main(\n",
139156
" generate: str = None, # path to a json schema that validates a dotenv\n",
140157
" schema: str = None, # path to json schema used for validation\n",
@@ -146,7 +163,9 @@
146163
" sys.stdout.write(generate_sample_dotenv(generate))\n",
147164
" elif schema and validate:\n",
148165
" dotenv_path = validate\n",
149-
" validate_env(schema, dotenv_path)"
166+
" validate_env(schema, dotenv_path)\n",
167+
" else:\n",
168+
" anno_parser(main, EXECUTABLE_NAME).print_help()\n"
150169
]
151170
},
152171
{

‎nbs/index.ipynb‎

Lines changed: 49 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -39,45 +39,78 @@
3939
"cell_type": "markdown",
4040
"metadata": {},
4141
"source": [
42-
"# development"
42+
"## usage"
4343
]
4444
},
4545
{
46-
"cell_type": "markdown",
46+
"cell_type": "code",
47+
"execution_count": null,
4748
"metadata": {},
49+
"outputs": [],
4850
"source": [
49-
"assuming you have [nix](https://nixos.org/download.html) installed and ready, make sure `nix-command` and `flake` are enabled (oneliner: run `export NIX_CONFIG=\"experimental-features = nix-command flakes\"` in the terminal), then enter the dev shell using `nix develop`\n",
51+
"from schematized_config.core import ConfigValidator"
52+
]
53+
},
54+
{
55+
"cell_type": "code",
56+
"execution_count": null,
57+
"metadata": {},
58+
"outputs": [],
59+
"source": [
60+
"import os\n",
61+
"import os.path as _p\n",
5062
"\n",
51-
"start the jupyter notebook using the provided alias or just `jupyter notebook`, and hack away"
63+
"if _p.exists('.env') and 'CONFIG_VALIDATOR_JSON_SCHEMA' in os.environ:\n",
64+
" ConfigValidator.load_dotenv() # use defaults of .env and CONFIG_VALIDATOR_JSON_SCHEMA"
5265
]
5366
},
5467
{
5568
"cell_type": "markdown",
5669
"metadata": {},
5770
"source": [
58-
"## How to use"
71+
"# development"
5972
]
6073
},
6174
{
62-
"cell_type": "code",
63-
"execution_count": null,
75+
"cell_type": "markdown",
6476
"metadata": {},
65-
"outputs": [],
6677
"source": [
67-
"from schematized_config.core import ConfigValidator"
78+
"<details>\n",
79+
" <summary>entering the development environment</summary>\n",
80+
"assuming you have [nix](https://nixos.org/download.html) installed and ready, make sure `nix-command` and `flake` are enabled (oneliner: run `export NIX_CONFIG=\"experimental-features = nix-command flakes\"` in the terminal), then enter the dev shell using `nix develop`\n",
81+
"\n",
82+
"start the jupyter notebook using the provided alias or just `jupyter notebook`, and hack away\n",
83+
"</details>"
6884
]
6985
},
7086
{
71-
"cell_type": "code",
72-
"execution_count": null,
87+
"cell_type": "markdown",
7388
"metadata": {},
74-
"outputs": [],
7589
"source": [
76-
"import os\n",
77-
"import os.path as _p\n",
90+
"## nbdev\n",
7891
"\n",
79-
"if _p.exists('.env') and 'CONFIG_VALIDATOR_JSON_SCHEMA' in os.environ:\n",
80-
" ConfigValidator.load_dotenv() # use defaults of .env and CONFIG_VALIDATOR_JSON_SCHEMA"
92+
"this package is developed using [nbdev](https://nbdev.fast.ai/), so we use an nbdev-centric development flow. For a quick guide, we recommend checking out the [end-to-end walkthrough](https://nbdev.fast.ai/tutorials/tutorial.html). But in short: edit notebooks, then run the `nbdev_*` management commands. The most essential flow is as follows:\n",
93+
"\n",
94+
"1. edit the notebook files (core fore core, and cli for the command line interface)\n",
95+
"1. `python setup.py install` # note we don't use `nbdev_install` because we manage `quarto` using `nix`\n",
96+
"1. `nbdev_prepare`\n",
97+
"1. run code using the package\n",
98+
"1. `nbdev_release`\n",
99+
"1. `nbdev_pypi`\n",
100+
"\n",
101+
"### running tests\n",
102+
"\n",
103+
"if you share variables between cells in your test blocks, this causes trouble during `nbdev_prepare` as it tries to run cells in isolation, leading to e.g.\n",
104+
"\n",
105+
"```\n",
106+
"NameError: name 'example_properties_schema' is not defined\n",
107+
"```\n",
108+
"\n",
109+
"to deal with this, you can merge cells that use a common variable\n",
110+
"\n",
111+
"### updating package dependencies\n",
112+
"\n",
113+
"note that package dependencies are specified in [settings.ini](./settings.ini); you shouldn't be editing `setup.py` by hand. To add a requirement, add it to the `requuirements` entry in `settings.ini`, then run `python setup.py install`"
81114
]
82115
}
83116
],

‎schematized_config/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.0.5"
1+
__version__ = "0.0.6"

‎schematized_config/_modidx.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
'doc_host': 'https://tutankalex.github.io',
66
'git_url': 'https://github.com/tutankalex/python-schematized-config',
77
'lib_path': 'schematized_config'},
8-
'syms': { 'schematized_config.cli': { 'schematized_config.cli.generate_sample_dotenv': ( 'cli.html#generate_sample_dotenv',
8+
'syms': { 'schematized_config.cli': { 'schematized_config.cli._hack_docstring': ('cli.html#_hack_docstring', 'schematized_config/cli.py'),
9+
'schematized_config.cli.generate_sample_dotenv': ( 'cli.html#generate_sample_dotenv',
910
'schematized_config/cli.py'),
1011
'schematized_config.cli.main': ('cli.html#main', 'schematized_config/cli.py'),
1112
'schematized_config.cli.validate_env': ('cli.html#validate_env', 'schematized_config/cli.py')},

0 commit comments

Comments
 (0)