Skip to content

Commit 13b191f

Browse files
loriabloriab
authored
install and conda envs (#4)
* install and conda envs * faq * conda docs * fix code syntax
1 parent aff21c6 commit 13b191f

File tree

5 files changed

+180
-31
lines changed

5 files changed

+180
-31
lines changed

‎README.md‎

Lines changed: 130 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,25 +4,42 @@ PandaModelsBackend provides a high-fidelity backend for [Grid2op](https://github
44

55
<img src="https://github.com/gt-sse-center/PandaModelsBackend/blob/main/devtools/power_software_map.png" height=450>
66

7-
## Installation
7+
## Installation & Usage
8+
89
```pip install pandamodelsbackend```
910

10-
## Refer to [tests/test_basic_usage.py]()
11+
Refer to [test_basic_usage](tests/test_basic_usage.py) for full independently runable example.
1112

1213
```
14+
import grid2op
15+
import pandapower as pp
1316
from pandamodelsbackend import PandaModelsBackend
1417
18+
...
1519
pp_net = pp.from_json(network_file) # Load PandaPower JSON
16-
.
17-
.
1820
backend = PandaModelsBackend(pp_net)
19-
env = grid2op.make(env_name, backend=backend)
20-
.
21+
env = grid2op.make(env_name, backend=backend)
2122
```
2223

23-
## Setting up dev environment
24+
-----
25+
26+
## Setting up the Development Environment
27+
28+
For newcomers to Julia, setting up a software stack with compatible Python and Julia is perhaps the trickiest part of using PandaModelsBackend.
29+
Like Python, one can get Julia through an installer provided by the language developers or through a package manager, like conda.
30+
Both are shown below and are demonstrated (sort-of, for installers) in [GitHub Actions CI](.github/workflows/ci.yml).
31+
You'll need to separately install certain dependencies into Python and others into Julia. You never need to enter the Julia REPL.
32+
If you get tangled up, the CI file is a good, proven route to consult.
33+
[Docs at pandapower](https://pandapower.readthedocs.io/en/v2.6.0/opf/powermodels.html) may also be helpful.
34+
There are a lot of acceptable ways to get Python+Julia working, so as long as [`python-jl tests/test_backend_api.py`](tests/test_backend_api.py) is working at the end, you're probably good to run grid2op.
35+
36+
##### Notes
2437

25-
### Using python virtual environment
38+
* This software stack won't work on M1 Macs.
39+
* Conda can be used for Python and Julia for Linux and Intel Macs. There isn't a `julia` package for Silicon Macs, so you'll need to use the Julia installer (+ conda or installer Python).
40+
* A few problems and solutions have been collected [here](#troubleshooting-the-software-stack).
41+
42+
#### Using a Python virtual environment
2643

2744
- Create a python virtual environment
2845
```python3 -m venv venv```
@@ -36,7 +53,7 @@ env = grid2op.make(env_name, backend=backend)
3653
```pip install grid2op pandapower```
3754
```pip install julia```
3855

39-
- Julia Installation refer to [this](https://docs.julialang.org/en/v1/manual/installation/)]
56+
- Julia Installation refer to [this](https://docs.julialang.org/en/v1/manual/installation/)
4057

4158
- set your path variable to find Julia e.g., on MAC
4259
```export PATH=/Applications/Julia-1.11.app/Contents/Resources/julia/bin:$PATH```
@@ -52,11 +69,61 @@ env = grid2op.make(env_name, backend=backend)
5269

5370
On MacOS:
5471
```ENV["PYTHON"]="<your python path>"```
55-
72+
5673
Access the package manager again in julia by typing ]. Now install the packages: ```add PyCall```. To pass the python environment variable, running build PyCall inside the julia package manager may be necessary.
5774

58-
### Using Conda
59-
[TODO]
75+
#### Using Conda
76+
77+
0. copy a conda env spec file like [unix.yaml](devtools/conda-envs/unix.yaml). Customize python version, environment name, etc.
78+
79+
```
80+
# 1. create a new conda env from the spec.
81+
conda env create -f unix.yaml && conda activate test
82+
83+
# 2. install editable PandaModelsBackend
84+
git clone https://github.com/gt-sse-center/PandaModelsBackend.git && cd PandaModelsBackend
85+
pip install -e .
86+
87+
# 3. install PowerModels into Julia
88+
# equivalent to entering pkg mode of Julia REPL, adding packages, and exiting REPL via:
89+
# julia <Enter> ] <Enter> add Ipopt PowerModels PyCall <Enter> # await compilation <Ctrl-D>
90+
julia -e 'using Pkg; Pkg.add(["Ipopt", "PowerModels", "PandaModels", "PyCall"])'
91+
92+
# 4. (optional) check languages and power projects installed. $CONDA_PREFIX is placeholder, not literal
93+
which python julia python-jl
94+
#> $CONDA_PREFIX/bin/python
95+
#> $CONDA_PREFIX/bin/julia
96+
#> $CONDA_PREFIX/bin/python-jl
97+
98+
conda list | grep -e power -e grid -e panda
99+
#>grid2op 1.10.5.post1 pypi_0 pypi
100+
#>pandamodelsbackend 0.1.1 pypi_0 pypi
101+
#>pandapower 2.14.9 pyhd8ed1ab_1 conda-forge
102+
#>pandas 2.2.3 py312hf9745cd_1 conda-forge
103+
104+
# equivalent to `julia <Enter> ] <Enter> status <Enter> # observe <Ctrl-D>
105+
julia -e 'using Pkg; Pkg.status(); Pkg.status(outdated=true)'
106+
#>Status `$CONDA_PREFIX/share/julia/environments/test/Project.toml`
107+
#>⌅ [b6b21f68] Ipopt v0.9.1
108+
#> [2dbab86a] PandaModels v0.7.3
109+
#>⌅ [c36e90e8] PowerModels v0.19.10
110+
#> [438e738f] PyCall v1.96.4
111+
#>Status `$CONDA_PREFIX/share/julia/environments/test/Project.toml`
112+
#>⌅ [b6b21f68] Ipopt v0.9.1 (<v1.7.2): PandaModels
113+
#>⌅ [c36e90e8] PowerModels v0.19.10 (<v0.21.3): PandaModels
114+
115+
# 5. (optional) check Python, Julia, PandaPower functioning together
116+
julia -e 'using PyCall; math = pyimport("math"); print(math.sin(math.pi/4))'
117+
#>0.7071067811865475
118+
pyMm=$(python -c "import sys;print(f'{sys.version_info.major}.{sys.version_info.minor}')")
119+
python-jl $CONDA_PREFIX/lib/python$pyMm/site-packages/pandapower/test/opf/test_pandamodels_runpm.py
120+
#>20 passed, 1 xpassed, 1226 warnings in 267.01s (0:04:27)
121+
python-jl tests/test_backend_api.py
122+
#>Ran 31 tests in 58.499s
123+
#>OK
124+
```
125+
126+
-----
60127

61128
## Build
62129
```
@@ -70,4 +137,55 @@ pip install -e .
70137
```
71138
cd tests
72139
python test_backend_api.py
140+
# depending on installation, `python-jl` may be needed instead of `python` above
73141
```
142+
143+
-----
144+
145+
## Release Procedure
146+
147+
* On a local clone on branch `main`, update the [CHANGELOG](changelog.md) with PRs, the new version number, and the release date. Commit it.
148+
* Make an annotated tag for the new version. Push it along with any cleanup commits (e.g., changelog above). If you've forked the repo, "origin" will probably be "upstream". See `git remote -v` for names.
149+
```
150+
git tag -a v0.5.0 -m "v0.5.0"
151+
git push --atomic origin main v0.5.0
152+
```
153+
* The CI workflow will take over publication to [PyPI](https://pypi.org/project/pandamodelsbackend/) and making a GitHub release. You can edit the frontmatter for the latter through the GitHub web interface with any particular details (and perhaps a link to the CHANGELOG section).
154+
155+
## Troubleshooting the Software Stack
156+
157+
1. **Problem:**
158+
Your Python interpreter "/path/to/miniconda/envs/rl2grid/bin/python"
159+
is statically linked to libpython. Currently, PyJulia does not fully
160+
support such Python interpreter.
161+
162+
**Solution:**
163+
Use `python-jl` from conda (or Julia installation) instead of `python`.
164+
165+
2. **Problem:**
166+
A pip installation involves compiling, and system compilers aren't working.
167+
168+
**Solution:**
169+
`conda install cxx-compiler -c conda-forge` and rerun pip.
170+
171+
3. **Problem:**
172+
AttributeError in pandapower file d2Sbus_dV2.
173+
174+
**Solution:**
175+
This was fixed between v2.14.9 and master for csr_matrix?. Alternately, edit the `diagV` to below:
176+
```
177+
/path/to/site-packages/pandapower/pypower/d2Sbus_dV2.py: try: # added
178+
/path/to/site-packages/pandapower/pypower/d2Sbus_dV2.py: D = Ybus.H * diagV # indented
179+
/path/to/site-packages/pandapower/pypower/d2Sbus_dV2.py: except AttributeError: # added
180+
/path/to/site-packages/pandapower/pypower/d2Sbus_dV2.py: D = Ybus.getH() * diagV # added
181+
```
182+
183+
4. **Problem:**
184+
Package testing for PowerModels, PandaModels, or PandaPower shows a few (not all) failures.
185+
186+
**Solution:**
187+
For a clean test suite, PowerModels needs Ipopt>=1 and the others need Ipopt<1. To toggle, do
188+
something like the below. See [CI](.github/workflows/ci.yml) for a known working testing sequence.
189+
```
190+
julia -e 'using Pkg; Pkg.add(Pkg.PackageSpec(;name="Ipopt", version="0.9")); using Ipopt'
191+
```

‎changelog.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@
2121
#### Breaking Changes
2222

2323
#### New Features
24+
* [\#4] (https://github.com/gt-sse-center/PandaModelsBackend/pull/4) Maint --
25+
added Conda install notes, refined dependencies.
2426

2527
#### Enhancements
2628

@@ -29,7 +31,7 @@
2931
#### Misc.
3032

3133

32-
## v0.1.1 / 2025-MM-DD (Unreleased)
34+
## v0.1.1 / 2025-3-12
3335

3436
#### Breaking Changes
3537

‎devtools/conda-envs/unix.yaml‎

Lines changed: 13 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,26 @@
1+
# This is the minimal environment for PandaModelsBackend plus setup for Julia.
2+
# See unix_rl2grid.yaml for an env spec accommodating rl2grid.
3+
14
name: test
25
channels:
36
- conda-forge
47
dependencies:
5-
- python=3.12
6-
- pytorch
7-
- tensorboard
8-
- wandb
9-
- gymnasium>=0.29.1
10-
- stable-baselines3>=2.3.2
8+
- python
9+
- numpy=1 # restriction from pandapower
1110
- scipy
12-
- pyjulia # brings julia
11+
- pandas
12+
- pyjulia # brings julia
1313
- pandapower=2
1414
- pytest-xdist # optional, for testing
15-
- curl<8.10 # avoid segfault in Julia w/ curl v8.10 . recheck periodically
15+
- curl<8.10 # avoid segfault in Julia w/ curl v8.10 . recheck periodically
16+
- setuptools
1617
- pip
1718
- pip:
1819
- grid2op
19-
- lightsim2grid # optional
20-
- matpowercaseframes # optional
21-
# - cxx-compiler # optional if needed for further pip packages
20+
# - lightsim2grid # optional
21+
# - matpowercaseframes # optional
22+
# - cxx-compiler # optional if needed for further pip packages requiring compilation
2223

2324
# Notes:
24-
# * defaults and pytorch channels removed as likely unneeded.
25-
# * cpuonly removed b/c not a c-f pkg
2625
# * py312 has been tested. others likely work
27-
# * pytorch was pinned at 2.2.0 and scipy at 1.11.3 . reset if needed for repro, but generic better.
28-
# * recent stable-baselines3 can accommodate gymnasium v1, so pins relaxed to mins. reset if needed for repro. etc.
29-
26+
# * works as-is for Linux and Intel Mac. conda packages for Julia aren't available for Silicon Mac.

‎devtools/conda-envs/unix_rl2grid.yaml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Whereas unix.yaml is the minimal environment for PandaModelsBackend, this conda env spec is derived
2+
# from that used by the Donti lab to include rl2grid in the codestack.
3+
4+
name: rl2grid_pm
5+
channels:
6+
- conda-forge
7+
dependencies:
8+
- python=3.12
9+
- pytorch
10+
- tensorboard
11+
- wandb
12+
- gymnasium>=0.29.1
13+
- stable-baselines3>=2.3.2
14+
- scipy
15+
- pyjulia # brings julia
16+
- pandapower=2
17+
- pytest-xdist # optional, for testing
18+
- curl<8.10 # avoid segfault in Julia w/ curl v8.10 . recheck periodically
19+
- pip
20+
- pip:
21+
- grid2op
22+
- lightsim2grid # optional
23+
- matpowercaseframes # optional
24+
# - cxx-compiler # optional if needed for further pip packages requiring compilation
25+
26+
# Notes:
27+
# * defaults and pytorch channels removed as likely unneeded.
28+
# * cpuonly removed b/c not a c-f pkg
29+
# * py312 has been tested. others likely work
30+
# * pytorch was pinned at 2.2.0 and scipy at 1.11.3 . reset if needed for repro, but generic better.
31+
# * recent stable-baselines3 can accommodate gymnasium v1, so pins relaxed to mins. reset if needed for repro. etc.
32+

‎pyproject.toml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,10 @@ classifiers = [
1818
"Programming Language :: Python :: 3.13",
1919
]
2020
dependencies = [
21-
"numpy",
21+
"numpy<2.0", # restriction for pandapower
2222
"scipy",
2323
"pandas",
24-
"pandapower",
24+
"pandapower<3.0",
2525
# packaging?
2626
"grid2op",
2727
]

0 commit comments

Comments
 (0)