Merge pull request #88 from staadecker/improve_load_aug

Improve loading inputs and docs

Author: pesap

README.md

@@ -3,21 +3,41 @@
 Welcome! This repository contains the SWITCH electricity planning model adapted for the
 REAM research lab.
 
-For **an overview** of what SWITCH is and how it works, read [`docs/Overview.md`](./docs/Overiew.md).
+## Available documentation
 
-To discover **how to install, run or debug** the different components of Switch, read [`docs/Usage.md`](./docs/Usage.md)
+In `docs/`:
 
-To **see examples** of smaller SWITCH models, see the `examples/` folder.
+- [`Overview.md`](./docs/Overiew.md): An overview of what SWITCH is and how it works, read
 
-To discover **how to contribute to the model**, read [`docs/Contribute.md`](/docs/Contribute.md)
+- [`Usage.md`](./docs/Usage.md): How to install, run or debug the different components of Switch
 
-To **learn about our database** (e.g. how to connect to it and modify it), read [`docs/Database.md`](/docs/Database.md).
+- [`Developing Modules.md`](./docs/Developing%20Modules.md): How to create SWITCH modules from scratch
 
-To **generate documentation**, run `pydoc -w switch_model` after having installed
+- [`Contribute.md`](/docs/Contribute.md): How to contribute code to the project.
+
+- [`Graphs`](/docs/Graphs.md): How to create new graphs to analyze your results.
+
+- [`Database.md`](/docs/Database.md): All about the REAM database (e.g. how to connect to it and modify it)
+
+- [`Numerical Solvers.md`](/docs/Numerical%20Solvers.md): Information about numerical solvers, specifically Gurobi.
+
+- [`Numerical Issues.md`](/docs/Numerical%20Issues.md): Information about detecting and resolving numerical issues.
+
+Finally, you can generate documentation for the SWITCH modules by running `pydoc -w switch_model` after having installed
 SWITCH. This will build HTML documentation files from python doc strings which
 will include descriptions of each module, their intentions, model
 components they define, and what input files they expect.
 
-To learn about **numerical solvers** read [`docs/Numerical Solvers.md`](/docs/Numerical%20Solvers.md)
+## Key folders
+
+- [`/database`](/database) Folder containing SQL scripts and files that keep track of updates to our PostgreSQL database.
+
+- [`/examples`](/examples) Folder with numerous examples of SWITCH projects often used for testing.
+
+- [`/switch_model`](/switch_model) Folder containing all the source code for the SWITCH modules.
+
+- [`/switch_model/wecc`](/switch_model/wecc) Folder containing modules specific to the REAM team.
+
+- [`/switch_model/wecc/get_inputs`](/switch_model/wecc/get_inputs) Scripts that fetch the input data from the PostgreSQL database.
 
-To learn about **numerical issues** read [`docs/Numerical Issues.md`](/docs/Numerical%20Issues.md)
+- [`/tests`](/tests) Folder containing tests for SWITCH. Can be run via `python run_tests.py`.

docs/Contribute.md

@@ -1,6 +1,7 @@
 # Contributing Code
 
-This document describes the best practices for contributing code.
+You've made changes to the codebase and now you want to share them
+with the rest of the team! Here are the best practices for the process.
 
 ## The process
 
@@ -39,43 +40,6 @@ supposed to alter the results of the examples, you'll need
 to follow the instructions that appear on screen to suppress the errors
 produced by `python run_tests.py`.
 
-## Contributing graphs
+## Important notes
 
-Read [`docs/Graphs.md`](./Graphs.md) to see learn to add graphs.
-
-## Modifying the database
-
-Read [`docs/Database.md`](./Database.md) to learn about the database.
-
-## Outputting results
-
-Once the model is solved, the `post_solve()` function in each module is called.
-Within the `post_solve()` function you may 
-
-- Call `write_table()` to create
-a .csv file with data from the solution (see existing modules for examples).
-  
-- Call `add_info()` (from `utilities/result_info.py`) to add a line 
-of information to the `outputs/info.txt` file. `add_info()` can also be added to `graph()`.
-
-### Example
-
-```python
-from switch_model.utilities.results_info import add_info
-from switch_model.reporting import write_table
-import os
-...
-def post_solve(instance, outdir):
-    ...
-    # This will add the a line to info.txt in the outputs folder
-    add_info("Some important value", instance.important_value)
-    ...
-    # This will create my_table.csv
-    write_table(
-        instance, 
-        instance.TIMEPOINTS, # Set that the values function will iterate over
-        output_file=os.path.join(outdir, "my_table.csv"),
-        headings=("timepoint", "some value"),
-        values=lambda m, t: (t, m.some_value[t])
-    )
-```
+- If your change is modifying the database, make sure you've read [`Database.md`](./Database.md).

docs/Developing Modules.md

@@ -0,0 +1,95 @@
+## Developing modules
+
+Modules are the core elements of SWITCH. Each module defines a specific functionality for the model. For example,
+the `switch_model.generators.core.build` defines how generators are allowed to be built while `switch_model.timescales`
+defines how time is handled in SWITCH.
+
+There are 3 important parts to any module.
+
+1. `define_components(model)`: This function specifies the Pyomo Sets, Parameters, Variables and Expressions for the module
+   as well as the input files that should be used. It's the first thing that gets called when you run `switch solve` as
+   it creates the Pyomo model.
+
+2. `post_solve()`: This function gets run after the solver has found the optimal solution. This is where you can output
+   results to e.g. a csv file.
+
+3. Functions with the `@graph(...)` decorator. These functions get called last and are responsible for creating graphs
+   for analysis using the data from the csv files you created in `post_solve()`.
+
+There are also a few other components that you may encounter:
+
+- `load_inputs()`: This function is the old way of loading inputs from .csv files into the model. It gets called right
+  after `define_components()`. Now the prefered way of loading inputs is by using `input_file=`
+  (see next section for details).
+  
+- `define_dynamic_lists(model)` and `define_dynamic_components(model)`: 
+Some modules need to define objects to be shared across multiple modules. 
+  The best example of this is `switch_model.balancing.load_zones` which
+  allows different modules to add elements to a dynamic list that is defined
+  in `define_dynamic_lists`. Then in `define_dynamic_components` it defines
+  the energy balance constraint using the dynamic list. See the module for details.
+
+## Writing `define_components()`
+
+`define_components(mod)` takes in the model as an argument and is responsible
+for adding constraints, expressions, variables, sets or parameters to the model.
+
+Sometimes Sets or Parameters should be initialized from an input csv file.
+If this is the case, add `input_file=` (and optionally `input_column`) to the
+set or parameter definition.
+
+For example the following code snippet defines a Set and a parameter
+indexed over that set. Both the set and parameter are initialized from
+the `input.csv` file.
+
+```python
+from pyomo.environ import *
+
+
+def define_components(mod):
+    mod.SetA = Set(
+        dimen=2,
+        input_file="input.csv",
+    )
+
+    mod.some_indexed_param = Param(
+        mod.SetA,
+        input_file="input.csv",
+        input_column="param1"  # only specify when the name of the column in the csv is not the same as the component name
+    )
+```
+
+## Writing `post_solve()`
+
+Once the model is solved, the `post_solve()` function in each module is called. Within the `post_solve()` function you
+may
+
+- Call `write_table()` to create a .csv file with data from the solution (see existing modules for examples).
+
+- Call `add_info()` (from `utilities/result_info.py`) to add a line of information to the `outputs/info.txt`
+  file. `add_info()` can also be added to `graph()`.
+
+### Example
+
+```python
+from switch_model.utilities.results_info import add_info
+from switch_model.reporting import write_table
+import os
+
+...
+
+
+def post_solve(instance, outdir):
+    ...
+    # This will add the a line to info.txt in the outputs folder
+    add_info("Some important value", instance.important_value)
+    ...
+    # This will create my_table.csv
+    write_table(
+        instance,
+        instance.TIMEPOINTS,  # Set that the values function will iterate over
+        output_file=os.path.join(outdir, "my_table.csv"),
+        headings=("timepoint", "some value"),
+        values=lambda m, t: (t, m.some_value[t])
+    )
+```

switch_model/balancing/demand_response/simple.py

@@ -8,6 +8,11 @@
 demand shifting. This does not include a Shed Service (curtailment of load),
 nor a Shimmy Service (fast dispatch for load following or regulation).
 
+INPUT FILE FORMAT
+    Import demand response-specific data from an input directory.
+
+    dr_data.csv
+        LOAD_ZONE, TIMEPOINT, dr_shift_down_limit, dr_shift_up_limit
 """
 
 import os
@@ -53,11 +58,21 @@ def define_components(mod):
         mod.TIMEPOINTS,
         default=0.0,
         within=NonNegativeReals,
+<<<<<<< HEAD
         validate=lambda m, value, z, t: value <= m.zone_demand_mw[z, t],
     )
     mod.dr_shift_up_limit = Param(
         mod.LOAD_ZONES, mod.TIMEPOINTS, default=float("inf"), within=NonNegativeReals
     )
+=======
+        input_file="dr_data.csv",
+        validate=lambda m, value, z, t: value <= m.zone_demand_mw[z, t])
+    mod.dr_shift_up_limit = Param(
+        mod.LOAD_ZONES, mod.TIMEPOINTS,
+        default= float('inf'),
+        input_file="dr_data.csv",
+        within=NonNegativeReals)
+>>>>>>> 941c8e2e (Further improvements)
     mod.ShiftDemand = Var(
         mod.LOAD_ZONES,
         mod.TIMEPOINTS,
@@ -77,6 +92,7 @@ def define_components(mod):
     try:
         mod.Distributed_Power_Withdrawals.append("ShiftDemand")
     except AttributeError:
+<<<<<<< HEAD
         mod.Zone_Power_Withdrawals.append("ShiftDemand")
 
 
@@ -96,3 +112,6 @@ def load_inputs(mod, switch_data, inputs_dir):
         autoselect=True,
         param=(mod.dr_shift_down_limit, mod.dr_shift_up_limit),
     )
+=======
+        mod.Zone_Power_Withdrawals.append('ShiftDemand')
+>>>>>>> 941c8e2e (Further improvements)

switch_model/balancing/electric_vehicles/simple.py

@@ -11,6 +11,13 @@
 of each timeseries they must be charged at some specific level according
 to users necessity.
 
+INPUT FILE FORMAT
+    Import virtual batteries specific location and power limits
+    from an input directory.
+
+    ev_limits.tab
+        LOAD_ZONES, TIMEPOINT, ev_cumulative_charge_upper_mwh,
+        ev_cumulative_charge_upper_mwh, ev_charge_limit_mw
 """
 
 import os
@@ -66,6 +73,7 @@ def define_components(mod):
     """
 
     mod.ev_charge_limit_mw = Param(
+<<<<<<< HEAD
         mod.LOAD_ZONES, mod.TIMEPOINTS, default=float("inf"), within=NonNegativeReals
     )
 
@@ -76,6 +84,24 @@ def define_components(mod):
     mod.ev_cumulative_charge_lower_mwh = Param(
         mod.LOAD_ZONES, mod.TIMEPOINTS, default=0.0, within=NonNegativeReals
     )
+=======
+        mod.LOAD_ZONES, mod.TIMEPOINTS,
+        default = float('inf'),
+        input_file="ev_limits.csv",
+        within=NonNegativeReals)
+
+    mod.ev_cumulative_charge_upper_mwh = Param(
+        mod.LOAD_ZONES, mod.TIMEPOINTS,
+        default = 0.0,
+        input_file="ev_limits.csv",
+        within=NonNegativeReals)
+
+    mod.ev_cumulative_charge_lower_mwh = Param(
+        mod.LOAD_ZONES, mod.TIMEPOINTS,
+        default = 0.0,
+        input_file="ev_limits.csv",
+        within=NonNegativeReals)
+>>>>>>> 941c8e2e (Further improvements)
 
     mod.EVCharge = Var(
         mod.LOAD_ZONES,
@@ -111,6 +137,7 @@ def define_components(mod):
     if "Distributed_Power_Injections" in dir(mod):
         mod.Distributed_Power_Withdrawals.append("EVCharge")
     else:
+<<<<<<< HEAD
         mod.Zone_Power_Withdrawals.append("EVCharge")
 
 
@@ -136,3 +163,6 @@ def load_inputs(mod, switch_data, inputs_dir):
             mod.ev_charge_limit_mw,
         ),
     )
+=======
+        mod.Zone_Power_Withdrawals.append('EVCharge')
+>>>>>>> 941c8e2e (Further improvements)