diff --git a/doc/OnlineDocs/explanation/analysis/alternative_solutions.rst b/doc/OnlineDocs/explanation/analysis/alternative_solutions.rst index 899db8e8757..6a7938320c3 100644 --- a/doc/OnlineDocs/explanation/analysis/alternative_solutions.rst +++ b/doc/OnlineDocs/explanation/analysis/alternative_solutions.rst @@ -2,48 +2,57 @@ Generating Alternative (Near-)Optimal Solutions ############################################### +.. py:currentmodule:: pyomo.contrib.alternative_solutions + Optimization solvers are generally designed to return a feasible solution to the user. However, there are many applications where a user needs more context than this result. For example, -* alternative solutions can support an assessment of trade-offs between +* alternative optimal solutions can be used to assess trade-offs between competing objectives; -* if the optimization formulation may be inaccurate or untrustworthy, - then comparisons amongst alternative solutions provide additional - insights into the reliability of these model predictions; or +* comparisons amongst alternative solutions provide + insights into the efficacy of model predictions with inaccurate or + untrusted optimization formulations; or + +* alternative optimal solutions create an opportunity to understand a + design space, including assessments of unexpressed objectives and + constraints; -* the user may have unexpressed objectives or constraints, which only - are realized in later stages of model analysis. +* alternative solutions can be identified to support the future + analysis of model revisions (e.g. to account for previously unexpressed + constraints). The *alternative-solutions library* provides a variety of functions that can be used to generate optimal or near-optimal solutions for a pyomo model. Conceptually, these functions are like pyomo solvers. They can -be configured with solver names and options, and they return a list of -solutions for the pyomo model. However, these functions are independent -of pyomo's solver interface because they return a custom solution object. +be configured with solver names and options, and they return a pool of +solutions for the pyomo model. However, these functions are independent of +pyomo's solver interfaces because they return a custom pool manager object. The following functions are defined in the alternative-solutions library: -* ``enumerate_binary_solutions`` +* :py:func:`enumerate_binary_solutions` * Finds alternative optimal solutions for a binary problem using no-good cuts. -* ``enumerate_linear_solutions`` +* :py:func:`enumerate_linear_solutions` - * Finds alternative optimal solutions for a (mixed-integer) linear program. + * Finds alternative optimal solutions for continuous variables in a + (mixed-integer) linear program using iterative solutions of an + integer programming formulation. -* ``enumerate_linear_solutions_soln_pool`` +* :py:func:`gurobi_enumerate_linear_solutions` * Finds alternative optimal solutions for a (mixed-binary) linear - program using Gurobi's solution pool feature. + program using Gurobi to generate lazy cuts. -* ``gurobi_generate_solutions`` +* :py:func:`gurobi_generate_solutions` * Finds alternative optimal solutions for discrete variables using Gurobi's built-in solution pool capability. -* ``obbt_analysis_bounds_and_solutions`` +* :py:func:`obbt_analysis_bounds_and_solutions` * Calculates the bounds on each variable by solving a series of min and max optimization problems where each variable is used as the @@ -51,20 +60,22 @@ The following functions are defined in the alternative-solutions library: supported by the selected solver. -Basic Usage Example -------------------- +A Simple Example +---------------- Many of the functions in the alternative-solutions library have similar -options, so we simply illustrate the ``enumerate_binary_solutions`` -function. We define a simple knapsack example whose alternative -solutions have integer objective values ranging from 0 to 90. +options, so we simply illustrate the :py:func:`enumerate_binary_solutions` +function. + +We define a simple knapsack example whose alternative +solutions have integer objective values ranging from 0 to 70. .. doctest:: >>> import pyomo.environ as pyo - >>> values = [10, 40, 30, 50] - >>> weights = [5, 4, 6, 3] + >>> values = [20, 10, 60, 50] + >>> weights = [5, 4, 6, 5] >>> capacity = 10 >>> m = pyo.ConcreteModel() @@ -72,8 +83,8 @@ solutions have integer objective values ranging from 0 to 90. >>> m.o = pyo.Objective(expr=sum(values[i] * m.x[i] for i in range(4)), sense=pyo.maximize) >>> m.c = pyo.Constraint(expr=sum(weights[i] * m.x[i] for i in range(4)) <= capacity) -We can execute the ``enumerate_binary_solutions`` function to generate a -list of ``Solution`` objects that represent alternative optimal +The function :py:func:`enumerate_binary_solutions` generates a +pool of :py:class:`Solution` objects that represent alternative optimal solutions: .. doctest:: @@ -81,154 +92,281 @@ solutions: >>> import pyomo.contrib.alternative_solutions as aos >>> solns = aos.enumerate_binary_solutions(m, num_solutions=100, solver="glpk") - >>> assert len(solns) == 10 + >>> assert len(solns) == 9 + >>> print( [soln.objective().value for soln in solns] ) + [70.0, 70.0, 60.0, 60.0, 50.0, 30.0, 20.0, 10.0, 0.0] -Each ``Solution`` object contains information about the objective and -variables, and it includes various methods to access this information. -For example: -.. doctest:: - :skipif: not glpk_available +Enumerating Near-Optimal Solutions +---------------------------------- - >>> print(solns[0]) - { - "fixed_variables": [], - "objective": "o", - "objective_value": 90.0, - "solution": { - "x[0]": 0, - "x[1]": 1, - "x[2]": 0, - "x[3]": 1 - } - } +The previous example enumerated all feasible solutions. However optimization models are typically +used to identify optimal or near-optimal solutions. The ``abs_opt_gap`` and ``rel_opt_gap`` +arguments are used to limit the search to these solutions: + +* ``rel_opt_gap`` : non-negative float or None + * The relative optimality gap for allowable alternative solutions. Specifying a gap of ``None`` indicates that there is no limit on the relative optimality gap (i.e. that any feasible solution can be considered). -Gap Usage Example ------------------ +* ``abs_opt_gap`` : non-negative float or None -When we only want some of the solutions based off a tolerance away from -optimal, this can be done using the ``abs_opt_gap`` parameter. This is -shown in the following simple knapsack examples where the weights and -values are the same. + * The absolute optimality gap for allowable alternative solutions. Specifying a gap of ``None`` indicates that there is no limit on the absolute optimality gap (i.e. that any feasible solution can be considered). + +For example, we can generate all optimal solutions as follows: .. doctest:: :skipif: not glpk_available - >>> import pyomo.environ as pyo - >>> import pyomo.contrib.alternative_solutions as aos + >>> solns = aos.enumerate_binary_solutions(m, num_solutions=100, solver="glpk", abs_opt_gap=0.0) + >>> print( [soln.objective().value for soln in solns] ) + [70.0, 70.0] + +Similarly, we can generate the six solutions within 40 of the optimum: - >>> values = [10,9,2,1,1] - >>> weights = [10,9,2,1,1] +.. doctest:: + :skipif: not glpk_available - >>> K = len(values) - >>> capacity = 12 + >>> solns = aos.enumerate_binary_solutions(m, num_solutions=100, solver="glpk", abs_opt_gap=40.0) + >>> print( [soln.objective().value for soln in solns] ) + [70.0, 70.0, 60.0, 60.0, 50.0, 30.0] - >>> m = pyo.ConcreteModel() - >>> m.x = pyo.Var(range(K), within=pyo.Binary) - >>> m.o = pyo.Objective(expr=sum(values[i] * m.x[i] for i in range(K)), sense=pyo.maximize) - >>> m.c = pyo.Constraint(expr=sum(weights[i] * m.x[i] for i in range(K)) <= capacity) - >>> solns = aos.enumerate_binary_solutions(m, num_solutions=10, solver="glpk", abs_opt_gap = 0.0) - >>> assert(len(solns) == 4) +Pyomo Solution Pools +-------------------- + +The *alternative-solutions library* uses solution pools to filter and store solutions generated by an optimizer. +The following types of solution pools are currently supported: -In this example, we only get the four ``Solution`` objects that have an -``objective_value`` of 12. Note that while we wanted only those four -solutions with no optimality gap, using a gap of half the smallest value -(in this case .5) will return the same solutions and avoids any machine -precision issues. +* ``keep_all`` : This pool stores all solutions. No solutions are filtered out. + +* ``keep_latest`` : This pool stores the latest ``max_pool_size`` solutions that are added to the pool. + + * ``max_pool_size`` (non-negative integer) : The maximum number of solutions that are stored. + +* ``keep_latest_unique`` : This pool stores the latest ``max_pool_size`` solutions, ignoring duplicate solutions. + + * ``max_pool_size`` (non-negative integer) : The maximum number of solutions that are stored. + +* ``keep_best`` : This pool stores the best solutions added to the pool. + + * ``max_pool_size`` (non-negative integer) : The maximum number of solutions that are stored. + + * ``objective`` (function) : A user-specified function that computes the objective value used for comparisons. + + * ``abs_tolerance`` (non-negative float) : The absolute tolerance that is used to filter solutions. + + * ``rel_tolernace`` (non-negative float) : The relative tolerance that is used to filter solutions. + + * ``sense_is_min`` (bool) : If True, then the pool will keep solutions with the minimal objective values. + + * ``best_value`` (float) : As solutions are added to this pool, it tracks the best solution value seen for tolerance comparisons. If specified, then this value provides an initial value for the best solution value. + +A pool manager class is used to manage one-or-more solution pools. This +allows for flexible collection of solutions with different criteria. For +example, the the best solutions might be stored along with all +per-iteration solutions in an optimization solver. The solution +generation functions in the *alternative-solutions library* return +a :py:class:`PyomoPoolManager`. By default, this pool manager uses +a solution pool that keeps the best solutions. However, the user can +provide a pool manager that is used to store solutions. + +For example, we can explicitly create a pool manager that keeps the +latest solutions. Consider the previous example, where all feasible +solutions are generated: .. doctest:: :skipif: not glpk_available - >>> import pyomo.environ as pyo - >>> import pyomo.contrib.alternative_solutions as aos + >>> solns = aos.enumerate_binary_solutions(m, num_solutions=100, solver="glpk") + >>> print( [soln.objective().value for soln in solns] ) + [70.0, 70.0, 60.0, 60.0, 50.0, 30.0, 20.0, 10.0, 0.0] + +Each solution has a unique index: + +.. doctest:: + :skipif: not glpk_available - >>> values = [10,9,2,1,1] - >>> weights = [10,9,2,1,1] + >>> print( [soln.id for soln in solns] ) + [0, 1, 2, 3, 4, 5, 6, 7, 8] - >>> K = len(values) - >>> capacity = 12 +Now we create a :py:class:`PyomoPoolManager` that is configured with a ``keep_latest`` pool: - >>> m = pyo.ConcreteModel() - >>> m.x = pyo.Var(range(K), within=pyo.Binary) - >>> m.o = pyo.Objective(expr=sum(values[i] * m.x[i] for i in range(K)), sense=pyo.maximize) - >>> m.c = pyo.Constraint(expr=sum(weights[i] * m.x[i] for i in range(K)) <= capacity) +.. doctest:: + :skipif: not glpk_available + + >>> pool_manager = aos.PyomoPoolManager() + >>> context = pool_manager.add_pool(policy=aos.PoolPolicy.keep_latest, max_pool_size=3) + >>> solns = aos.enumerate_binary_solutions(m, num_solutions=100, solver="glpk", pool_manager=pool_manager) + + >>> assert id(pool_manager) == id(solns) + >>> print( [soln.id for soln in solns] ) + [6, 7, 8] + +The default solution pool has policy ``keep_best`` with name ``None``. +If a new Solution pool is added without a name, then the ``None`` +pool is replaced. Otherwise, if a solution pool is added with an +existing name an error occurs. + +The pool manager always has an active pool. The pool manager has the +same API as a solution pool, and the methods and data of the active +pool are exposed to the user through the pool manager. The active pool +defaults to the pool that was most recently added to the pool manager. + + +Solution Objects +---------------- + +Each :py:class:`Solution` object contains information about the objective and +variables. Solutions can be sorted based on their variable values: + +.. doctest:: + :skipif: not glpk_available + + >>> solns = aos.enumerate_binary_solutions(m, num_solutions=100, solver="glpk", abs_opt_gap=0.0) + >>> print( [soln.objective().value for soln in solns] ) + [70.0, 70.0] - >>> solns = aos.enumerate_binary_solutions(m, num_solutions=10, solver="glpk", abs_opt_gap = 0.5) - >>> assert(len(solns) == 4) - >>> for soln in sorted(solns, key=lambda s: str(s.get_variable_name_values())): + >>> sorted_solns = list(sorted(solns)) + >>> for soln in sorted_solns: ... print(soln) { - "fixed_variables": [], - "objective": "o", - "objective_value": 12.0, - "solution": { - "x[0]": 0, - "x[1]": 1, - "x[2]": 1, - "x[3]": 0, - "x[4]": 1 - } + "id": 1, + "objectives": [ + { + "index": 0, + "name": "o", + "suffix": {}, + "value": 70.0 + } + ], + "suffix": {}, + "variables": [ + { + "discrete": true, + "fixed": false, + "index": 0, + "name": "x[0]", + "suffix": {}, + "value": 0 + }, + { + "discrete": true, + "fixed": false, + "index": 1, + "name": "x[1]", + "suffix": {}, + "value": 1 + }, + { + "discrete": true, + "fixed": false, + "index": 2, + "name": "x[2]", + "suffix": {}, + "value": 1 + }, + { + "discrete": true, + "fixed": false, + "index": 3, + "name": "x[3]", + "suffix": {}, + "value": 0 + } + ] } { - "fixed_variables": [], - "objective": "o", - "objective_value": 12.0, - "solution": { - "x[0]": 0, - "x[1]": 1, - "x[2]": 1, - "x[3]": 1, - "x[4]": 0 - } - } - { - "fixed_variables": [], - "objective": "o", - "objective_value": 12.0, - "solution": { - "x[0]": 1, - "x[1]": 0, - "x[2]": 0, - "x[3]": 1, - "x[4]": 1 - } - } - { - "fixed_variables": [], - "objective": "o", - "objective_value": 12.0, - "solution": { - "x[0]": 1, - "x[1]": 0, - "x[2]": 1, - "x[3]": 0, - "x[4]": 0 - } + "id": 0, + "objectives": [ + { + "index": 0, + "name": "o", + "suffix": {}, + "value": 70.0 + } + ], + "suffix": {}, + "variables": [ + { + "discrete": true, + "fixed": false, + "index": 0, + "name": "x[0]", + "suffix": {}, + "value": 1 + }, + { + "discrete": true, + "fixed": false, + "index": 1, + "name": "x[1]", + "suffix": {}, + "value": 0 + }, + { + "discrete": true, + "fixed": false, + "index": 2, + "name": "x[2]", + "suffix": {}, + "value": 0 + }, + { + "discrete": true, + "fixed": false, + "index": 3, + "name": "x[3]", + "suffix": {}, + "value": 1 + } + ] } +Further, variable and objective values can be retrieved either using an integer index or the corresponding name: + +.. doctest:: + :skipif: not glpk_available + + >>> soln = sorted_solns[0] + + >>> print(f"{soln.objective().value=} {soln.objective(0).value=} {soln.objective('o').value=}") + soln.objective().value=70.0 soln.objective(0).value=70.0 soln.objective('o').value=70.0 + + >>> print(f"{soln.variable(0).value=} {soln.variable('x[0]').value=}") + soln.variable(0).value=0 soln.variable('x[0]').value=0 + Interface Documentation ----------------------- -.. currentmodule:: pyomo.contrib.alternative_solutions +Functions that Generate Alternative Solutions +============================================= .. autofunction:: enumerate_binary_solutions - :noindex: .. autofunction:: enumerate_linear_solutions - :noindex: -.. autofunction:: pyomo.contrib.alternative_solutions.lp_enum_solnpool.enumerate_linear_solutions_soln_pool - :noindex: +.. autofunction:: gurobi_enumerate_linear_solutions .. autofunction:: gurobi_generate_solutions - :noindex: .. autofunction:: obbt_analysis_bounds_and_solutions - :noindex: + +Classes for Solutions and Pools +=============================== + +.. autoclass:: VariableInfo + :members: + +.. autoclass:: ObjectiveInfo + :members: .. autoclass:: Solution - :noindex: + :members: + +.. autoclass:: PoolManager + :members: +.. autoclass:: PyomoPoolManager + :members: + :show-inheritance: diff --git a/pyomo/common/collections/bunch.py b/pyomo/common/collections/bunch.py index 34568565994..171a0742f8e 100644 --- a/pyomo/common/collections/bunch.py +++ b/pyomo/common/collections/bunch.py @@ -16,6 +16,7 @@ # the U.S. Government retains certain rights in this software. # ___________________________________________________________________________ +import types import shlex from collections.abc import Mapping @@ -36,31 +37,38 @@ class Bunch(dict): def __init__(self, *args, **kw): self._name_ = self.__class__.__name__ for arg in args: - if not isinstance(arg, str): - raise TypeError("Bunch() positional arguments must be strings") - for item in shlex.split(arg): - item = item.split('=', 1) - if len(item) != 2: - raise ValueError( - "Bunch() positional arguments must be space separated " - f"strings of form 'key=value', got '{item[0]}'" - ) - - # Historically, this used 'exec'. That is unsafe in - # this context (because anyone can pass arguments to a - # Bunch). While not strictly backwards compatible, - # Pyomo was not using this for anything past parsing - # None/float/int values. We will explicitly parse those - # values - try: - val = float(item[1]) - if int(val) == val: - val = int(val) - item[1] = val - except: - if item[1].strip() == 'None': - item[1] = None - self[item[0]] = item[1] + if isinstance(arg, types.GeneratorType): + for k, v in arg: + self[k] = v + elif isinstance(arg, str): + for item in shlex.split(arg): + item = item.split('=', 1) + if len(item) != 2: + raise ValueError( + "Bunch() positional arguments must be space separated " + f"strings of form 'key=value', got '{item[0]}'" + ) + + # Historically, this used 'exec'. That is unsafe in + # this context (because anyone can pass arguments to a + # Bunch). While not strictly backwards compatible, + # Pyomo was not using this for anything past parsing + # None/float/int values. We will explicitly parse those + # values + try: + val = float(item[1]) + if int(val) == val: + val = int(val) + item[1] = val + except: + if item[1].strip() == 'None': + item[1] = None + self[item[0]] = item[1] + else: + raise TypeError( + "Bunch() positional arguments must either by generators returning tuples defining a dictionary, or " + "space separated strings of form 'key=value'" + ) for k, v in kw.items(): self[k] = v diff --git a/pyomo/common/tests/test_bunch.py b/pyomo/common/tests/test_bunch.py index 70149761486..37386f35e2e 100644 --- a/pyomo/common/tests/test_bunch.py +++ b/pyomo/common/tests/test_bunch.py @@ -22,7 +22,7 @@ class Test(unittest.TestCase): - def test_Bunch1(self): + def test_Bunch_fromString(self): opt = Bunch('a=None c=d e="1 2 3" f=" 5 "', foo=1, bar='x') self.assertEqual(opt.ll, None) self.assertEqual(opt.a, None) @@ -85,7 +85,8 @@ def test_Bunch1(self): ) with self.assertRaisesRegex( - TypeError, r"Bunch\(\) positional arguments must be strings" + TypeError, + r"Bunch\(\) positional arguments must either by generators returning tuples defining a dictionary, or space separated strings of form 'key=value'", ): Bunch(5) @@ -96,6 +97,19 @@ def test_Bunch1(self): ): Bunch('a=5 foo = 6') + def test_Bunch_fromGenerator(self): + data = dict(a=None, c='d', e="1 2 3", f=" 5 ", foo=1, bar='x') + o1 = Bunch((k, v) for k, v in data.items()) + self.assertEqual( + str(o1), + """a: None +bar: 'x' +c: 'd' +e: '1 2 3' +f: ' 5 ' +foo: 1""", + ) + def test_pickle(self): o1 = Bunch('a=None c=d e="1 2 3"', foo=1, bar='x') s = pickle.dumps(o1) diff --git a/pyomo/contrib/alternative_solutions/__init__.py b/pyomo/contrib/alternative_solutions/__init__.py index ead886ae0f8..7a21cb561e1 100644 --- a/pyomo/contrib/alternative_solutions/__init__.py +++ b/pyomo/contrib/alternative_solutions/__init__.py @@ -10,11 +10,26 @@ # ___________________________________________________________________________ from pyomo.contrib.alternative_solutions.aos_utils import logcontext -from pyomo.contrib.alternative_solutions.solution import Solution -from pyomo.contrib.alternative_solutions.solnpool import gurobi_generate_solutions +from pyomo.contrib.alternative_solutions.solution import ( + PyomoSolution, + Solution, + VariableInfo, + ObjectiveInfo, +) +from pyomo.contrib.alternative_solutions.solnpool import ( + PoolManager, + PyomoPoolManager, + PoolPolicy, +) from pyomo.contrib.alternative_solutions.balas import enumerate_binary_solutions from pyomo.contrib.alternative_solutions.obbt import ( obbt_analysis, obbt_analysis_bounds_and_solutions, ) from pyomo.contrib.alternative_solutions.lp_enum import enumerate_linear_solutions +from pyomo.contrib.alternative_solutions.gurobi_lp_enum import ( + gurobi_enumerate_linear_solutions, +) +from pyomo.contrib.alternative_solutions.gurobi_solnpool import ( + gurobi_generate_solutions, +) diff --git a/pyomo/contrib/alternative_solutions/aos_utils.py b/pyomo/contrib/alternative_solutions/aos_utils.py index c2efbf934b3..78ba8a09de6 100644 --- a/pyomo/contrib/alternative_solutions/aos_utils.py +++ b/pyomo/contrib/alternative_solutions/aos_utils.py @@ -9,11 +9,12 @@ # This software is distributed under the 3-clause BSD License. # ___________________________________________________________________________ +from pyomo.common.collections import Bunch as Munch import logging +from contextlib import contextmanager logger = logging.getLogger(__name__) -from contextlib import contextmanager from pyomo.common.dependencies import numpy as numpy, numpy_available @@ -57,11 +58,10 @@ def get_active_objective(model): """ active_objs = list(model.component_data_objects(pyo.Objective, active=True)) - assert ( - len(active_objs) == 1 - ), "Model has {} active objective functions, exactly one is required.".format( - len(active_objs) - ) + if len(active_objs) != 1: + raise RuntimeError( + f"Model has {len(active_objs)} active objective functions, exactly one is required." + ) return active_objs[0] @@ -81,12 +81,10 @@ def _add_objective_constraint( specified block. """ - assert ( - rel_opt_gap is None or rel_opt_gap >= 0.0 - ), "rel_opt_gap must be None or >= 0.0" - assert ( - abs_opt_gap is None or abs_opt_gap >= 0.0 - ), "abs_opt_gap must be None or >= 0.0" + if not (rel_opt_gap is None or rel_opt_gap >= 0.0): + raise ValueError(f"rel_opt_gap ({rel_opt_gap}) must be None or >= 0.0") + if not (abs_opt_gap is None or abs_opt_gap >= 0.0): + raise ValueError(f"abs_opt_gap ({abs_opt_gap}) must be None or >= 0.0") objective_constraints = [] @@ -302,3 +300,21 @@ def get_model_variables( ) return variable_set + + +class MyMunch(Munch): + # WEH, MPV needed to add a to_dict since Bunch did not have one + def to_dict(self): + return to_dict(self) + + +def to_dict(x): + xtype = type(x) + if xtype in [tuple, set, frozenset]: + return list(x) + elif xtype in [dict, Munch, MyMunch]: + return {k: to_dict(v) for k, v in x.items()} + elif hasattr(x, "to_dict"): + return x.to_dict() + else: + return x diff --git a/pyomo/contrib/alternative_solutions/balas.py b/pyomo/contrib/alternative_solutions/balas.py index 95d4ac21fb1..f1fffa387b3 100644 --- a/pyomo/contrib/alternative_solutions/balas.py +++ b/pyomo/contrib/alternative_solutions/balas.py @@ -15,7 +15,7 @@ import pyomo.environ as pyo from pyomo.common.collections import ComponentSet -from pyomo.contrib.alternative_solutions import Solution +from pyomo.contrib.alternative_solutions import PyomoPoolManager, PoolPolicy import pyomo.contrib.alternative_solutions.aos_utils as aos_utils @@ -31,6 +31,7 @@ def enumerate_binary_solutions( solver_options={}, tee=False, seed=None, + pool_manager=None, ): """ Finds alternative optimal solutions for a binary problem using no-good @@ -44,7 +45,7 @@ def enumerate_binary_solutions( model : ConcreteModel A concrete Pyomo model num_solutions : int - The maximum number of solutions to generate. + The maximum number of solutions to generate. Must be positive variables: None or a collection of Pyomo _GeneralVarData variables The variables for which bounds will be generated. None indicates that all variables will be included. Alternatively, a collection of @@ -71,25 +72,34 @@ def enumerate_binary_solutions( Boolean indicating that the solver output should be displayed. seed : int Optional integer seed for the numpy random number generator + pool_manager : None + Optional pool manager that will be used to collect solutions Returns ------- - solutions - A list of Solution objects. - [Solution] + pool_manager + A PyomoPoolManager object """ logger.info("STARTING NO-GOOD CUT ANALYSIS") - assert search_mode in [ - "optimal", - "random", - "hamming", - ], 'search mode must be "optimal", "random", or "hamming".' + if not (num_solutions >= 1): + raise RuntimeError("num_solutions must be positive integer") + if num_solutions == 1: + logger.warning("Running alternative_solutions method to find only 1 solution!") + + if not (search_mode in ["optimal", "random", "hamming"]): + raise ValueError('search mode must be "optimal", "random", or "hamming".') if seed is not None: aos_utils._set_numpy_rng(seed) + if pool_manager is None: + pool_manager = PyomoPoolManager() + pool_manager.add_pool( + name="enumerate_binary_solutions", policy=PoolPolicy.keep_all + ) + all_variables = aos_utils.get_model_variables(model, include_fixed=True) if variables == None: binary_variables = [ @@ -108,18 +118,18 @@ def enumerate_binary_solutions( else: # pragma: no cover non_binary_variables.append(var.name) if len(non_binary_variables) > 0: - logger.warn( + logger.warning( ( "Warning: The following non-binary variables were included" "in the variable list and will be ignored:" ) ) - logger.warn(", ".join(non_binary_variables)) + logger.warning(", ".join(non_binary_variables)) orig_objective = aos_utils.get_active_objective(model) if len(binary_variables) == 0: - logger.warn("No binary variables found!") + logger.warning("No binary variables found!") # # Setup solver @@ -152,7 +162,6 @@ def enumerate_binary_solutions( else: opt.update_config.check_for_new_objective = False opt.update_config.update_objective = False - # # Initial solve of the model # @@ -172,12 +181,12 @@ def enumerate_binary_solutions( model.solutions.load_from(results) orig_objective_value = pyo.value(orig_objective) logger.info("Found optimal solution, value = {}.".format(orig_objective_value)) - solutions = [Solution(model, all_variables, objective=orig_objective)] + pool_manager.add(variables=all_variables, objective=orig_objective) # # Return just this solution if there are no binary variables # if len(binary_variables) == 0: - return solutions + return pool_manager aos_block = aos_utils._add_aos_block(model, name="_balas") logger.info("Added block {} to the model.".format(aos_block)) @@ -231,7 +240,7 @@ def enumerate_binary_solutions( logger.info( "Iteration {}: objective = {}".format(solution_number, orig_obj_value) ) - solutions.append(Solution(model, all_variables, objective=orig_objective)) + pool_manager.add(variables=all_variables, objective=orig_objective) solution_number += 1 elif ( condition == pyo.TerminationCondition.infeasibleOrUnbounded @@ -257,4 +266,4 @@ def enumerate_binary_solutions( logger.info("COMPLETED NO-GOOD CUT ANALYSIS") - return solutions + return pool_manager diff --git a/pyomo/contrib/alternative_solutions/lp_enum_solnpool.py b/pyomo/contrib/alternative_solutions/gurobi_lp_enum.py similarity index 84% rename from pyomo/contrib/alternative_solutions/lp_enum_solnpool.py rename to pyomo/contrib/alternative_solutions/gurobi_lp_enum.py index 089c2dec37d..6d9f50f2159 100644 --- a/pyomo/contrib/alternative_solutions/lp_enum_solnpool.py +++ b/pyomo/contrib/alternative_solutions/gurobi_lp_enum.py @@ -19,7 +19,12 @@ import pyomo.environ as pyo import pyomo.common.errors -from pyomo.contrib.alternative_solutions import aos_utils, shifted_lp, solution +from pyomo.contrib.alternative_solutions import ( + aos_utils, + shifted_lp, + PyomoPoolManager, + PoolPolicy, +) from pyomo.contrib import appsi @@ -33,6 +38,7 @@ def __init__( all_variables, orig_objective, num_solutions, + pool_manager, ): self.model = model self.zero_threshold = zero_threshold @@ -41,8 +47,9 @@ def __init__( self.orig_model = orig_model self.all_variables = all_variables self.orig_objective = orig_objective - self.solutions = [] self.num_solutions = num_solutions + self.pool_manager = pool_manager + self.soln_count = 0 def cut_generator_callback(self, cb_m, cb_opt, cb_where): if cb_where == gurobipy.GRB.Callback.MIPSOL: @@ -51,13 +58,18 @@ def cut_generator_callback(self, cb_m, cb_opt, cb_where): for var, index in self.model.var_map.items(): var.set_value(var.lb + self.model.var_lower[index].value) - sol = solution.Solution( - self.orig_model, self.all_variables, objective=self.orig_objective + self.pool_manager.add( + variables=self.all_variables, objective=self.orig_objective ) - self.solutions.append(sol) - if len(self.solutions) >= self.num_solutions: + # We explicitly count the number of solutions generated, rather than rely on the + # size of the solution pool, since that may be configured to filter + # solutions. + self.soln_count += 1 + + if self.soln_count >= self.num_solutions: cb_opt._solver_model.terminate() + num_non_zero = 0 non_zero_basic_expr = 1 for idx in range(len(self.variable_groups)): @@ -78,7 +90,7 @@ def cut_generator_callback(self, cb_m, cb_opt, cb_where): cb_opt.cbLazy(new_con) -def enumerate_linear_solutions_soln_pool( +def gurobi_enumerate_linear_solutions( model, num_solutions=10, rel_opt_gap=None, @@ -86,17 +98,20 @@ def enumerate_linear_solutions_soln_pool( zero_threshold=1e-5, solver_options={}, tee=False, + pool_manager=None, ): """ Finds alternative optimal solutions for a (mixed-binary) linear program - using Gurobi's solution pool feature. + using Gurobi's cut generator to enumerate corners of the feasible polytope + using lazy cuts. + Parameters ---------- model : ConcreteModel A concrete Pyomo model num_solutions : int - The maximum number of solutions to generate. + The maximum number of solutions to generate. Must be positive. variables: None or a collection of Pyomo _GeneralVarData variables The variables for which bounds will be generated. None indicates that all variables will be included. Alternatively, a collection of @@ -116,14 +131,27 @@ def enumerate_linear_solutions_soln_pool( Solver option-value pairs to be passed to the solver. tee : boolean Boolean indicating that the solver output should be displayed. + pool_manager : None + Optional pool manager that will be used to collect solutions Returns ------- - solutions - A list of Solution objects. - [Solution] + pool_manager + A PyomoPoolManager object """ logger.info("STARTING LP ENUMERATION ANALYSIS USING GUROBI SOLUTION POOL") + + if not (num_solutions >= 1): + raise ValueError("num_solutions must be positive integer") + if num_solutions == 1: + logger.warning("Running alternative_solutions method to find only 1 solution!") + + if pool_manager is None: + pool_manager = PyomoPoolManager() + pool_manager.add_pool( + name="enumerate_binary_solutions", policy=PoolPolicy.keep_all + ) + # # Setup gurobi # @@ -217,6 +245,7 @@ def bound_slack_rule(m, var_index): all_variables, orig_objective, num_solutions, + pool_manager, ) opt = appsi.solvers.Gurobi() @@ -232,4 +261,4 @@ def bound_slack_rule(m, var_index): aos_block.deactivate() logger.info("COMPLETED LP ENUMERATION ANALYSIS") - return cut_generator.solutions + return cut_generator.pool_manager diff --git a/pyomo/contrib/alternative_solutions/gurobi_solnpool.py b/pyomo/contrib/alternative_solutions/gurobi_solnpool.py new file mode 100644 index 00000000000..839130cffea --- /dev/null +++ b/pyomo/contrib/alternative_solutions/gurobi_solnpool.py @@ -0,0 +1,134 @@ +# ___________________________________________________________________________ +# +# Pyomo: Python Optimization Modeling Objects +# Copyright (c) 2008-2025 +# National Technology and Engineering Solutions of Sandia, LLC +# Under the terms of Contract DE-NA0003525 with National Technology and +# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain +# rights in this software. +# This software is distributed under the 3-clause BSD License. +# ___________________________________________________________________________ + +import logging + +logger = logging.getLogger(__name__) + +from pyomo.common.dependencies import attempt_import +from pyomo.common.errors import ApplicationError + +from pyomo.contrib import appsi +import pyomo.contrib.alternative_solutions.aos_utils as aos_utils +from pyomo.contrib.alternative_solutions import PyomoPoolManager, PoolPolicy + + +def gurobi_generate_solutions( + model, + *, + num_solutions=10, + rel_opt_gap=None, + abs_opt_gap=None, + solver_options={}, + tee=False, + pool_manager=None, + pool_search_mode=2, +): + """ + Finds alternative optimal solutions for discrete variables using Gurobi's + built-in Solution Pool capability. See the Gurobi Solution Pool + documentation for additional details. + + Parameters + ---------- + model : ConcreteModel + A concrete Pyomo model. + num_solutions : int + The maximum number of solutions to generate. This parameter maps to + the PoolSolutions parameter in Gurobi. Must be positive. + rel_opt_gap : non-negative float or None + The relative optimality gap for allowable alternative solutions. + None implies that there is no limit on the relative optimality gap + (i.e. that any feasible solution can be considered by Gurobi). + This parameter maps to the PoolGap parameter in Gurobi. + abs_opt_gap : non-negative float or None + The absolute optimality gap for allowable alternative solutions. + None implies that there is no limit on the absolute optimality gap + (i.e. that any feasible solution can be considered by Gurobi). + This parameter maps to the PoolGapAbs parameter in Gurobi. + solver_options : dict + Solver option-value pairs to be passed to the Gurobi solver. + tee : boolean + Boolean indicating that the solver output should be displayed. + pool_manager : None + Optional pool manager that will be used to collect solutions + pool_search_mode : 1 or 2 + The generation method for filling the pool. + This parameter maps to the PoolSearchMode in gurobi. + Method designed to work with value 2 as optimality ordered. + + Returns + ------- + pool_manager + A PyomoPoolManager object + """ + + if not (num_solutions >= 1): + raise ValueError("num_solutions must be positive integer") + if num_solutions == 1: + logger.warning("Running alternative_solutions method to find only 1 solution!") + + if not (pool_search_mode in [1, 2]): + raise ValueError("pool_search_mode must be 1 or 2") + if pool_search_mode == 1: + logger.warning( + "Running gurobi_solnpool with PoolSearchMode=1, best effort search may lead to unexpected behavior" + ) + + if pool_manager is None: + pool_manager = PyomoPoolManager() + pool_manager.add_pool( + name="gurobi_generate_solutions", policy=PoolPolicy.keep_all + ) + # + # Setup gurobi + # + opt = appsi.solvers.Gurobi() + if not opt.available(): + raise ApplicationError("Solver (gurobi) not available") + + opt.config.stream_solver = tee + opt.config.load_solution = False + opt.gurobi_options["PoolSolutions"] = num_solutions + opt.gurobi_options["PoolSearchMode"] = pool_search_mode + if rel_opt_gap is not None: + opt.gurobi_options["PoolGap"] = rel_opt_gap + if abs_opt_gap is not None: + opt.gurobi_options["PoolGapAbs"] = abs_opt_gap + for parameter, value in solver_options.items(): + opt.gurobi_options[parameter] = value + # + # Run gurobi + # + results = opt.solve(model) + condition = results.termination_condition + if not (condition == appsi.base.TerminationCondition.optimal): + raise ApplicationError( + "Model cannot be solved, " "TerminationCondition = {}" + ).format(condition.value) + # + # Collect solutions + # + solution_count = opt.get_model_attr("SolCount") + variables = aos_utils.get_model_variables(model, include_fixed=True) + objective = aos_utils.get_active_objective(model) + solutions = [] + for i in range(solution_count): + # + # Load the i-th solution into the model + # + results.solution_loader.load_vars(solution_number=i) + # + # Pull the solution from the model, and cache it in a solution pool. + # + pool_manager.add(variable=variables, objective=objective) + + return pool_manager diff --git a/pyomo/contrib/alternative_solutions/lp_enum.py b/pyomo/contrib/alternative_solutions/lp_enum.py index b943314a708..bdfeece76f3 100644 --- a/pyomo/contrib/alternative_solutions/lp_enum.py +++ b/pyomo/contrib/alternative_solutions/lp_enum.py @@ -17,8 +17,8 @@ from pyomo.contrib.alternative_solutions import ( aos_utils, shifted_lp, - solution, - solnpool, + PyomoPoolManager, + PoolPolicy, ) from pyomo.contrib import appsi @@ -35,9 +35,11 @@ def enumerate_linear_solutions( solver_options={}, tee=False, seed=None, + pool_manager=None, ): """ - Finds alternative optimal solutions a (mixed-integer) linear program. + Finds alternative optimal solutions a (mixed-integer) linear program by iteratively + generating corners of the feasible polytope. This function implements the technique described here: @@ -51,7 +53,7 @@ def enumerate_linear_solutions( model : ConcreteModel A concrete Pyomo model num_solutions : int - The maximum number of solutions to generate. + The maximum number of solutions to generate. Must be positive rel_opt_gap : float or None The relative optimality gap for the original objective for which variable bounds will be found. None indicates that a relative gap @@ -77,20 +79,23 @@ def enumerate_linear_solutions( Boolean indicating that the solver output should be displayed. seed : int Optional integer seed for the numpy random number generator + pool_manager : None + Optional pool manager that will be used to collect solutions Returns ------- - solutions - A list of Solution objects. - [Solution] + pool_manager + A PyomoPoolManager object """ logger.info("STARTING LP ENUMERATION ANALYSIS") - assert search_mode in [ - "optimal", - "random", - "norm", - ], 'search mode must be "optimal", "random", or "norm".' + if not (num_solutions >= 1): + raise ValueError("num_solutions must be positive integer") + if num_solutions == 1: + logger.warning("Running alternative_solutions method to find only 1 solution!") + + if not (search_mode in ["optimal", "random", "norm"]): + raise ValueError('search mode must be "optimal", "random", or "norm".') # TODO: Implement the random and norm objectives. I think it is sufficient # to only consider the cb.var_lower variables in the objective for these two # cases. The cb.var_upper variables are directly linked to these to diversity @@ -98,6 +103,12 @@ def enumerate_linear_solutions( # variables doesn't really matter since we only really care about diversity # in the original problem and not in the slack space (I think) + if pool_manager is None: + pool_manager = PyomoPoolManager() + pool_manager.add_pool( + name="enumerate_binary_solutions", policy=PoolPolicy.keep_all + ) + all_variables = aos_utils.get_model_variables(model) # else: # binary_variables = ComponentSet() @@ -116,7 +127,8 @@ def enumerate_linear_solutions( # TODO: Relax this if possible - Should allow for the mixed-binary case for var in all_variables: - assert var.is_continuous(), "Model must be an LP" + if not var.is_continuous(): + raise RuntimeError("Model must be an LP") use_appsi = False if "appsi" in solver: @@ -235,9 +247,8 @@ def enumerate_linear_solutions( for var, index in cb.var_map.items(): var.set_value(var.lb + cb.var_lower[index].value) - sol = solution.Solution(model, all_variables, objective=orig_objective) - solutions.append(sol) - orig_objective_value = sol.objective[1] + pool_manager.add(variables=all_variables, objective=orig_objective) + orig_objective_value = pyo.value(orig_objective) if logger.isEnabledFor(logging.INFO): logger.info("Solved, objective = {}".format(orig_objective_value)) @@ -327,4 +338,4 @@ def enumerate_linear_solutions( logger.info("COMPLETED LP ENUMERATION ANALYSIS") - return solutions + return pool_manager diff --git a/pyomo/contrib/alternative_solutions/obbt.py b/pyomo/contrib/alternative_solutions/obbt.py index 700acdc2276..f016fc1b8cd 100644 --- a/pyomo/contrib/alternative_solutions/obbt.py +++ b/pyomo/contrib/alternative_solutions/obbt.py @@ -15,7 +15,7 @@ import pyomo.environ as pyo from pyomo.contrib.alternative_solutions import aos_utils -from pyomo.contrib.alternative_solutions import Solution +from pyomo.contrib.alternative_solutions import PyomoPoolManager, PoolPolicy from pyomo.contrib import appsi @@ -74,7 +74,7 @@ def obbt_analysis( {variable: (lower_bound, upper_bound)}. An exception is raised when the solver encountered an issue. """ - bounds, solns = obbt_analysis_bounds_and_solutions( + bounds, pool_manager = obbt_analysis_bounds_and_solutions( model, variables=variables, rel_opt_gap=rel_opt_gap, @@ -99,6 +99,7 @@ def obbt_analysis_bounds_and_solutions( solver="gurobi", solver_options={}, tee=False, + pool_manager=None, ): """ Calculates the bounds on each variable by solving a series of min and max @@ -135,6 +136,8 @@ def obbt_analysis_bounds_and_solutions( Solver option-value pairs to be passed to the solver. tee : boolean Boolean indicating that the solver output should be displayed. + pool_manager : None + Optional pool manager that will be used to collect solutions Returns ------- @@ -142,18 +145,25 @@ def obbt_analysis_bounds_and_solutions( A Pyomo ComponentMap containing the bounds for each variable. {variable: (lower_bound, upper_bound)}. An exception is raised when the solver encountered an issue. - solutions - [Solution] + pool_manager + [PyomoPoolManager] """ # TODO - parallelization logger.info("STARTING OBBT ANALYSIS") + if pool_manager is None: + pool_manager = PyomoPoolManager() + pool_manager.add_pool( + name="enumerate_binary_solutions", policy=PoolPolicy.keep_all + ) + if warmstart: - assert ( - variables == None - ), "Cannot restrict variable list when warmstart is specified" + if not (variables == None): + raise ValueError( + "Cannot restrict variable list when warmstart is specified" + ) all_variables = aos_utils.get_model_variables(model, include_fixed=False) if variables == None: variable_list = all_variables @@ -242,7 +252,7 @@ def obbt_analysis_bounds_and_solutions( opt.update_config.treat_fixed_vars_as_params = False variable_bounds = pyo.ComponentMap() - solns = [Solution(model, all_variables, objective=orig_objective)] + pool_manager.add(variables=all_variables, objective=orig_objective) senses = [(pyo.minimize, "LB"), (pyo.maximize, "UB")] @@ -284,7 +294,7 @@ def obbt_analysis_bounds_and_solutions( results.solution_loader.load_vars(solution_number=0) else: model.solutions.load_from(results) - solns.append(Solution(model, all_variables, objective=orig_objective)) + pool_manager.add(variables=all_variables, objective=orig_objective) if warmstart: _add_solution(solutions) @@ -332,7 +342,7 @@ def obbt_analysis_bounds_and_solutions( logger.info("COMPLETED OBBT ANALYSIS") - return variable_bounds, solns + return variable_bounds, pool_manager def _add_solution(solutions): diff --git a/pyomo/contrib/alternative_solutions/solnpool.py b/pyomo/contrib/alternative_solutions/solnpool.py index d252fcd1a00..9d6edffcea0 100644 --- a/pyomo/contrib/alternative_solutions/solnpool.py +++ b/pyomo/contrib/alternative_solutions/solnpool.py @@ -9,119 +9,939 @@ # This software is distributed under the 3-clause BSD License. # ___________________________________________________________________________ -import logging +import copy +from enum import Enum +import heapq +import collections +import dataclasses +import json +import weakref -logger = logging.getLogger(__name__) +from pyomo.contrib.alternative_solutions.aos_utils import MyMunch, to_dict +from pyomo.contrib.alternative_solutions.solution import Solution, PyomoSolution -from pyomo.common.dependencies import attempt_import -from pyomo.common.errors import ApplicationError +nan = float("nan") -from pyomo.contrib import appsi -import pyomo.contrib.alternative_solutions.aos_utils as aos_utils -from pyomo.contrib.alternative_solutions import Solution +def default_as_solution(*args, **kwargs): + """ + A default function that creates a solution from the args and kwargs that + are passed-in to the add() method in a solution pool. + + This passes arguments to the Solution() class constructor, so the API for this method is + the same as that method. + """ + return Solution(*args, **kwargs) + + +def _as_pyomo_solution(*args, **kwargs): + """ + A pyomo-specific function that creates a solution from the args and kwargs that + are passed-in to the add() method in a solution pool. -def gurobi_generate_solutions( - model, - *, - num_solutions=10, - rel_opt_gap=None, - abs_opt_gap=None, - solver_options={}, - tee=False, -): + This passes arguments to the PyomoSolution() class constructor, so the API for this method is + the same as that method. """ - Finds alternative optimal solutions for discrete variables using Gurobi's - built-in Solution Pool capability. + return PyomoSolution(*args, **kwargs) + + +class PoolCounter: + """ + A class to wrap the counter element for solution pools. + It contains just the solution_counter element. + """ + + solution_counter = 0 + - This method defaults to the optimality-enforced discovery method with PoolSearchMode = 2. - There are two other options, standard single optimal solution (PoolSearchMode = 0) and - best-effort discovery with no guarantees (PoolSearchMode = 1). Please consult the Gurobi - documentation on PoolSearchMode for details on impact on Gurobi results. - Changes to this mode can be made by included PoolSearchMode set to the intended value - in solver_options. +class PoolPolicy(Enum): + unspecified = 'unspecified' + keep_all = 'keep_all' + keep_best = 'keep_best' + keep_latest = 'keep_latest' + keep_latest_unique = 'keep_latest_unique' + + def __str__(self): + return f"{self.value}" + + +class SolutionPoolBase: + """ + A class to manage groups of solutions as a pool. + This is the general base pool class. + + This class is designed to integrate with the alternative_solution generation methods. + Additionally, groups of SolutionPool objects can be handled with the PoolManager class. Parameters ---------- - model : ConcreteModel - A concrete Pyomo model. - num_solutions : int - The maximum number of solutions to generate. This parameter maps to - the PoolSolutions parameter in Gurobi. - rel_opt_gap : non-negative float or None - The relative optimality gap for allowable alternative solutions. - None implies that there is no limit on the relative optimality gap - (i.e. that any feasible solution can be considered by Gurobi). - This parameter maps to the PoolGap parameter in Gurobi. - abs_opt_gap : non-negative float or None - The absolute optimality gap for allowable alternative solutions. - None implies that there is no limit on the absolute optimality gap - (i.e. that any feasible solution can be considered by Gurobi). - This parameter maps to the PoolGapAbs parameter in Gurobi. - solver_options : dict - Solver option-value pairs to be passed to the Gurobi solver. - tee : boolean - Boolean indicating that the solver output should be displayed. - - Returns - ------- - solutions - A list of Solution objects. [Solution] + name : str + String name to describe the pool. + as_solution : Callable[..., Solution][..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the default_as_solution function being used. + counter : PoolCounter or None + PoolCounter object to manage solution indexing. + A value of None will result in a new PoolCounter object being created and used. + policy : PoolPolicy + Enum value to describe the pool construction and management policy. """ - # - # Setup gurobi - # - opt = appsi.solvers.Gurobi() - if not opt.available(): - raise ApplicationError("Solver (gurobi) not available") - - assert num_solutions >= 1, "num_solutions must be positive integer" - if num_solutions == 1: - logger.warning("Running alternative_solutions method to find only 1 solution!") - - opt.config.stream_solver = tee - opt.config.load_solution = False - opt.gurobi_options["PoolSolutions"] = num_solutions - if rel_opt_gap is not None: - opt.gurobi_options["PoolGap"] = rel_opt_gap - if abs_opt_gap is not None: - opt.gurobi_options["PoolGapAbs"] = abs_opt_gap - for parameter, value in solver_options.items(): - opt.gurobi_options[parameter] = value - if "PoolSearchMode" not in opt.gurobi_options: - opt.gurobi_options["PoolSearchMode"] = 2 - elif opt.gurobi_options["PoolSearchMode"] == 0: - logger.warning( - "Running gurobi_solnpool with PoolSearchMode=0, this is single search mode and not the intended use case for gurobi_generate_solutions" + + def __init__(self, name, as_solution, counter, policy=PoolPolicy.unspecified): + self._solutions = {} + if as_solution is None: + self._as_solution = default_as_solution + else: + self._as_solution = as_solution + if counter is None: + self.counter = PoolCounter() + else: + self.counter = counter + # TODO: consider renaming context_name to name + self._metadata = MyMunch( + context_name=name, + policy=policy, + as_solution_source=f"{self._as_solution.__module__}.{self._as_solution.__qualname__}", ) - elif opt.gurobi_options["PoolSearchMode"] == 1: - logger.warning( - "Running gurobi_solnpool with PoolSearchMode=1, best effort search may lead to unexpected behavior" + + @property + def metadata(self): + """ + Property to return SolutionPool metadata that all SolutionPool subclasses have. + """ + return self._metadata + + @property + def solutions(self): + """ + Property to return values of the dictionary of solutions. + """ + return self._solutions.values() + + @property + def last_solution(self): + """ + Property to return last (successfully) added solution. + """ + index = next(reversed(self._solutions.keys())) + return self._solutions[index] + + @property + def policy(self): + """ + Property to return pool construction policy. + """ + return self.metadata['policy'] + + @property + def as_solution(self): + """ + Property to return solution conversion method. + """ + return self._as_solution + + @property + def pool_config(self): + """ + Property to return SolutionPool class specific configuration data. + """ + return dict() + + def __iter__(self): + for soln in self._solutions.values(): + yield soln + + def __len__(self): + return len(self._solutions) + + def __getitem__(self, soln_id): + return self._solutions[soln_id] + + def _next_solution_counter(self): + tmp = self.counter.solution_counter + self.counter.solution_counter += 1 + return tmp + + def to_dict(self): + """ + Converts SolutionPool to a dictionary object. + + Returns + ---------- + dict + Dictionary with three keys: 'metadata', 'solutions', 'pool_config' + 'metadata' contains a dictionary of information about SolutionPools that is always present. + 'solutions' contains a dictionary of the pool's solutions. + 'pool_config' contains a dictionary of details conditional to the SolutionPool type. + """ + md = copy.copy(self.metadata) + md.policy = str(md.policy) + return dict( + metadata=to_dict(md), + solutions=to_dict(self._solutions), + pool_config=to_dict(self.pool_config), ) - # - # Run gurobi - # - results = opt.solve(model) - condition = results.termination_condition - if not (condition == appsi.base.TerminationCondition.optimal): - raise ApplicationError( - "Model cannot be solved, " "TerminationCondition = {}" - ).format(condition.value) - # - # Collect solutions - # - solution_count = opt.get_model_attr("SolCount") - variables = aos_utils.get_model_variables(model, include_fixed=True) - solutions = [] - for i in range(solution_count): + + +class SolutionPool_KeepAll(SolutionPoolBase): + """ + A SolutionPool subclass to keep all added solutions. + + This class is designed to integrate with the alternative_solution generation methods. + Additionally, groups of solution pools can be handled with the PoolManager class. + + Parameters + ---------- + name : str + String name to describe the pool. + as_solution : Callable[..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the default_as_solution function being used. + counter : PoolCounter or None + PoolCounter object to manage solution indexing. + A value of None will result in a new PoolCounter object being created and used. + """ + + def __init__(self, name=None, as_solution=None, counter=None): + super().__init__(name, as_solution, counter, policy=PoolPolicy.keep_all) + + def add(self, *args, **kwargs): + """ + Add inputted solution to SolutionPool. + Relies on the instance as_solution conversion method to convert inputs to Solution Object. + Adds the converted Solution object to the pool dictionary. + ID value for the solution generated as next increment of instance PoolCounter. + + Parameters + ---------- + Input needs to match as_solution format from pool initialization. + + Returns + ---------- + int + The ID value to match the added solution from the solution pool's PoolCounter. + The ID value is also the pool dictionary key for this solution. + """ + if len(args) == 1 and not kwargs and isinstance(args[0], Solution): + soln = args[0] + else: + soln = self._as_solution(*args, **kwargs) + # + soln.id = self._next_solution_counter() + if soln.id in self._solutions: + raise DeveloperError( + f"Solution id {soln.id} already in solution pool context '{self._context_name}'" + ) + # + self._solutions[soln.id] = soln + return soln.id + + +class SolutionPool_KeepLatest(SolutionPoolBase): + """ + A subclass of SolutionPool with the policy of keep the latest k solutions. + Added solutions are not checked for uniqueness. + + This class is designed to integrate with the alternative_solution generation methods. + Additionally, groups of solution pools can be handled with the PoolManager class. + + Parameters + ---------- + name : str + String name to describe the pool. + as_solution : Callable[..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the default_as_solution function being used. + counter : PoolCounter or None + PoolCounter object to manage solution indexing. + A value of None will result in a new PoolCounter object being created and used. + max_pool_size : int + The max_pool_size is the K value for keeping the latest K solutions. + Must be a positive integer. + """ + + def __init__(self, name=None, as_solution=None, counter=None, *, max_pool_size=1): + if not (max_pool_size >= 1): + raise ValueError("max_pool_size must be positive integer") + super().__init__(name, as_solution, counter, policy=PoolPolicy.keep_latest) + self.max_pool_size = max_pool_size + self._int_deque = collections.deque() + + @property + def pool_config(self): + return dict(max_pool_size=self.max_pool_size) + + def add(self, *args, **kwargs): + """ + Add inputted solution to SolutionPool. + + This method relies on the instance as_solution conversion function + to convert the inputs to a Solution object. This solution is + added to the pool dictionary. The ID value for the solution + generated is the next increment of instance PoolCounter. + + When pool size < max_pool_size, new solution is added without deleting old solutions. + When pool size == max_pool_size, new solution is added and oldest solution deleted. + + Parameters + ---------- + Input needs to match as_solution format from pool initialization. + + Returns + ---------- + int + The ID value to match the added solution from the solution pool's PoolCounter. + The ID value is also the pool dictionary key for this solution. + """ + if len(args) == 1 and not kwargs and isinstance(args[0], Solution): + soln = args[0] + else: + soln = self._as_solution(*args, **kwargs) + # + soln.id = self._next_solution_counter() + if soln.id in self._solutions: + raise DeveloperError( + f"Solution id {soln.id} already in solution pool context '{self._context_name}'" + ) + # + self._solutions[soln.id] = soln + self._int_deque.append(soln.id) + if len(self._int_deque) > self.max_pool_size: + index = self._int_deque.popleft() + del self._solutions[index] + # + return soln.id + + +class SolutionPool_KeepLatestUnique(SolutionPoolBase): + """ + A subclass of SolutionPool with the policy of keep the latest k unique solutions. + Added solutions are checked for uniqueness. + + This class is designed to integrate with the alternative_solution generation methods. + Additionally, groups of solution pools can be handled with the PoolManager class. + + Parameters + ---------- + name : str + String name to describe the pool. + as_solution : Callable[..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the default_as_solution function being used. + counter : PoolCounter or None + PoolCounter object to manage solution indexing. + A value of None will result in a new PoolCounter object being created and used. + max_pool_size : int + The max_pool_size is the K value for keeping the latest K solutions. + Must be a positive integer. + """ + + def __init__(self, name=None, as_solution=None, counter=None, *, max_pool_size=1): + if not (max_pool_size >= 1): + raise ValueError("max_pool_size must be positive integer") + super().__init__( + name, as_solution, counter, policy=PoolPolicy.keep_latest_unique + ) + self.max_pool_size = max_pool_size + self._int_deque = collections.deque() + self._unique_solutions = set() + + @property + def pool_config(self): + return dict(max_pool_size=self.max_pool_size) + + def add(self, *args, **kwargs): + """ + Add inputted solution to SolutionPool. + Relies on the instance as_solution conversion method to convert inputs to Solution Object. + If solution already present, new solution is not added. + If input solution is new, the converted Solution object to the pool dictionary. + ID value for the solution generated as next increment of instance PoolCounter. + When pool size < max_pool_size, new solution is added without deleting old solutions. + When pool size == max_pool_size, new solution is added and oldest solution deleted. + + Parameters + ---------- + Input needs to match as_solution format from pool initialization. + + Returns + ---------- + None or int + None value corresponds to solution was already present and is ignored. + When not present, the ID value to match the added solution from the solution pool's PoolCounter. + The ID value is also the pool dictionary key for this solution. + """ + if len(args) == 1 and not kwargs and isinstance(args[0], Solution): + soln = args[0] + else: + soln = self._as_solution(*args, **kwargs) + # + # Return None if the solution has already been added to the pool # - # Load the i-th solution into the model + tuple_repn = soln._tuple_repn() + if tuple_repn in self._unique_solutions: + return None + self._unique_solutions.add(tuple_repn) # - results.solution_loader.load_vars(solution_number=i) + soln.id = self._next_solution_counter() + if soln.id in self._solutions: + raise DeveloperError( + f"Solution id {soln.id} already in solution pool context '{self._context_name}'" + ) # - # Pull the solution from the model into a Solution object, - # and append to our list of solutions + self._int_deque.append(soln.id) + if len(self._int_deque) > self.max_pool_size: + index = self._int_deque.popleft() + del self._solutions[index] # - solutions.append(Solution(model, variables)) + self._solutions[soln.id] = soln + return soln.id + - return solutions +@dataclasses.dataclass(order=True) +class HeapItem: + value: float + id: int = dataclasses.field(compare=False) + + +class SolutionPool_KeepBest(SolutionPoolBase): + """ + A subclass of SolutionPool with the policy of keep the best k unique solutions based on objective. + Added solutions are checked for uniqueness. + Both the relative and absolute tolerance must be passed to add a solution. + + + This class is designed to integrate with the alternative_solution generation methods. + Additionally, groups of solution pools can be handled with the PoolManager class. + + Parameters + ---------- + name : str + String name to describe the pool. + as_solution : Callable[..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the default_as_solution function being used. + counter : PoolCounter or None + PoolCounter object to manage solution indexing. + A value of None will result in a new PoolCounter object being created and used. + max_pool_size : None or int + Value of None results in no max pool limit based on number of solutions. + If not None, the value must be a positive integer. + The max_pool_size is the K value for keeping the latest K solutions. + objective : int + The index of the objective function that is used to compare solutions. + abs_tolerance : None or int + absolute tolerance from best solution based on objective beyond which to reject a solution. + None results in absolute tolerance test passing new solution. + rel_tolernace : None or int + relative tolerance from best solution based on objective beyond which to reject a solution. + None results in relative tolerance test passing new solution. + sense_is_min : Boolean + Sense information to encode either minimization or maximization. + True means minimization problem. False means maximization problem. + best_value : float + Optional information to provide a starting best-discovered value for tolerance comparisons. + Defaults to a 'nan' value that the first added solution's value will replace. + """ + + def __init__( + self, + name=None, + as_solution=None, + counter=None, + *, + max_pool_size=None, + objective=0, + abs_tolerance=0.0, + rel_tolerance=None, + sense_is_min=True, + best_value=nan, + ): + super().__init__(name, as_solution, counter, policy=PoolPolicy.keep_best) + if not ((max_pool_size is None) or (max_pool_size >= 1)): + raise ValueError("max_pool_size must be None or positive integer") + self.max_pool_size = max_pool_size + self.objective = objective + self.abs_tolerance = abs_tolerance + self.rel_tolerance = rel_tolerance + self.sense_is_min = sense_is_min + self.best_value = best_value + self._heap = [] + self._unique_solutions = set() + + @property + def pool_config(self): + return dict( + max_pool_size=self.max_pool_size, + objective=self.objective, + abs_tolerance=self.abs_tolerance, + rel_tolerance=self.rel_tolerance, + sense_is_min=self.sense_is_min, + best_value=self.best_value, + ) + + def add(self, *args, **kwargs): + """ + Add inputted solution to SolutionPool. + Relies on the instance as_solution conversion method to convert inputs to Solution Object. + If solution already present or outside tolerance of the best objective value, new solution is not added. + If input solution is new and within tolerance of the best objective value, the converted Solution object to the pool dictionary. + ID value for the solution generated as next increment of instance PoolCounter. + When pool size < max_pool_size, new solution is added without deleting old solutions. + When pool size == max_pool_size, new solution is added and oldest solution deleted. + + Parameters + ---------- + Input needs to match as_solution format from pool initialization. + + Returns + ---------- + None or int + None value corresponds to solution was already present and is ignored. + When not present, the ID value to match the added solution from the solution pool's PoolCounter. + The ID value is also the pool dictionary key for this solution. + """ + if len(args) == 1 and not kwargs and isinstance(args[0], Solution): + soln = args[0] + else: + soln = self._as_solution(*args, **kwargs) + # + # Return None if the solution has already been added to the pool + # + tuple_repn = soln._tuple_repn() + if tuple_repn in self._unique_solutions: + return None + self._unique_solutions.add(tuple_repn) + # + value = soln.objective(self.objective).value + keep = False + new_best_value = False + if self.best_value is nan: + self.best_value = value + keep = True + else: + diff = ( + value - self.best_value + if self.sense_is_min + else self.best_value - value + ) + if diff < 0.0: + # Keep if this is a new best value + self.best_value = value + keep = True + new_best_value = True + elif ((self.abs_tolerance is None) or (diff <= self.abs_tolerance)) and ( + (self.rel_tolerance is None) + or ( + diff / min(math.fabs(value), math.fabs(self.best_value)) + <= self.rel_tolerance + ) + ): + # Keep if the absolute or relative difference with the best value is small enough + keep = True + + if not keep: + return None + + soln.id = self._next_solution_counter() + if soln.id in self._solutions: + raise DeveloperError( + f"Solution id {soln.id} already in solution pool context '{self._context_name}'" + ) + # + self._solutions[soln.id] = soln + # + item = HeapItem(value=-value if self.sense_is_min else value, id=soln.id) + if self.max_pool_size is None or len(self._heap) < self.max_pool_size: + # There is room in the pool, so we just add it + heapq.heappush(self._heap, item) + else: + # We add the item to the pool and pop the worst item in the pool + item = heapq.heappushpop(self._heap, item) + del self._solutions[item.id] + + if new_best_value: + # We have a new best value, so we need to check that all existing solutions are close enough and re-heapify + tmp = [] + for item in self._heap: + value = -item.value if self.sense_is_min else item.value + diff = ( + value - self.best_value + if self.sense_is_min + else self.best_value - value + ) + if ((self.abs_tolerance is None) or (diff <= self.abs_tolerance)) and ( + (self.rel_tolerance is None) + or ( + diff / min(math.fabs(value), math.fabs(self.best_value)) + <= self.rel_tolerance + ) + ): + tmp.append(item) + else: + del self._solutions[item.id] + heapq.heapify(tmp) + self._heap = tmp + + if len(self._solutions) != len(self._heap): + raise DeveloperError( + f"Num solutions is {len(self._solutions)} but the heap size is {len(self._heap)}" + ) + return soln.id + + +class PoolManager: + """ + Manages one or more solution pools. + + The default solution pool has policy ``keep_best`` with name ``None``. + If a new Solution pool is added without a name, then the ``None`` + pool is replaced. Otherwise, if a solution pool is added with an + existing name an error occurs. + + The pool manager always has an active pool. The pool manager has the + same API as a solution pool, and the envelope design pattern is used + to expose the methods and data for the active pool. The active pool + defaults to the pool that was most recently added to the pool manager. + + Note that all pools share the same Counter object to enable overall + solution count tracking and unique solution id values. + + """ + + _policy_dispatcher = { + PoolPolicy.keep_all: SolutionPool_KeepAll, + PoolPolicy.keep_best: SolutionPool_KeepBest, + PoolPolicy.keep_latest: SolutionPool_KeepLatest, + PoolPolicy.keep_latest_unique: SolutionPool_KeepLatestUnique, + } + + def __init__(self): + self._name = None + self._pools = {} + self.add_pool(name=self._name) + self._solution_counter = 0 + + # + # The following methods give the PoolManager the same API as a pool. + # These methods pass-though and operate on the active pool. + # + + @property + def name(self): + """ + Returns + ------- + str + The name of the active pool. + """ + return self._name + + @property + def metadata(self): + """ + Returns + ------- + Munch + Metadata for the active pool. + """ + return self.active_pool.metadata + + @property + def policy(self): + """ + Returns + ------- + str + The policy that is executed by the active pool. + """ + return self.active_pool.policy + + @property + def solutions(self): + """ + Returns + ------- + list + The solutions in the active pool. + """ + return self.active_pool.solutions.values() + + @property + def last_solution(self): + """ + Returns + ------- + Solution + The last solution added to the active pool. + """ + return self.active_pool.last_solution + + @property + def max_pool_size(self): + """ + Returns + ------- + int or None + The maximum pool size value for the active pool, or None if this parameter is not by this pool. + """ + return getattr(self.active_pool, 'max_pool_size', None) + + def to_dict(self): + """ + Returns + ------- + dict + A dictionary representation of the active pool. + """ + return self.active_pool.to_dict() + + def __iter__(self): + """ + Yields + ------- + Solution + The solutions in the active pool. + """ + for soln in self.active_pool.solutions: + yield soln + + def __len__(self): + """ + Returns + ------- + int + The number of solutions in the active pool. + """ + return len(self.active_pool) + + def __getitem__(self, soln_id): + """ + Returns + ------- + Solution + The specified solution in the active pool. + """ + return self._pools[self._name][soln_id] + + def add(self, *args, **kwargs): + """ + Adds a solution to the active pool. + + Returns + ---------- + int + The index of the solution that is added. + """ + return self.active_pool.add(*args, **kwargs) + + # + # The following methods support the management of multiple + # pools within a PoolManager. + # + + @property + def active_pool(self): + """ + Gets the underlying active SolutionPool in PoolManager + + Returns + ---------- + SolutionPool + Active pool object + + """ + if self._name not in self._pools: + raise ValueError(f"Unknown pool '{self._name}'") + return self._pools[self._name] + + def add_pool( + self, *, name=None, policy=PoolPolicy.keep_best, as_solution=None, **kwds + ): + """ + Initializes a new solution pool and adds it to this pool manager. + + The method expects parameters for the constructor of the corresponding solution pool. + Supported pools are `keep_all`, `keep_best`, `keep_latest`, and `keep_latest_unique`. + + Parameters + ---------- + name : str + The name of the solution pool. If name is already used then, then an error is generated. + policy : PoolPolicy + This enum value indicates the policy that is enforced new solution pool. + (Default is PoolPolicy.keep_best.) + as_solution : Callable[..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the default_as_solution function being used. + **kwds + Other associated arguments that are used to initialize the solution pool. + + Returns + ---------- + dict + Metadata for the newly create solution pool. + """ + if name is None and None in self._pools: + del self._pools[None] + + if name not in self._pools: + # Delete the 'None' pool if it isn't being used + if name is not None and None in self._pools and len(self._pools[None]) == 0: + del self._pools[None] + + if not policy in self._policy_dispatcher: + raise ValueError(f"Unknown pool policy: {policy} {type(policy)}") + self._pools[name] = self._policy_dispatcher[policy]( + name=name, as_solution=as_solution, counter=weakref.proxy(self), **kwds + ) + + self._name = name + return self.metadata + + def activate(self, name): + """ + Sets the named SolutionPool to be the active pool in PoolManager + + Parameters + ---------- + name : str + name key to pick the SolutionPool in the PoolManager object to the active pool + If name not a valid key then ValueError is thrown + Returns + ---------- + dict + Metadata attribute of the now active SolutionPool + + """ + if not name in self._pools: + raise ValueError(f"Unknown pool '{name}'") + self._name = name + return self.metadata + + # + # The following methods provide information about all + # pools in the pool manager. + # + + def get_pool_dicts(self): + """ + Converts the set of pools to dictionary object with underlying dictionary of pools + + Returns + ---------- + dict + Keys are names of each pool in PoolManager + Values are to_dict called on corresponding pool + + """ + return {k: v.to_dict() for k, v in self._pools.items()} + + def get_pool_names(self): + """ + Returns the list of name keys for the pools in PoolManager + + Returns + ---------- + List + List of name keys of all pools in this PoolManager + + """ + return list(self._pools.keys()) + + def get_pool_policies(self): + """ + Returns the dictionary of name:policy pairs to identify policies in all Pools + + Returns + ---------- + List + List of name keys of all pools in this PoolManager + + """ + return {k: v.policy for k, v in self._pools.items()} + + def get_max_pool_sizes(self): + """ + Returns the max_pool_size of all pools in the PoolManager as a dict. + If a pool does not have a max_pool_size that value defaults to none + + Returns + ---------- + dict + keys as name of the pool + values as max_pool_size attribute, if not defined, defaults to None + + """ + return {k: getattr(v, "max_pool_size", None) for k, v in self._pools.items()} + + def get_pool_sizes(self): + """ + Returns the len of all pools in the PoolManager as a dict. + + Returns + ---------- + dict + keys as name of the pool + values as the number of solutions in the underlying pool + + """ + return {k: len(v) for k, v in self._pools.items()} + + # + # The following methods treat the PoolManager as a PoolCounter. + # This allows the PoolManager to be used to provide a global solution count + # for all pools that it manages. + # + + @property + def solution_counter(self): + return self._solution_counter + + @solution_counter.setter + def solution_counter(self, value): + self._solution_counter = value + + +class PyomoPoolManager(PoolManager): + """ + A subclass of PoolManager for handing pools of Pyomo solutions. + + This class redefines the add_pool method to use the + default_as_pyomo_solution method to construct Solution objects. + Otherwise, this class inherits from PoolManager. + """ + + def add_pool( + self, *, name=None, policy=PoolPolicy.keep_best, as_solution=None, **kwds + ): + """ + Initializes a new solution pool and adds it to this pool manager. + + The method expects parameters for the constructor of the corresponding solution pool. + Supported pools are `keep_all`, `keep_best`, `keep_latest`, and `keep_latest_unique`. + + Parameters + ---------- + name : str + The name of the solution pool. If name is already used then, then an error is generated. + policy : PoolPolicy + This enum value indicates the policy that is enforced new solution pool. + (Default is PoolPolicy.keep_best.) + as_solution : Callable[..., Solution] or None + Method for converting inputs into Solution objects. + A value of None will result in the _as_pyomo_solution function being used. + **kwds + Other associated arguments that are used to initialize the solution pool. + + Returns + ---------- + dict + Metadata for the newly create solution pool. + + """ + if as_solution is None: + as_solution = _as_pyomo_solution + return PoolManager.add_pool( + self, name=name, policy=policy, as_solution=as_solution, **kwds + ) diff --git a/pyomo/contrib/alternative_solutions/solution.py b/pyomo/contrib/alternative_solutions/solution.py index cd5ad8a19d9..9eb8e430b27 100644 --- a/pyomo/contrib/alternative_solutions/solution.py +++ b/pyomo/contrib/alternative_solutions/solution.py @@ -9,150 +9,295 @@ # This software is distributed under the 3-clause BSD License. # ___________________________________________________________________________ +import sys +import heapq +import collections +import dataclasses import json +import functools + import pyomo.environ as pyo -from pyomo.common.collections import ComponentMap, ComponentSet -from pyomo.contrib.alternative_solutions import aos_utils +from .aos_utils import MyMunch, to_dict -class Solution: +nan = float("nan") + + +def _custom_dict_factory(data): + return {k: to_dict(v) for k, v in data} + + +if sys.version_info >= (3, 10): + dataclass_kwargs = dict(kw_only=True) +else: + dataclass_kwargs = dict() + + +@dataclasses.dataclass(**dataclass_kwargs) +class VariableInfo: """ - A class to store solutions from a Pyomo model. + Represents a variable in a solution. Attributes ---------- - variables : ComponentMap - A map between Pyomo variables and their values for a solution. - fixed_vars : ComponentSet - The set of Pyomo variables that are fixed in a solution. - objective : ComponentMap - A map between Pyomo objectives and their values for a solution. - - Methods - ------- - pprint(): - Prints a solution. - get_variable_name_values(self, ignore_fixed_vars=False): - Get a dictionary of variable name-variable value pairs. - get_fixed_variable_names(self): - Get a list of fixed-variable names. - get_objective_name_values(self): - Get a dictionary of objective name-objective value pairs. + value : float + The value of the variable. + fixed : bool + If True, then the variable was fixed during optimization. + name : str + The name of the variable. + index : int + The unique identifier for this variable. + discrete : bool + If True, then this is a discrete variable + suffix : dict + Other information about this variable. """ - def __init__(self, model, variable_list, include_fixed=True, objective=None): - """ - Constructs a Pyomo Solution object. + value: float = nan + fixed: bool = False + name: str = None + repn = None + index: int = None + discrete: bool = False + suffix: MyMunch = dataclasses.field(default_factory=MyMunch) + + def to_dict(self): + return dataclasses.asdict(self, dict_factory=_custom_dict_factory) + + +@dataclasses.dataclass(**dataclass_kwargs) +class ObjectiveInfo: + """ + Represents an objective in a solution. + + Attributes + ---------- + value : float + The objective value. + name : str + The name of the objective. + index : int + The unique identifier for this objective. + suffix : dict + Other information about this objective. + """ + + value: float = nan + name: str = None + index: int = None + suffix: MyMunch = dataclasses.field(default_factory=MyMunch) + + def to_dict(self): + return dataclasses.asdict(self, dict_factory=_custom_dict_factory) + + +@functools.total_ordering +class Solution: + """ + An object that describes an optimization solution. + + Parameters + ----------- + variables : None or list + A list of :py:class:`VariableInfo` objects. (default is None) + objective : None or :py:class:`ObjectiveInfo` + A :py:class:`ObjectiveInfo` object. (default is None) + objectives : None or list + A list of :py:class:`ObjectiveInfo` objects. (default is None) + kwargs : dict + A dictionary of auxiliary data that is stored with the core solution values. If the 'suffix' + keyword is specified, then its value is use to define suffix data. Otherwise, all + of the keyword arguments are treated as suffix data. + """ + + def __init__(self, *, variables=None, objective=None, objectives=None, **kwargs): + if objective is not None and objectives is not None: + raise ValueError( + "The objective= and objectives= keywords cannot both be specified." + ) + self.id = None + + self._variables = [] + self.name_to_variable = {} + self.fixed_variable_names = set() + if variables is not None: + self._variables = variables + for v in variables: + if v.name is not None: + if v.fixed: + self.fixed_variable_names.add(v.name) + self.name_to_variable[v.name] = v + + self._objectives = [] + self.name_to_objective = {} + if objective is not None: + objectives = [objective] + if objectives is not None: + self._objectives = objectives + for o in objectives: + if o.name is not None: + self.name_to_objective[o.name] = o + + if "suffix" in kwargs: + self.suffix = MyMunch(kwargs.pop("suffix")) + else: + self.suffix = MyMunch(**kwargs) + + def variable(self, index): + """Returns the specified variable. Parameters ---------- - model : ConcreteModel - A concrete Pyomo model. - variable_list: A collection of Pyomo _GeneralVarData variables - The variables for which the solution will be stored. - include_fixed : boolean - Boolean indicating that fixed variables should be added to the - solution. - objective: None or Objective - The objective functions for which the value will be saved. None - indicates that the active objective should be used, but a - different objective can be stored as well. + index : int or str + The index or name of the objective. (default is 0) + + Returns + ------- + VariableInfo """ + if type(index) is int: + return self._variables[index] + else: + return self.name_to_variable[index] - self.variables = ComponentMap() - self.fixed_vars = ComponentSet() - for var in variable_list: - is_fixed = var.is_fixed() - if is_fixed: - self.fixed_vars.add(var) - if include_fixed or not is_fixed: - self.variables[var] = pyo.value(var) - - if objective is None: - objective = aos_utils.get_active_objective(model) - self.objective = (objective, pyo.value(objective)) - - @property - def objective_value(self): + def variables(self): """ Returns ------- - The value of the objective. + list + The list of variables in the solution. """ - return self.objective[1] + return self._variables - def pprint(self, round_discrete=True, sort_keys=True, indent=4): - """ - Print the solution variables and objective values. + def objective(self, index=0): + """Returns the specified objective. Parameters ---------- - rounded_discrete : boolean - If True, then round discrete variable values before printing. - """ - print( - self.to_string( - round_discrete=round_discrete, sort_keys=sort_keys, indent=indent - ) - ) # pragma: no cover + index : int or str + The index or name of the objective. (default is 0) - def to_string(self, round_discrete=True, sort_keys=True, indent=4): - return json.dumps( - self.to_dict(round_discrete=round_discrete), - sort_keys=sort_keys, - indent=indent, - ) - - def to_dict(self, round_discrete=True): - ans = {} - ans["objective"] = str(self.objective[0]) - ans["objective_value"] = self.objective[1] - soln = {} - for variable, value in self.variables.items(): - val = self._round_variable_value(variable, value, round_discrete) - soln[variable.name] = val - ans["solution"] = soln - ans["fixed_variables"] = [str(v) for v in self.fixed_vars] - return ans + Returns + ------- + :py:class:`ObjectiveInfo` + """ + if type(index) is int: + return self._objectives[index] + else: + return self.name_to_objective[index] - def __str__(self): - return self.to_string() + def objectives(self): + """ + Returns + ------- + list + The list of objectives in the solution. + """ + return self._objectives - __repn__ = __str__ + def to_dict(self): + """ + Returns + ------- + dict + A dictionary representation of the solution. + """ + return dict( + id=self.id, + variables=[v.to_dict() for v in self.variables()], + objectives=[o.to_dict() for o in self.objectives()], + suffix=self.suffix.to_dict(), + ) - def get_variable_name_values(self, include_fixed=True, round_discrete=True): + def to_string(self, sort_keys=True, indent=4): """ - Get a dictionary of variable name-variable value pairs. + Returns a string representation of the solution, which is generated + from a dictionary representation of the solution. Parameters ---------- - include_fixed : boolean - If True, then include fixed variables in the dictionary. - round_discrete : boolean - If True, then round discrete variable values in the dictionary. + sort_keys : bool + If True, then sort the keys in the dictionary representation. (default is True) + indent : int + Specifies the number of whitespaces to indent each element of the dictionary. Returns ------- - Dictionary mapping variable names to variable values. + str + A string representation of the solution. """ - return { - var.name: self._round_variable_value(var, val, round_discrete) - for var, val in self.variables.items() - if include_fixed or not var in self.fixed_vars - } + return json.dumps(self.to_dict(), sort_keys=sort_keys, indent=indent) - def get_fixed_variable_names(self): - """ - Get a list of fixed-variable names. + def __str__(self): + return self.to_string() - Returns - ------- - A list of the variable names that are fixed. - """ - return [var.name for var in self.fixed_vars] + __repn__ = __str__ - def _round_variable_value(self, variable, value, round_discrete=True): + def _tuple_repn(self): """ - Returns a rounded value unless the variable is discrete or rounded_discrete is False. + Generate a tuple that represents the variables in the model. + + We use string names if possible, because they more explicit than the integer index values. """ - return value if not round_discrete or variable.is_continuous() else round(value) + if len(self.name_to_variable) == len(self._variables): + return tuple( + tuple([k, var.value]) for k, var in self.name_to_variable.items() + ) + else: + return tuple(tuple([k, var.value]) for k, var in enumerate(self._variables)) + + def __eq__(self, soln): + if not isinstance(soln, Solution): + return NotImplemented + return self._tuple_repn() == soln._tuple_repn() + + def __lt__(self, soln): + if not isinstance(soln, Solution): + return NotImplemented + return self._tuple_repn() <= soln._tuple_repn() + + +class PyomoSolution(Solution): + + def __init__(self, *, variables=None, objective=None, objectives=None, **kwargs): + # + # Q: Do we want to use an index relative to the list of variables specified here? Or use the Pyomo variable ID? + # Q: Should this object cache the Pyomo variable object? Or CUID? + # + # TODO: Capture suffix info here. + # + vlist = [] + if variables is not None: + index = 0 + for var in variables: + vlist.append( + VariableInfo( + value=( + pyo.value(var) + if var.is_continuous() + else round(pyo.value(var)) + ), + fixed=var.is_fixed(), + name=str(var), + index=index, + discrete=not var.is_continuous(), + ) + ) + index += 1 + + # + # TODO: Capture suffix info here. + # + if objective is not None: + objectives = [objective] + olist = [] + if objectives is not None: + index = 0 + for obj in objectives: + olist.append( + ObjectiveInfo(value=pyo.value(obj), name=str(obj), index=index) + ) + index += 1 + + super().__init__(variables=vlist, objectives=olist, **kwargs) diff --git a/pyomo/contrib/alternative_solutions/tests/test_aos_utils.py b/pyomo/contrib/alternative_solutions/tests/test_aos_utils.py index 481e79f68ca..ebe79092c16 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_aos_utils.py +++ b/pyomo/contrib/alternative_solutions/tests/test_aos_utils.py @@ -41,7 +41,7 @@ def test_multiple_objectives(self): assert_text = ( "Model has 3 active objective functions, exactly one " "is required." ) - with self.assertRaisesRegex(AssertionError, assert_text): + with self.assertRaisesRegex(RuntimeError, assert_text): au.get_active_objective(m) def test_no_objectives(self): @@ -52,7 +52,7 @@ def test_no_objectives(self): assert_text = ( "Model has 0 active objective functions, exactly one " "is required." ) - with self.assertRaisesRegex(AssertionError, assert_text): + with self.assertRaisesRegex(RuntimeError, assert_text): au.get_active_objective(m) def test_one_objective(self): diff --git a/pyomo/contrib/alternative_solutions/tests/test_balas.py b/pyomo/contrib/alternative_solutions/tests/test_balas.py index 984cde09a79..3a148249211 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_balas.py +++ b/pyomo/contrib/alternative_solutions/tests/test_balas.py @@ -11,34 +11,46 @@ from collections import Counter +from pyomo.common import unittest +from pyomo.common.dependencies import attempt_import from pyomo.common.dependencies import numpy as numpy, numpy_available +import pyomo.opt + +parameterized, param_available = attempt_import('parameterized') +if not param_available: + raise unittest.SkipTest('Parameterized is not available.') +parameterized = parameterized.parameterized if numpy_available: from numpy.testing import assert_array_almost_equal -from pyomo.common import unittest -import pyomo.opt - from pyomo.contrib.alternative_solutions import enumerate_binary_solutions import pyomo.contrib.alternative_solutions.tests.test_cases as tc solvers = list(pyomo.opt.check_available_solvers("glpk", "gurobi", "appsi_gurobi")) -pytestmark = unittest.pytest.mark.parametrize("mip_solver", solvers) -@unittest.pytest.mark.default -class TestBalasUnit: +class TestBalasUnit(unittest.TestCase): + @parameterized.expand(input=solvers) def test_bad_solver(self, mip_solver): """ Confirm that an exception is thrown with a bad solver name. """ m = tc.get_triangle_ip() - try: + with self.assertRaises(pyomo.common.errors.ApplicationError): enumerate_binary_solutions(m, solver="unknown_solver") - except pyomo.common.errors.ApplicationError as e: - pass + @parameterized.expand(input=solvers) + def test_non_positive_num_solutions(self, mip_solver): + """ + Confirm that an exception is thrown with a non-positive num solutions + """ + m = tc.get_triangle_ip() + with self.assertRaises(RuntimeError): + enumerate_binary_solutions(m, num_solutions=-1, solver=mip_solver) + + @parameterized.expand(input=solvers) def test_ip_feasibility(self, mip_solver): """ Enumerate solutions for an ip: triangle_ip. @@ -48,8 +60,10 @@ def test_ip_feasibility(self, mip_solver): m = tc.get_triangle_ip() results = enumerate_binary_solutions(m, num_solutions=100, solver=mip_solver) assert len(results) == 1 - assert results[0].objective_value == unittest.pytest.approx(5) + for soln in results: + assert soln.objective().value == unittest.pytest.approx(5) + @parameterized.expand(input=solvers) @unittest.skipIf(True, "Ignoring fragile test for solver timeout.") def test_no_time(self, mip_solver): """ @@ -63,6 +77,7 @@ def test_no_time(self, mip_solver): m, num_solutions=100, solver=mip_solver, solver_options={"TimeLimit": 0} ) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_knapsack_all(self, mip_solver): """ @@ -74,12 +89,13 @@ def test_knapsack_all(self, mip_solver): ) results = enumerate_binary_solutions(m, num_solutions=100, solver=mip_solver) objectives = list( - sorted((round(result.objective[1], 2) for result in results), reverse=True) + sorted((round(soln.objective().value, 2) for soln in results), reverse=True) ) assert_array_almost_equal(objectives, m.ranked_solution_values) unique_solns_by_obj = [val for val in Counter(objectives).values()] assert_array_almost_equal(unique_solns_by_obj, m.num_ranked_solns) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_knapsack_x0_x1(self, mip_solver): """ @@ -94,12 +110,13 @@ def test_knapsack_x0_x1(self, mip_solver): m, num_solutions=100, solver=mip_solver, variables=[m.x[0], m.x[1]] ) objectives = list( - sorted((round(result.objective[1], 2) for result in results), reverse=True) + sorted((round(soln.objective().value, 2) for soln in results), reverse=True) ) assert_array_almost_equal(objectives, [6, 5, 4, 3]) unique_solns_by_obj = [val for val in Counter(objectives).values()] assert_array_almost_equal(unique_solns_by_obj, [1, 1, 1, 1]) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_knapsack_optimal_3(self, mip_solver): """ @@ -111,10 +128,11 @@ def test_knapsack_optimal_3(self, mip_solver): ) results = enumerate_binary_solutions(m, num_solutions=3, solver=mip_solver) objectives = list( - sorted((round(result.objective[1], 2) for result in results), reverse=True) + sorted((round(soln.objective().value, 2) for soln in results), reverse=True) ) assert_array_almost_equal(objectives, m.ranked_solution_values[:3]) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_knapsack_hamming_3(self, mip_solver): """ @@ -128,10 +146,11 @@ def test_knapsack_hamming_3(self, mip_solver): m, num_solutions=3, solver=mip_solver, search_mode="hamming" ) objectives = list( - sorted((round(result.objective[1], 2) for result in results), reverse=True) + sorted((round(soln.objective().value, 2) for soln in results), reverse=True) ) assert_array_almost_equal(objectives, [6, 3, 1]) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_knapsack_random_3(self, mip_solver): """ @@ -145,7 +164,7 @@ def test_knapsack_random_3(self, mip_solver): m, num_solutions=3, solver=mip_solver, search_mode="random", seed=1118798374 ) objectives = list( - sorted((round(result.objective[1], 2) for result in results), reverse=True) + sorted((round(soln.objective().value, 2) for soln in results), reverse=True) ) assert_array_almost_equal(objectives, [6, 5, 4]) diff --git a/pyomo/contrib/alternative_solutions/tests/test_lp_enum_solnpool.py b/pyomo/contrib/alternative_solutions/tests/test_gurobi_lp_enum.py similarity index 70% rename from pyomo/contrib/alternative_solutions/tests/test_lp_enum_solnpool.py rename to pyomo/contrib/alternative_solutions/tests/test_gurobi_lp_enum.py index c46466779e1..7efff9bc775 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_lp_enum_solnpool.py +++ b/pyomo/contrib/alternative_solutions/tests/test_gurobi_lp_enum.py @@ -12,15 +12,14 @@ from pyomo.common.dependencies import numpy_available from pyomo.common import unittest +import pyomo.common.errors import pyomo.contrib.alternative_solutions.tests.test_cases as tc -from pyomo.contrib.alternative_solutions import lp_enum -from pyomo.contrib.alternative_solutions import lp_enum_solnpool +from pyomo.contrib.alternative_solutions import gurobi_enumerate_linear_solutions from pyomo.opt import check_available_solvers import pyomo.environ as pyo -# lp_enum_solnpool uses both 'gurobi' and 'appsi_gurobi' -gurobi_available = len(check_available_solvers('gurobi', 'appsi_gurobi')) == 2 +gurobi_available = len(check_available_solvers("gurobi")) == 2 # # TODO: Setup detailed tests here @@ -31,15 +30,20 @@ @unittest.skipUnless(numpy_available, "NumPy not found") class TestLPEnumSolnpool(unittest.TestCase): + def test_non_positive_num_solutions(self): + """ + Confirm that an exception is thrown with a non-positive num solutions + """ + n = tc.get_pentagonal_pyramid_mip() + with self.assertRaises(ValueError): + gurobi_enumerate_linear_solutions(n, num_solutions=-1) + def test_here(self): n = tc.get_pentagonal_pyramid_mip() n.x.domain = pyo.Reals n.y.domain = pyo.Reals - try: - sols = lp_enum_solnpool.enumerate_linear_solutions_soln_pool(n, tee=True) - except pyomo.common.errors.ApplicationError as e: - sols = [] + sols = gurobi_enumerate_linear_solutions(n, tee=True) # TODO - Confirm how solnpools deal with duplicate solutions if gurobi_available: diff --git a/pyomo/contrib/alternative_solutions/tests/test_gurobi_solnpool.py b/pyomo/contrib/alternative_solutions/tests/test_gurobi_solnpool.py new file mode 100644 index 00000000000..cff63b7b674 --- /dev/null +++ b/pyomo/contrib/alternative_solutions/tests/test_gurobi_solnpool.py @@ -0,0 +1,159 @@ +# ___________________________________________________________________________ +# +# Pyomo: Python Optimization Modeling Objects +# Copyright (c) 2008-2025 +# National Technology and Engineering Solutions of Sandia, LLC +# Under the terms of Contract DE-NA0003525 with National Technology and +# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain +# rights in this software. +# This software is distributed under the 3-clause BSD License. +# ___________________________________________________________________________ + +from collections import Counter + +from pyomo.common import unittest +from pyomo.common.dependencies import numpy as np, numpy_available + +from pyomo.contrib.alternative_solutions import gurobi_generate_solutions +from pyomo.contrib.appsi.solvers import Gurobi +import pyomo.contrib.alternative_solutions.tests.test_cases as tc + +gurobipy_available = Gurobi().available() + + +@unittest.skipIf(not gurobipy_available, "Gurobi MIP solver not available") +class TestGurobiSolnPoolUnit(unittest.TestCase): + """ + Cases to cover: + + LP feasibility (for an LP just one solution should be returned since gurobi cannot enumerate over continuous vars) + + Pass at least one solver option to make sure that work, e.g. time limit + + We need a utility to check that a two sets of solutions are the same. + Maybe this should be an AOS utility since it may be a thing we will want to do often. + """ + + def test_non_positive_num_solutions(self): + """ + Confirm that an exception is thrown with a non-positive num solutions + """ + m = tc.get_triangle_ip() + with self.assertRaises(ValueError): + gurobi_generate_solutions(m, num_solutions=-1) + + def test_search_mode(self): + """ + Confirm that an exception is thrown with pool_search_mode not in [1,2] + """ + m = tc.get_triangle_ip() + with self.assertRaises(ValueError): + gurobi_generate_solutions(m, pool_search_mode=0) + + @unittest.skipIf(not numpy_available, "Numpy not installed") + def test_ip_feasibility(self): + """ + Enumerate all solutions for an ip: triangle_ip. + + Check that the correct number of alternate solutions are found. + """ + m = tc.get_triangle_ip() + results = gurobi_generate_solutions(m, num_solutions=100) + objectives = [round(soln.objective().value, 2) for soln in results] + actual_solns_by_obj = m.num_ranked_solns + unique_solns_by_obj = [val for val in Counter(objectives).values()] + np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) + + @unittest.skipIf(not numpy_available, "Numpy not installed") + def test_ip_num_solutions(self): + """ + Enumerate 8 solutions for an ip: triangle_ip. + + Check that the correct number of alternate solutions are found. + """ + m = tc.get_triangle_ip() + results = gurobi_generate_solutions(m, num_solutions=8) + assert len(results) == 8 + objectives = [round(soln.objective().value, 2) for soln in results] + actual_solns_by_obj = [6, 2] + unique_solns_by_obj = [val for val in Counter(objectives).values()] + np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) + + @unittest.skipIf(not numpy_available, "Numpy not installed") + def test_mip_feasibility(self): + """ + Enumerate all solutions for a mip: indexed_pentagonal_pyramid_mip. + + Check that the correct number of alternate solutions are found. + """ + m = tc.get_indexed_pentagonal_pyramid_mip() + results = gurobi_generate_solutions(m, num_solutions=100) + objectives = [round(soln.objective().value, 2) for soln in results] + actual_solns_by_obj = m.num_ranked_solns + unique_solns_by_obj = [val for val in Counter(objectives).values()] + np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) + + @unittest.skipIf(not numpy_available, "Numpy not installed") + def test_mip_rel_feasibility(self): + """ + Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. + + Check that only solutions within a relative tolerance of 0.2 are + found. + """ + m = tc.get_pentagonal_pyramid_mip() + results = gurobi_generate_solutions(m, num_solutions=100, rel_opt_gap=0.2) + objectives = [round(soln.objective().value, 2) for soln in results] + actual_solns_by_obj = m.num_ranked_solns[0:2] + unique_solns_by_obj = [val for val in Counter(objectives).values()] + np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) + + @unittest.skipIf(not numpy_available, "Numpy not installed") + def test_mip_rel_feasibility_options(self): + """ + Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. + + Check that only solutions within a relative tolerance of 0.2 are + found. + """ + m = tc.get_pentagonal_pyramid_mip() + results = gurobi_generate_solutions( + m, num_solutions=100, solver_options={"PoolGap": 0.2} + ) + objectives = [round(soln.objective().value, 2) for soln in results] + actual_solns_by_obj = m.num_ranked_solns[0:2] + unique_solns_by_obj = [val for val in Counter(objectives).values()] + np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) + + @unittest.skipIf(not numpy_available, "Numpy not installed") + def test_mip_abs_feasibility(self): + """ + Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. + + Check that only solutions within an absolute tolerance of 1.99 are + found. + """ + m = tc.get_pentagonal_pyramid_mip() + results = gurobi_generate_solutions(m, num_solutions=100, abs_opt_gap=1.99) + objectives = [round(soln.objective().value, 2) for soln in results] + actual_solns_by_obj = m.num_ranked_solns[0:3] + unique_solns_by_obj = [val for val in Counter(objectives).values()] + np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) + + @unittest.skipIf(True, "Ignoring fragile test for solver timeout.") + def test_mip_no_time(self): + """ + Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. + + Check that no solutions are returned with a timelimit of 0. + """ + m = tc.get_pentagonal_pyramid_mip() + # Use quiet=False to test error message + results = gurobi_generate_solutions( + m, num_solutions=100, solver_options={"TimeLimit": 0.0}, quiet=False + ) + assert len(results) == 0 + + +if __name__ == "__main__": + unittest.main() diff --git a/pyomo/contrib/alternative_solutions/tests/test_lp_enum.py b/pyomo/contrib/alternative_solutions/tests/test_lp_enum.py index 27e6fe0cfb1..6f92a63b62a 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_lp_enum.py +++ b/pyomo/contrib/alternative_solutions/tests/test_lp_enum.py @@ -12,8 +12,14 @@ from pyomo.common.dependencies import numpy as numpy, numpy_available import pyomo.environ as pyo -from pyomo.common import unittest import pyomo.opt +from pyomo.common import unittest +from pyomo.common.dependencies import attempt_import + +parameterized, param_available = attempt_import('parameterized') +if not param_available: + raise unittest.SkipTest('Parameterized is not available.') +parameterized = parameterized.parameterized import pyomo.contrib.alternative_solutions.tests.test_cases as tc from pyomo.contrib.alternative_solutions import lp_enum @@ -21,28 +27,33 @@ # # Find available solvers. Just use GLPK if it's available. # -solvers = list( - pyomo.opt.check_available_solvers("glpk", "gurobi") -) # , "appsi_gurobi")) -pytestmark = unittest.pytest.mark.parametrize("mip_solver", solvers) +solvers = list(pyomo.opt.check_available_solvers("glpk", "gurobi")) timelimit = {"gurobi": "TimeLimit", "appsi_gurobi": "TimeLimit", "glpk": "tmlim"} -@unittest.pytest.mark.default -class TestLPEnum: +class TestLPEnum(unittest.TestCase): + @parameterized.expand(input=solvers) def test_bad_solver(self, mip_solver): """ Confirm that an exception is thrown with a bad solver name. """ m = tc.get_3d_polyhedron_problem() - try: + with self.assertRaises(pyomo.common.errors.ApplicationError): lp_enum.enumerate_linear_solutions(m, solver="unknown_solver") - except pyomo.common.errors.ApplicationError as e: - pass + + @parameterized.expand(input=solvers) + def test_non_positive_num_solutions(self, mip_solver): + """ + Confirm that an exception is thrown with a non-positive num solutions + """ + m = tc.get_3d_polyhedron_problem() + with self.assertRaises(ValueError): + lp_enum.enumerate_linear_solutions(m, num_solutions=-1, solver=mip_solver) @unittest.skipIf(True, "Ignoring fragile test for solver timeout.") + @parameterized.expand(input=solvers) def test_no_time(self, mip_solver): """ Check that the correct bounds are found for a discrete problem where @@ -54,6 +65,7 @@ def test_no_time(self, mip_solver): m, solver=mip_solver, solver_options={timelimit[mip_solver]: 0} ) + @parameterized.expand(input=solvers) def test_3d_polyhedron(self, mip_solver): m = tc.get_3d_polyhedron_problem() m.o.deactivate() @@ -62,8 +74,9 @@ def test_3d_polyhedron(self, mip_solver): sols = lp_enum.enumerate_linear_solutions(m, solver=mip_solver) assert len(sols) == 2 for s in sols: - assert s.objective_value == unittest.pytest.approx(4) + assert s.objective().value == unittest.pytest.approx(4) + @parameterized.expand(input=solvers) def test_3d_polyhedron(self, mip_solver): m = tc.get_3d_polyhedron_problem() m.o.deactivate() @@ -72,19 +85,21 @@ def test_3d_polyhedron(self, mip_solver): sols = lp_enum.enumerate_linear_solutions(m, solver=mip_solver) assert len(sols) == 2 for s in sols: - assert s.objective_value == unittest.pytest.approx( + assert s.objective().value == unittest.pytest.approx( 9 - ) or s.objective_value == unittest.pytest.approx(10) + ) or s.objective().value == unittest.pytest.approx(10) + @parameterized.expand(input=solvers) def test_2d_diamond_problem(self, mip_solver): m = tc.get_2d_diamond_problem() sols = lp_enum.enumerate_linear_solutions(m, solver=mip_solver, num_solutions=2) assert len(sols) == 2 for s in sols: print(s) - assert sols[0].objective_value == unittest.pytest.approx(6.789473684210527) - assert sols[1].objective_value == unittest.pytest.approx(3.6923076923076916) + assert sols[0].objective().value == unittest.pytest.approx(6.789473684210527) + assert sols[1].objective().value == unittest.pytest.approx(3.6923076923076916) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_pentagonal_pyramid(self, mip_solver): n = tc.get_pentagonal_pyramid_mip() @@ -97,6 +112,7 @@ def test_pentagonal_pyramid(self, mip_solver): print(s) assert len(sols) == 6 + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_pentagon(self, mip_solver): n = tc.get_pentagonal_lp() diff --git a/pyomo/contrib/alternative_solutions/tests/test_obbt.py b/pyomo/contrib/alternative_solutions/tests/test_obbt.py index ac40c31a1f4..3716344d2e3 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_obbt.py +++ b/pyomo/contrib/alternative_solutions/tests/test_obbt.py @@ -11,15 +11,20 @@ import math +import pyomo.environ as pyo +import pyomo.opt +from pyomo.common import unittest +from pyomo.common.dependencies import attempt_import from pyomo.common.dependencies import numpy as numpy, numpy_available if numpy_available: from numpy.testing import assert_array_almost_equal -import pyomo.environ as pyo -from pyomo.common import unittest +parameterized, param_available = attempt_import('parameterized') +if not param_available: + raise unittest.SkipTest('Parameterized is not available.') +parameterized = parameterized.parameterized -import pyomo.opt from pyomo.contrib.alternative_solutions import ( obbt_analysis_bounds_and_solutions, obbt_analysis, @@ -27,25 +32,23 @@ import pyomo.contrib.alternative_solutions.tests.test_cases as tc solvers = list(pyomo.opt.check_available_solvers("glpk", "gurobi", "appsi_gurobi")) -pytestmark = unittest.pytest.mark.parametrize("mip_solver", solvers) timelimit = {"gurobi": "TimeLimit", "appsi_gurobi": "TimeLimit", "glpk": "tmlim"} -@unittest.pytest.mark.default -class TestOBBTUnit: +class TestOBBTUnit(unittest.TestCase): + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_bad_solver(self, mip_solver): """ Confirm that an exception is thrown with a bad solver name. """ m = tc.get_2d_diamond_problem() - try: + with self.assertRaises(pyomo.common.errors.ApplicationError): obbt_analysis(m, solver="unknown_solver") - except pyomo.common.errors.ApplicationError as e: - pass + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_obbt_analysis(self, mip_solver): """ @@ -57,14 +60,16 @@ def test_obbt_analysis(self, mip_solver): for var, bounds in all_bounds.items(): assert_array_almost_equal(bounds, m.continuous_bounds[var]) + @parameterized.expand(input=solvers) def test_obbt_error1(self, mip_solver): """ ERROR: Cannot restrict variable list when warmstart is specified """ m = tc.get_2d_diamond_problem() - with unittest.pytest.raises(AssertionError): + with unittest.pytest.raises(ValueError): obbt_analysis_bounds_and_solutions(m, variables=[m.x], solver=mip_solver) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_obbt_some_vars(self, mip_solver): """ @@ -79,6 +84,7 @@ def test_obbt_some_vars(self, mip_solver): for var, bounds in all_bounds.items(): assert_array_almost_equal(bounds, m.continuous_bounds[var]) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_obbt_continuous(self, mip_solver): """ @@ -91,6 +97,7 @@ def test_obbt_continuous(self, mip_solver): for var, bounds in all_bounds.items(): assert_array_almost_equal(bounds, m.continuous_bounds[var]) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_mip_rel_objective(self, mip_solver): """ @@ -103,6 +110,7 @@ def test_mip_rel_objective(self, mip_solver): assert len(solns) == 2 * len(all_bounds) + 1 assert m._obbt.optimality_tol_rel.lb == unittest.pytest.approx(2.5) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_mip_abs_objective(self, mip_solver): """ @@ -115,6 +123,7 @@ def test_mip_abs_objective(self, mip_solver): assert len(solns) == 2 * len(all_bounds) + 1 assert m._obbt.optimality_tol_abs.lb == unittest.pytest.approx(3.01) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_obbt_warmstart(self, mip_solver): """ @@ -131,6 +140,7 @@ def test_obbt_warmstart(self, mip_solver): for var, bounds in all_bounds.items(): assert_array_almost_equal(bounds, m.continuous_bounds[var]) + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_obbt_mip(self, mip_solver): """ @@ -156,6 +166,7 @@ def test_obbt_mip(self, mip_solver): assert bounds_tightened assert bounds_not_tightened + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_obbt_unbounded(self, mip_solver): """ @@ -173,6 +184,7 @@ def test_obbt_unbounded(self, mip_solver): assert_array_almost_equal(bounds, m.continuous_bounds[var]) assert len(solns) == num + @parameterized.expand(input=solvers) @unittest.skipIf(not numpy_available, "Numpy not installed") def test_bound_tightening(self, mip_solver): """ @@ -187,6 +199,7 @@ def test_bound_tightening(self, mip_solver): assert_array_almost_equal(bounds, m.var_bounds[var]) @unittest.skipIf(True, "Ignoring fragile test for solver timeout.") + @parameterized.expand(input=solvers) def test_no_time(self, mip_solver): """ Check that the correct bounds are found for a discrete problem where @@ -198,6 +211,7 @@ def test_no_time(self, mip_solver): m, solver=mip_solver, solver_options={timelimit[mip_solver]: 0} ) + @parameterized.expand(input=solvers) def test_bound_refinement(self, mip_solver): """ Check that the correct bounds are found for a discrete problem where @@ -231,6 +245,7 @@ def test_bound_refinement(self, mip_solver): var, bounds[1] ) + @parameterized.expand(input=solvers) def test_obbt_infeasible(self, mip_solver): """ Check that code catches cases where the problem is infeasible. diff --git a/pyomo/contrib/alternative_solutions/tests/test_solnpool.py b/pyomo/contrib/alternative_solutions/tests/test_solnpool.py index 590a5eee4f7..555575760eb 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_solnpool.py +++ b/pyomo/contrib/alternative_solutions/tests/test_solnpool.py @@ -9,202 +9,965 @@ # This software is distributed under the 3-clause BSD License. # ___________________________________________________________________________ -from collections import Counter - -from pyomo.common.dependencies import numpy as np, numpy_available +from pyomo.common.unittest import pytest from pyomo.common import unittest -from pyomo.contrib.alternative_solutions import gurobi_generate_solutions -from pyomo.contrib.appsi.solvers import Gurobi - -import pyomo.contrib.alternative_solutions.tests.test_cases as tc -from pyomo.common.log import LoggingIntercept - -gurobipy_available = Gurobi().available() - - -@unittest.skipIf(not gurobipy_available, "Gurobi MIP solver not available") -class TestSolnPoolUnit(unittest.TestCase): - """ - Cases to cover: - - LP feasibility (for an LP just one solution should be returned since gurobi cannot enumerate over continuous vars) - - Pass at least one solver option to make sure that work, e.g. time limit - - We need a utility to check that a two sets of solutions are the same. - Maybe this should be an AOS utility since it may be a thing we will want to do often. - """ - - @unittest.skipIf(not numpy_available, "Numpy not installed") - def test_ip_feasibility(self): - """ - Enumerate all solutions for an ip: triangle_ip. - Check that the correct number of alternate solutions are found. - """ - m = tc.get_triangle_ip() - results = gurobi_generate_solutions(m, num_solutions=100) - objectives = [round(result.objective[1], 2) for result in results] - actual_solns_by_obj = m.num_ranked_solns - unique_solns_by_obj = [val for val in Counter(objectives).values()] - np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) - - def test_ip_num_solutions_best_effort(self): - """ - Enumerate solutions for an ip: triangle_ip. - Test best effort mode in solution pool. - - Check that the correct number of alternate solutions are found. - """ - m = tc.get_triangle_ip() - with LoggingIntercept() as LOG: - results = gurobi_generate_solutions( - m, num_solutions=8, solver_options={"PoolSearchMode": 1} +from pyomo.contrib.alternative_solutions import ( + PoolManager, + PoolPolicy, + Solution, + VariableInfo, + ObjectiveInfo, +) + + +def soln(value, objective): + return Solution( + variables=[VariableInfo(value=value)], + objectives=[ObjectiveInfo(value=objective)], + ) + + +class TestSolnPool(unittest.TestCase): + + def test_pool_active_name(self): + pm = PoolManager() + assert pm.name == None, "Should only have the None pool" + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + assert pm.name == "pool_1", "Should only have 'pool_1'" + + def test_get_pool_names(self): + pm = PoolManager() + assert pm.get_pool_names() == [None], "Should only be [None]" + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + assert pm.get_pool_names() == ["pool_1"], "Should only be ['pool_1']" + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + assert pm.get_pool_names() == [ + "pool_1", + "pool_2", + ], "Should be ['pool_1', 'pool_2']" + + def test_get_active_pool_policy(self): + pm = PoolManager() + assert pm.policy == PoolPolicy.keep_best, "Should only be 'keep_best'" + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + assert pm.policy == PoolPolicy.keep_all, "Should only be 'keep_best'" + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + assert pm.policy == PoolPolicy.keep_latest, "Should only be 'keep_latest'" + + def test_get_pool_policies(self): + pm = PoolManager() + assert pm.get_pool_policies() == { + None: PoolPolicy.keep_best + }, "Should only be {None : 'keep_best'}" + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + assert pm.get_pool_policies() == { + "pool_1": PoolPolicy.keep_all + }, "Should only be {'pool_1' : 'keep_best'}" + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + assert pm.get_pool_policies() == { + "pool_1": PoolPolicy.keep_all, + "pool_2": PoolPolicy.keep_latest, + }, "Should only be {'pool_1' : 'keep_best', 'pool_2' : 'keep_latest'}" + + def test_get_max_pool_size(self): + pm = PoolManager() + assert pm.max_pool_size == None, "Should only be None" + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + assert pm.max_pool_size == None, "Should only be None" + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + assert pm.max_pool_size == 1, "Should only be 1" + + def test_get_max_pool_sizes(self): + pm = PoolManager() + assert pm.get_max_pool_sizes() == {None: None}, "Should only be {None: None}" + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + assert pm.get_max_pool_sizes() == { + "pool_1": None + }, "Should only be {'pool_1': None}" + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + assert pm.get_max_pool_sizes() == { + "pool_1": None, + "pool_2": 1, + }, "Should only be {'pool_1': None, 'pool_2': 1}" + + def test_get_pool_sizes(self): + pm = PoolManager() + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 3 + + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + retval = pm.add(soln(0, 0)) + assert len(pm) == 1 + retval = pm.add(soln(0, 1)) + + assert pm.get_pool_sizes() == { + "pool_1": 3, + "pool_2": 1, + }, "Should be {'pool_1' :3, 'pool_2' : 1}" + + def test_multiple_pools(self): + pm = PoolManager() + pm.add_pool(name="pool_1", policy=PoolPolicy.keep_all) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 3 + + assert pm.get_pool_dicts() == { + "pool_1": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool_1", + "policy": "keep_all", + }, + "pool_config": {}, + "solutions": { + 0: { + "id": 0, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 0} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 1: { + "id": 1, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 1, + } + ], + }, + }, + } + } + pm.add_pool(name="pool_2", policy=PoolPolicy.keep_latest, max_pool_size=1) + retval = pm.add(soln(0, 0)) + assert len(pm) == 1 + retval = pm.add(soln(0, 1)) + assert pm.get_pool_dicts() == { + "pool_1": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool_1", + "policy": "keep_all", + }, + "solutions": { + 0: { + "id": 0, + "variables": [ + { + "value": 0, + "fixed": False, + "name": None, + "index": None, + "discrete": False, + "suffix": {}, + } + ], + "objectives": [ + {"value": 0, "name": None, "index": None, "suffix": {}} + ], + "suffix": {}, + }, + 1: { + "id": 1, + "variables": [ + { + "value": 0, + "fixed": False, + "name": None, + "index": None, + "discrete": False, + "suffix": {}, + } + ], + "objectives": [ + {"value": 1, "name": None, "index": None, "suffix": {}} + ], + "suffix": {}, + }, + 2: { + "id": 2, + "variables": [ + { + "value": 1, + "fixed": False, + "name": None, + "index": None, + "discrete": False, + "suffix": {}, + } + ], + "objectives": [ + {"value": 1, "name": None, "index": None, "suffix": {}} + ], + "suffix": {}, + }, + }, + "pool_config": {}, + }, + "pool_2": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool_2", + "policy": "keep_latest", + }, + "solutions": { + 4: { + "id": 4, + "variables": [ + { + "value": 0, + "fixed": False, + "name": None, + "index": None, + "discrete": False, + "suffix": {}, + } + ], + "objectives": [ + {"value": 1, "name": None, "index": None, "suffix": {}} + ], + "suffix": {}, + } + }, + "pool_config": {"max_pool_size": 1}, + }, + } + assert len(pm) == 1 + + def test_keepall_add(self): + pm = PoolManager() + pm.add_pool(name="pool", policy=PoolPolicy.keep_all) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 3 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_all", + }, + "pool_config": {}, + "solutions": { + 0: { + "id": 0, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 0} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 1: { + "id": 1, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 1, + } + ], + }, + }, + } + } + + def Xtest_invalid_policy_1(self): + pm = PoolManager() + with self.assertRaises(ValueError): + pm.add_pool(name="pool", policy=PoolPolicy.invalid_policy) + + def Xtest_invalid_policy_2(self): + pm = PoolManager() + with self.assertRaises(ValueError): + pm.add_pool(name="pool", policy=PoolPolicy.invalid_policy, max_pool_size=-2) + + def test_keeplatest_bad_max_pool_size(self): + pm = PoolManager() + with self.assertRaises(ValueError): + pm.add_pool(name="pool", policy=PoolPolicy.keep_latest, max_pool_size=-2) + + def test_keeplatest_add(self): + pm = PoolManager() + pm.add_pool(name="pool", policy=PoolPolicy.keep_latest, max_pool_size=2) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 2 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_latest", + }, + "pool_config": {"max_pool_size": 2}, + "solutions": { + 1: { + "id": 1, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 1, + } + ], + }, + }, + } + } + + def test_keeplatestunique_bad_max_pool_size(self): + pm = PoolManager() + with self.assertRaises(ValueError): + pm.add_pool( + name="pool", policy=PoolPolicy.keep_latest_unique, max_pool_size=-2 ) - self.assertRegex( - 'Running gurobi_solnpool with PoolSearchMode=1, best effort search may lead to unexpected behavior\n', - LOG.getvalue(), - ) - assert len(results) >= 1, 'Need to find some solutions' - - def test_ip_num_solutions_standard_single_solution_solve(self): - """ - Enumerate solutions for an ip: triangle_ip. - Test single solve mode in solution pool. - - Check that the correct number of solutions (1) are found. - This is not the intended use case for this method. - This is a warning check. - """ - m = tc.get_triangle_ip() - with LoggingIntercept() as LOG: - results = gurobi_generate_solutions( - m, num_solutions=8, solver_options={"PoolSearchMode": 0} - ) - self.assertRegex( - 'Running gurobi_solnpool with PoolSearchMode=0, this is single search mode and not the intended use case for gurobi_generate_solutions\n', - LOG.getvalue(), - ) - assert len(results) == 1, 'Need to find only 1 solution' - - def test_ip_num_solutions_seeking_one(self): - """ - Enumerate solutions for an ip: triangle_ip. - Test case where only one solution is asked for. - - This is not the intended use case for this code. - This is a warning check. - """ - m = tc.get_triangle_ip() - with LoggingIntercept() as LOG: - results = gurobi_generate_solutions(m, num_solutions=1) - self.assertRegex( - 'Running alternative_solutions method to find only 1 solution!\n', - LOG.getvalue(), - ) - assert len(results) == 1, 'Need to find only 1 solution' - - def test_ip_num_solutions_seeking_zero(self): - """ - Enumerate solutions for an ip: triangle_ip. - Test case where zero solutions are asked for to check assert error. - """ - m = tc.get_triangle_ip() - with self.assertRaisesRegex( - AssertionError, "num_solutions must be positive integer" - ): - gurobi_generate_solutions(m, num_solutions=0) - - @unittest.skipIf(not numpy_available, "Numpy not installed") - def test_ip_num_solutions(self): - """ - Enumerate 8 solutions for an ip: triangle_ip. - - Check that the correct number of alternate solutions are found. - """ - m = tc.get_triangle_ip() - results = gurobi_generate_solutions(m, num_solutions=8) - assert len(results) == 8 - objectives = [round(result.objective[1], 2) for result in results] - actual_solns_by_obj = [6, 2] - unique_solns_by_obj = [val for val in Counter(objectives).values()] - np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) - - @unittest.skipIf(not numpy_available, "Numpy not installed") - def test_mip_feasibility(self): - """ - Enumerate all solutions for a mip: indexed_pentagonal_pyramid_mip. - - Check that the correct number of alternate solutions are found. - """ - m = tc.get_indexed_pentagonal_pyramid_mip() - results = gurobi_generate_solutions(m, num_solutions=100) - objectives = [round(result.objective[1], 2) for result in results] - actual_solns_by_obj = m.num_ranked_solns - unique_solns_by_obj = [val for val in Counter(objectives).values()] - np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) - - @unittest.skipIf(not numpy_available, "Numpy not installed") - def test_mip_rel_feasibility(self): - """ - Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. - - Check that only solutions within a relative tolerance of 0.2 are - found. - """ - m = tc.get_pentagonal_pyramid_mip() - results = gurobi_generate_solutions(m, num_solutions=100, rel_opt_gap=0.2) - objectives = [round(result.objective[1], 2) for result in results] - actual_solns_by_obj = m.num_ranked_solns[0:2] - unique_solns_by_obj = [val for val in Counter(objectives).values()] - np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) - - @unittest.skipIf(not numpy_available, "Numpy not installed") - def test_mip_rel_feasibility_options(self): - """ - Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. - - Check that only solutions within a relative tolerance of 0.2 are - found. - """ - m = tc.get_pentagonal_pyramid_mip() - results = gurobi_generate_solutions( - m, num_solutions=100, solver_options={"PoolGap": 0.2} - ) - objectives = [round(result.objective[1], 2) for result in results] - actual_solns_by_obj = m.num_ranked_solns[0:2] - unique_solns_by_obj = [val for val in Counter(objectives).values()] - np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) - - @unittest.skipIf(not numpy_available, "Numpy not installed") - def test_mip_abs_feasibility(self): - """ - Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. - - Check that only solutions within an absolute tolerance of 1.99 are - found. - """ - m = tc.get_pentagonal_pyramid_mip() - results = gurobi_generate_solutions(m, num_solutions=100, abs_opt_gap=1.99) - objectives = [round(result.objective[1], 2) for result in results] - actual_solns_by_obj = m.num_ranked_solns[0:3] - unique_solns_by_obj = [val for val in Counter(objectives).values()] - np.testing.assert_array_almost_equal(unique_solns_by_obj, actual_solns_by_obj) - - @unittest.skipIf(True, "Ignoring fragile test for solver timeout.") - def test_mip_no_time(self): - """ - Enumerate solutions for a mip: indexed_pentagonal_pyramid_mip. - - Check that no solutions are returned with a timelimit of 0. - """ - m = tc.get_pentagonal_pyramid_mip() - # Use quiet=False to test error message - results = gurobi_generate_solutions( - m, num_solutions=100, solver_options={"TimeLimit": 0.0}, quiet=False - ) - assert len(results) == 0 + def test_keeplatestunique_add(self): + pm = PoolManager() + pm.add_pool(name="pool", policy=PoolPolicy.keep_latest_unique, max_pool_size=2) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) + assert retval is None + assert len(pm) == 1 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 2 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_latest_unique", + }, + "pool_config": {"max_pool_size": 2}, + "solutions": { + 0: { + "id": 0, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 0} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 1: { + "id": 1, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 1, + } + ], + }, + }, + } + } + + def test_keepbest_bad_max_pool_size(self): + pm = PoolManager() + with self.assertRaises(ValueError): + pm.add_pool(name="pool", policy=PoolPolicy.keep_best, max_pool_size=-2) + + def test_pool_manager_to_dict_passthrough(self): + pm = PoolManager() + pm = PoolManager() + pm.add_pool(name="pool", policy=PoolPolicy.keep_best, abs_tolerance=1) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) # not unique + assert retval is None + assert len(pm) == 1 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 2 + + assert pm.to_dict() == { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_best", + }, + "pool_config": { + 'abs_tolerance': 1, + 'best_value': 0, + 'max_pool_size': None, + 'objective': 0, + 'rel_tolerance': None, + 'sense_is_min': True, + }, + "solutions": { + 0: { + "id": 0, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 0} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 1: { + "id": 1, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 1, + } + ], + }, + }, + } + + def test_keepbest_add1(self): + pm = PoolManager() + pm.add_pool(name="pool", policy=PoolPolicy.keep_best, abs_tolerance=1) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) # not unique + assert retval is None + assert len(pm) == 1 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 2 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_best", + }, + "pool_config": { + 'abs_tolerance': 1, + 'best_value': 0, + 'max_pool_size': None, + 'objective': 0, + 'rel_tolerance': None, + 'sense_is_min': True, + }, + "solutions": { + 0: { + "id": 0, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 0} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 1: { + "id": 1, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 1, + } + ], + }, + }, + } + } + + def test_keepbest_add2(self): + pm = PoolManager() + pm.add_pool(name="pool", policy=PoolPolicy.keep_best, abs_tolerance=1) + + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) # not unique + assert retval is None + assert len(pm) == 1 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(2, -1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(3, -0.5)) + assert retval is not None + assert len(pm) == 3 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_best", + }, + "pool_config": { + 'abs_tolerance': 1, + 'best_value': -1, + 'max_pool_size': None, + 'objective': 0, + 'rel_tolerance': None, + 'sense_is_min': True, + }, + "solutions": { + 0: { + "id": 0, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": 0} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 0, + } + ], + }, + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 2, + } + ], + }, + 3: { + "id": 3, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -0.5} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 3, + } + ], + }, + }, + } + } + + retval = pm.add(soln(4, -1.5)) + assert retval is not None + assert len(pm) == 3 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_best", + }, + "pool_config": { + "abs_tolerance": 1, + 'best_value': -1.5, + "max_pool_size": None, + "objective": 0, + "rel_tolerance": None, + 'sense_is_min': True, + }, + "solutions": { + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 2, + } + ], + }, + 3: { + "id": 3, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -0.5} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 3, + } + ], + }, + 4: { + "id": 4, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -1.5} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 4, + } + ], + }, + }, + } + } + + def test_keepbest_add3(self): + pm = PoolManager() + pm.add_pool( + name="pool", policy=PoolPolicy.keep_best, abs_tolerance=1, max_pool_size=2 + ) -if __name__ == "__main__": - unittest.main() + retval = pm.add(soln(0, 0)) + assert retval is not None + assert len(pm) == 1 + + retval = pm.add(soln(0, 1)) # not unique + assert retval is None + assert len(pm) == 1 + + retval = pm.add(soln(1, 1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(2, -1)) + assert retval is not None + assert len(pm) == 2 + + retval = pm.add(soln(3, -0.5)) + assert retval is not None + assert len(pm) == 2 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_best", + }, + "pool_config": { + "abs_tolerance": 1, + 'best_value': -1, + "max_pool_size": 2, + "objective": 0, + "rel_tolerance": None, + 'sense_is_min': True, + }, + "solutions": { + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 2, + } + ], + }, + 3: { + "id": 3, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -0.5} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 3, + } + ], + }, + }, + } + } + + retval = pm.add(soln(4, -1.5)) + assert retval is not None + assert len(pm) == 2 + + assert pm.get_pool_dicts() == { + "pool": { + "metadata": { + "as_solution_source": "pyomo.contrib.alternative_solutions.solnpool.default_as_solution", + "context_name": "pool", + "policy": "keep_best", + }, + "pool_config": { + "abs_tolerance": 1, + 'best_value': -1.5, + "max_pool_size": 2, + "objective": 0, + "rel_tolerance": None, + "sense_is_min": True, + }, + "solutions": { + 2: { + "id": 2, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -1} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 2, + } + ], + }, + 4: { + "id": 4, + "objectives": [ + {"index": None, "name": None, "suffix": {}, "value": -1.5} + ], + "suffix": {}, + "variables": [ + { + "discrete": False, + "fixed": False, + "index": None, + "name": None, + "suffix": {}, + "value": 4, + } + ], + }, + }, + } + } diff --git a/pyomo/contrib/alternative_solutions/tests/test_solution.py b/pyomo/contrib/alternative_solutions/tests/test_solution.py index 961068420be..5e33b64b5c6 100644 --- a/pyomo/contrib/alternative_solutions/tests/test_solution.py +++ b/pyomo/contrib/alternative_solutions/tests/test_solution.py @@ -13,13 +13,15 @@ import pyomo.environ as pyo import pyomo.common.unittest as unittest import pyomo.contrib.alternative_solutions.aos_utils as au -from pyomo.contrib.alternative_solutions import Solution +from pyomo.contrib.alternative_solutions import PyomoSolution +from pyomo.contrib.alternative_solutions import enumerate_binary_solutions -mip_solver = "gurobi" -mip_available = pyomo.opt.check_available_solvers(mip_solver) +solvers = list(pyomo.opt.check_available_solvers("glpk", "gurobi")) +pytestmark = unittest.pytest.mark.parametrize("mip_solver", solvers) -class TestSolutionUnit(unittest.TestCase): +@unittest.pytest.mark.default +class TestSolutionUnit: def get_model(self): """ @@ -40,8 +42,7 @@ def get_model(self): m.con_z = pyo.Constraint(expr=m.z <= 3) return m - @unittest.skipUnless(mip_available, "MIP solver not available") - def test_solution(self): + def test_solution(self, mip_solver): """ Create a Solution Object, call its functions, and ensure the correct data is returned. @@ -49,44 +50,131 @@ def test_solution(self): model = self.get_model() opt = pyo.SolverFactory(mip_solver) opt.solve(model) - all_vars = au.get_model_variables(model, include_fixed=True) + all_vars = au.get_model_variables(model, include_fixed=False) + obj = au.get_active_objective(model) - solution = Solution(model, all_vars, include_fixed=False) + solution = PyomoSolution(variables=all_vars, objective=obj) sol_str = """{ - "fixed_variables": [ - "f" + "id": null, + "objectives": [ + { + "index": 0, + "name": "obj", + "suffix": {}, + "value": 6.5 + } ], - "objective": "obj", - "objective_value": 6.5, - "solution": { - "x": 1.5, - "y": 1, - "z": 3 - } + "suffix": {}, + "variables": [ + { + "discrete": false, + "fixed": false, + "index": 0, + "name": "x", + "suffix": {}, + "value": 1.5 + }, + { + "discrete": true, + "fixed": false, + "index": 1, + "name": "y", + "suffix": {}, + "value": 1 + }, + { + "discrete": true, + "fixed": false, + "index": 2, + "name": "z", + "suffix": {}, + "value": 3 + } + ] }""" assert str(solution) == sol_str - solution = Solution(model, all_vars) + all_vars = au.get_model_variables(model, include_fixed=True) + solution = PyomoSolution(variables=all_vars, objective=obj) sol_str = """{ - "fixed_variables": [ - "f" + "id": null, + "objectives": [ + { + "index": 0, + "name": "obj", + "suffix": {}, + "value": 6.5 + } ], - "objective": "obj", - "objective_value": 6.5, - "solution": { - "f": 1, - "x": 1.5, - "y": 1, - "z": 3 - } + "suffix": {}, + "variables": [ + { + "discrete": false, + "fixed": false, + "index": 0, + "name": "x", + "suffix": {}, + "value": 1.5 + }, + { + "discrete": true, + "fixed": false, + "index": 1, + "name": "y", + "suffix": {}, + "value": 1 + }, + { + "discrete": true, + "fixed": false, + "index": 2, + "name": "z", + "suffix": {}, + "value": 3 + }, + { + "discrete": false, + "fixed": true, + "index": 3, + "name": "f", + "suffix": {}, + "value": 1 + } + ] }""" - assert solution.to_string(round_discrete=True) == sol_str + assert solution.to_string() == sol_str + + sol_val = solution.name_to_variable + assert set(sol_val.keys()) == {"x", "y", "z", "f"} + assert set(solution.fixed_variable_names) == {"f"} + + def test_soln_order(self, mip_solver): + """ """ + values = [10, 9, 2, 1, 1] + weights = [10, 9, 2, 1, 1] + + K = len(values) + capacity = 12 + + m = pyo.ConcreteModel() + m.x = pyo.Var(range(K), within=pyo.Binary) + m.o = pyo.Objective( + expr=sum(values[i] * m.x[i] for i in range(K)), sense=pyo.maximize + ) + m.c = pyo.Constraint( + expr=sum(weights[i] * m.x[i] for i in range(K)) <= capacity + ) - sol_val = solution.get_variable_name_values( - include_fixed=True, round_discrete=True + solns = enumerate_binary_solutions( + m, num_solutions=10, solver=mip_solver, abs_opt_gap=0.5 ) - self.assertEqual(set(sol_val.keys()), {"x", "y", "z", "f"}) - self.assertEqual(set(solution.get_fixed_variable_names()), {"f"}) + assert len(solns) == 4 + assert [[v.value for v in soln.variables()] for soln in sorted(solns)] == [ + [0, 1, 1, 0, 1], + [0, 1, 1, 1, 0], + [1, 0, 0, 1, 1], + [1, 0, 1, 0, 0], + ] if __name__ == "__main__":