MindtPy: maintain a valid dual bound in each iteration when no-good c…

doc/OnlineDocs/explanation/solvers/mindtpy.rst

@@ -270,7 +270,9 @@ MindtPy provides two ways to guarantee the finite convergence of the algorithm.
 Bound Calculation
 """""""""""""""""
 
-Since no-good cuts or tabu list is applied in the Global Outer-Approximation (GOA) method, the MILP master problem cannot provide a valid bound for the original problem. After the GOA method has converged, MindtPy will remove the no-good cuts or the tabu integer combinations added when and after the optimal solution has been found. Solving this problem will give us a valid bound for the original problem.
+With no-good cuts or a tabu list, the MILP master problem bounds only the remaining integer assignments. MindtPy combines that bound with the incumbent objective: it takes the minimum for minimization, or the maximum for maximization. If no feasible assignments remain in the master problem, the incumbent gives both bounds and MindtPy terminates as optimal. No additional master solve is required.
+
+This calculation assumes that every excluded fixed NLP was solved to optimality or proven infeasible, and that the master problem provides a valid relaxation. OA requires model convexity; GOA requires global NLP solves. A local optimum is sufficient under the convex algorithms' assumptions, but not for GOA. If a fixed NLP instead reports a merely feasible solution, no solution, or a solver limit, MindtPy retains the dual bound obtained before that assignment was excluded. Exhausting the remaining assignments then reports a feasible solution (or no solution if there is no incumbent), without claiming optimality or infeasibility from the exhausted master alone. Using OA as a heuristic on a nonconvex model does not guarantee valid bounds.
 
 
 The GOA method also has a single-tree implementation with ``cplex_persistent`` and ``gurobi_persistent``. Notice that this method is more computationally expensive than the other strategies implemented for convex MINLP like OA and ECP, which can be used as heuristics for nonconvex MINLP problems.

pyomo/contrib/mindtpy/algorithm_base_class.py

@@ -105,6 +105,7 @@ def __init__(self, **kwds):
         self.curr_int_sol = []
         self.should_terminate = False
         self.integer_list = []
+        self._integer_exclusion_bound_valid = True
         # Dictionary {integer solution (tuple): [cuts begin index, cuts end index] (list)}
         self.integer_solution_to_cuts_index = dict()
 
@@ -143,9 +144,6 @@ def __init__(self, **kwds):
         self.best_solution_found = None
         self.best_solution_found_time = None
 
-        self.stored_bound = {}
-        self.num_no_good_cuts_added = {}
-        self.last_iter_cuts = False
         # Store the OA cuts generated in the mip_start_process.
         self.mip_start_lazy_oa_cuts = []
         # Whether to load solutions in solve() function
@@ -800,6 +798,20 @@ def update_dual_bound(self, bound_value):
         """
         if math.isnan(bound_value):
             return
+        # Under the algorithm's assumptions (convexity for OA, globally solved
+        # subproblems for GOA), the incumbent bounds the optimum over excluded
+        # assignments only if their NLPs were solved to optimality or infeasibility.
+        # The main problem bounds the remaining assignments, so combine its bound
+        # with the incumbent. If an excluded NLP was unresolved, retain the last
+        # bound obtained before that exclusion instead.
+        if self.config.add_no_good_cuts or self.config.use_tabu_list:
+            if not self._integer_exclusion_bound_valid:
+                self.dual_bound_improved = False
+                return
+            if self.objective_sense == minimize:
+                bound_value = min(self.primal_bound, bound_value)
+            else:
+                bound_value = max(self.primal_bound, bound_value)
         if self.objective_sense == minimize:
             self.dual_bound = max(bound_value, self.dual_bound)
             self.dual_bound_improved = self.dual_bound > self.dual_bound_progress[-1]
@@ -984,6 +996,7 @@ def set_up_solve_data(self, model):
             The original model to be solved in MindtPy.
         """
         config = self.config
+        self._integer_exclusion_bound_valid = True
         self.original_model = model
         obj, objective_count, dummy_name = self._get_main_objective(
             model, logger=config.logger
@@ -1374,6 +1387,22 @@ def solve_subproblem(self):
         TransformationFactory('contrib.deactivate_trivial_constraints').revert(
             self.fixed_nlp
         )
+        # Both the ordinary loop and the single-tree callbacks solve fixed NLPs
+        # here. Record unresolved subproblems before either can exclude an integer
+        # assignment with a no-good cut or the tabu list. Local optimality suffices
+        # for the convex algorithms, but not for GOA.
+        if config.add_no_good_cuts or config.use_tabu_list:
+            termination = results.solver.termination_condition
+            if termination not in {tc.optimal, tc.infeasible} and not (
+                termination is tc.locallyOptimal and config.strategy != 'GOA'
+            ):
+                if self._integer_exclusion_bound_valid:
+                    config.logger.info(
+                        'Fixed NLP subproblem terminated with %s. Retaining the '
+                        'last dual bound before excluding unresolved assignments.',
+                        termination,
+                    )
+                self._integer_exclusion_bound_valid = False
         return self.fixed_nlp, results
 
     def handle_nlp_subproblem_tc(self, fixed_nlp, result, cb_opt=None):
@@ -1781,82 +1810,6 @@ def algorithm_should_terminate(self, check_cycling):
             or (check_cycling and self.iteration_cycling())
         )
 
-    def fix_dual_bound(self, last_iter_cuts):
-        """Fix the dual bound when no-good cuts or tabu list is activated.
-
-        Parameters
-        ----------
-        last_iter_cuts : bool
-            Whether the cuts in the last iteration have been added.
-        """
-        # If no-good cuts or tabu list is activated, the dual bound is not valid for the final optimal solution.
-        # Therefore, we need to correct it at the end.
-        # In singletree implementation, the dual bound at one iteration before the optimal solution, is valid for the optimal solution.
-        # So we will set the dual bound to it.
-        config = self.config
-        if config.single_tree:
-            config.logger.info(
-                'Fix the bound to the value of one iteration before optimal solution is found.'
-            )
-            try:
-                self.dual_bound = self.stored_bound[self.primal_bound]
-            except KeyError as e:
-                config.logger.error(e, exc_info=True)
-                config.logger.error('No stored bound found. Bound fix failed.')
-        else:
-            config.logger.info(
-                'Solve the main problem without the last no_good cut to fix the bound.'
-                'zero_tolerance is set to 1E-4'
-            )
-            config.zero_tolerance = 1e-4
-            # Solve NLP subproblem
-            # The constraint linearization happens in the handlers
-            if not last_iter_cuts:
-                fixed_nlp, fixed_nlp_result = self.solve_subproblem()
-                self.handle_nlp_subproblem_tc(fixed_nlp, fixed_nlp_result)
-
-            MindtPy = self.mip.MindtPy_utils
-            # Deactivate the integer cuts generated after the best solution was found.
-            self.deactivate_no_good_cuts_when_fixing_bound(MindtPy.cuts.no_good_cuts)
-            if (
-                config.add_regularization is not None
-                and MindtPy.component('mip_obj') is None
-            ):
-                MindtPy.objective_list[-1].activate()
-            # determine if persistent solver is called.
-            if isinstance(self.mip_opt, PersistentSolver):
-                self.mip_opt.set_instance(self.mip, symbolic_solver_labels=True)
-            mip_args = dict(config.mip_solver_args)
-            update_solver_timelimit(
-                self.mip_opt, config.mip_solver, self.timing, config
-            )
-            main_mip_results = self.mip_opt.solve(
-                self.mip,
-                tee=config.mip_solver_tee,
-                load_solutions=self.mip_load_solutions,
-                **mip_args,
-            )
-            if len(main_mip_results.solution) > 0:
-                self.mip.solutions.load_from(main_mip_results)
-
-            if main_mip_results.solver.termination_condition is tc.infeasible:
-                config.logger.info(
-                    'Bound fix failed. The bound fix problem is infeasible'
-                )
-            else:
-                self.update_suboptimal_dual_bound(main_mip_results)
-                config.logger.info(
-                    'Fixed bound values: Primal Bound: {}  Dual Bound: {}'.format(
-                        self.primal_bound, self.dual_bound
-                    )
-                )
-            # Check bound convergence
-            if (
-                abs(self.primal_bound - self.dual_bound)
-                <= config.absolute_bound_tolerance
-            ):
-                self.results.solver.termination_condition = tc.optimal
-
     def set_up_tabulist_callback(self):
         """Sets up the tabulist using IncumbentCallback.
         Currently only support CPLEX.
@@ -1952,7 +1905,9 @@ def solve_main(self):
                 self.mip_opt._pyomo_var_to_solver_var_map
             )
         if main_mip_results.solver.termination_condition is tc.optimal:
-            if config.single_tree and not config.add_no_good_cuts:
+            if config.single_tree:
+                # With no-good cuts or tabu list, the bound of the B&B tree is
+                # clamped to the primal bound in update_dual_bound.
                 self.update_suboptimal_dual_bound(main_mip_results)
         elif main_mip_results.solver.termination_condition is tc.infeasibleOrUnbounded:
             # Linear solvers will sometimes tell me that it's infeasible or
@@ -2147,19 +2102,33 @@ def handle_main_infeasible(self):
             self.config.logger.warning(
                 'MindtPy initialization may have generated poor quality cuts.'
             )
-        # TODO no-good cuts for single tree case
-        # set optimistic bound to infinity
         self.config.logger.info(
             'MindtPy exiting due to MILP main problem infeasibility.'
         )
         if self.results.solver.termination_condition is None:
+            has_integer_exclusions = (
+                self.config.add_no_good_cuts or self.config.use_tabu_list
+            )
+            if (
+                has_integer_exclusions
+                and self._integer_exclusion_bound_valid
+                and math.isfinite(self.primal_bound)
+            ):
+                # The remaining assignments are infeasible. The incumbent is
+                # therefore optimal over all assignments, including the excluded
+                # ones, without another main problem solve.
+                self.update_dual_bound(self.primal_bound)
+                if self.bounds_converged():
+                    return
             if (
                 self.primal_bound == float('inf') and self.objective_sense == minimize
             ) or (
                 self.primal_bound == float('-inf') and self.objective_sense == maximize
             ):
-                # if self.mip_iter == 0:
-                self.results.solver.termination_condition = tc.infeasible
+                if has_integer_exclusions and not self._integer_exclusion_bound_valid:
+                    self.results.solver.termination_condition = tc.noSolution
+                else:
+                    self.results.solver.termination_condition = tc.infeasible
             else:
                 self.results.solver.termination_condition = tc.feasible
 
@@ -3126,13 +3095,12 @@ def objective_reformulation(self):
     def handle_main_mip_termination(self, main_mip, main_mip_results):
         should_terminate = False
         if main_mip_results is not None:
-            if not self.config.single_tree:
+            if main_mip_results.solver.termination_condition is tc.infeasible:
+                self.handle_main_infeasible()
+                should_terminate = True
+            elif not self.config.single_tree:
                 if main_mip_results.solver.termination_condition is tc.optimal:
                     self.handle_main_optimal(main_mip)
-                elif main_mip_results.solver.termination_condition is tc.infeasible:
-                    self.handle_main_infeasible()
-                    self.last_iter_cuts = True
-                    should_terminate = True
                 elif main_mip_results.solver.termination_condition is tc.unbounded:
                     temp_results = self.handle_main_unbounded(main_mip)
                 elif (
@@ -3236,7 +3204,6 @@ def MindtPy_iteration_loop(self):
                         self.handle_nlp_subproblem_tc(fixed_nlp, fixed_nlp_result)
 
             if self.algorithm_should_terminate(check_cycling=True):
-                self.last_iter_cuts = False
                 break
 
             if not config.single_tree:  # if we don't use lazy callback, i.e. LP_NLP
@@ -3255,7 +3222,6 @@ def MindtPy_iteration_loop(self):
                         config.call_after_subproblem_solve(fixed_nlp)
 
                     if self.algorithm_should_terminate(check_cycling=False):
-                        self.last_iter_cuts = True
                         break
                 else:
                     solution_name_obj = self.get_solution_name_obj(main_mip_results)
@@ -3292,19 +3258,8 @@ def MindtPy_iteration_loop(self):
                             config.call_after_subproblem_solve(fixed_nlp)
 
                         if self.algorithm_should_terminate(check_cycling=False):
-                            self.last_iter_cuts = True
                             break  # TODO: break two loops.
 
-        # if add_no_good_cuts is True, the bound obtained in the last iteration is no reliable.
-        # we correct it after the iteration.
-        # There is no need to fix the dual bound if no feasible solution has been found.
-        if (
-            (config.add_no_good_cuts or config.use_tabu_list)
-            and not self.should_terminate
-            and config.add_regularization is None
-            and self.best_solution_found is not None
-        ):
-            self.fix_dual_bound(self.last_iter_cuts)
         config.logger.info(
             ' ==============================================================================================='
         )

pyomo/contrib/mindtpy/extended_cutting_plane.py

@@ -59,7 +59,6 @@ def MindtPy_iteration_loop(self):
                 self.config.call_after_main_solve(main_mip)
 
             if self.algorithm_should_terminate():
-                self.last_iter_cuts = False
                 break
 
             add_ecp_cuts(self.mip, self.jacobians, self.config, self.timing)

pyomo/contrib/mindtpy/global_outer_approximation.py

@@ -10,7 +10,6 @@
 # ____________________________________________________________________________________
 
 
-from pyomo.contrib.gdpopt.util import get_main_elapsed_time
 from pyomo.core import ConstraintList
 from pyomo.opt import SolverFactory
 from pyomo.contrib.mindtpy.config_options import _get_MindtPy_GOA_config
@@ -69,24 +68,6 @@ def initialize_mip_problem(self):
         super().initialize_mip_problem()
         self.mip.MindtPy_utils.cuts.aff_cuts = ConstraintList(doc='Affine cuts')
 
-    def update_primal_bound(self, bound_value):
-        """Update the primal bound.
-
-        Call after solve fixed NLP subproblem.
-        Use the optimal primal bound of the relaxed problem to update the dual bound.
-
-        Parameters
-        ----------
-        bound_value : float
-            The input value used to update the primal bound.
-        """
-        super().update_primal_bound(bound_value)
-        self.primal_bound_progress_time.append(get_main_elapsed_time(self.timing))
-        if self.primal_bound_improved:
-            self.num_no_good_cuts_added.update(
-                {self.primal_bound: len(self.mip.MindtPy_utils.cuts.no_good_cuts)}
-            )
-
     def add_cuts(
         self,
         dual_values=None,
@@ -96,15 +77,3 @@ def add_cuts(
         nlp=None,
     ):
         add_affine_cuts(self.mip, self.config, self.timing)
-
-    def deactivate_no_good_cuts_when_fixing_bound(self, no_good_cuts):
-        try:
-            valid_no_good_cuts_num = self.num_no_good_cuts_added[self.primal_bound]
-            if self.config.add_no_good_cuts:
-                for i in range(valid_no_good_cuts_num + 1, len(no_good_cuts) + 1):
-                    no_good_cuts[i].deactivate()
-            if self.config.use_tabu_list:
-                self.integer_list = self.integer_list[:valid_no_good_cuts_num]
-        except KeyError as e:
-            self.config.logger.error(e, exc_info=True)
-            self.config.logger.error('Deactivating no-good cuts failed.')

pyomo/contrib/mindtpy/outer_approximation.py

@@ -128,14 +128,6 @@ def add_cuts(
                 self.mip, nlp, self.config, self.objective_sense, self.mip_iter, cb_opt
             )
 
-    def deactivate_no_good_cuts_when_fixing_bound(self, no_good_cuts):
-        # Only deactivate the last OA cuts may not be correct.
-        # Since integer solution may also be cut off by OA cuts due to calculation approximation.
-        if self.config.add_no_good_cuts:
-            no_good_cuts[len(no_good_cuts)].deactivate()
-        if self.config.use_tabu_list:
-            self.integer_list = self.integer_list[:-1]
-
     def objective_reformulation(self):
         # In the process_objective function, as long as the objective function is nonlinear, it will be reformulated and the variable/constraint/objective lists will be updated.
         # For OA/GOA/LP-NLP algorithm, if the objective function is linear, it will not be reformulated as epigraph constraint.

pyomo/contrib/mindtpy/single_tree.py

@@ -506,10 +506,6 @@ def handle_lazy_subproblem_optimal(self, fixed_nlp, mindtpy_solver, config, opt)
             mindtpy_solver.best_solution_found_time = get_main_elapsed_time(
                 mindtpy_solver.timing
             )
-            if config.add_no_good_cuts or config.use_tabu_list:
-                mindtpy_solver.stored_bound.update(
-                    {mindtpy_solver.primal_bound: mindtpy_solver.dual_bound}
-                )
         config.logger.info(
             mindtpy_solver.fixed_nlp_log_formatter.format(
                 '*' if mindtpy_solver.primal_bound_improved else ' ',