Handling Challenging Models

Gurobi Summit EMEAI 2024
The Decision Intelligence Summit

Mario Ruthmair

Senior Optimization Engineer

Agenda

  1. What makes a model challenging?
  2. How to identify sources of difficulty?
  3. What can we do?

What makes a model challenging?

  • Returns infeasible
  • Takes long to solve because
    • Hard to find any or good feasible solutions
    • Hard to improve dual bound
    • Model is large
    • Numerical issues \(\rightarrow\) Mastering Numerical Challenges

Infeasible Model

Optimize a model with 177 rows, 548 columns and 1714 nonzeros
Model fingerprint: 0x0ff2e72c
Variable types: 0 continuous, 548 integer (0 binary)
Coefficient statistics:
  Matrix range     [1e+00, 1e+04]
  Objective range  [5e+00, 1e+04]
  Bounds range     [1e+00, 1e+00]
  RHS range        [1e+00, 1e+04]
Presolve removed 12 rows and 2 columns
Presolve time: 0.00s

Explored 0 nodes (0 simplex iterations) in 0.00 seconds (0.00 work units)
Thread count was 1 (of 8 available processors)

Solution count 0

Model is infeasible
Best objective -, best bound -, gap -

Why infeasible?

  • Compute an Irreducible Inconsistent Subsystem (IIS) with computeIIS()
  • IIS \(=\) “minimal” subset of constraints and variable bounds that conflict
  • But:
    • Computational time can be huge, especially for MIPs
    • IIS can be large and difficult to interpret

IIS

Computing Irreducible Inconsistent Subsystem (IIS)...

           Constraints          |            Bounds           |  Runtime
      Min       Max     Guess   |   Min       Max     Guess   |
--------------------------------------------------------------------------
        0       177         -         0      1096         -           0s
        2         2         2         2         2         2           0s

IIS computed: 2 constraints, 2 bounds
IIS runtime: 0.01 seconds (0.00 work units)

IIS in LP file format (default bounds \(x \ge 0\)):

Minimize
 
Subject To
 c79: 93 x1 - 189 x2 + 10 x3 >= 12
 c156: x1 - x2 + x3 <= 0
Bounds
 x2 free
Generals
 x1 x2 x3
End

Feasibility Relaxation

  • Find a solution with minimum violation with feasRelax()
  • Artificial slack variables show constraint/bound violation
  • Usually quite fast

Feasibility Relaxation

Optimize a model with 1273 rows, 1821 columns and 4083 nonzeros
Model fingerprint: 0xe02d128a
Variable types: 1273 continuous, 548 integer (0 binary)
Coefficient statistics:
  Matrix range     [1e+00, 1e+04]
  Objective range  [1e+00, 1e+00]
  Bounds range     [0e+00, 0e+00]
  RHS range        [1e+00, 1e+04]
Found heuristic solution: objective 1768.0000000
Presolve added 0 rows and 36 columns
Presolve removed 593 rows and 0 columns
Presolve time: 0.00s
Presolved: 680 rows, 1857 columns, 3096 nonzeros
Variable types: 124 continuous, 1733 integer (548 binary)
Found heuristic solution: objective 1306.0000000

Root relaxation: objective 1.269841e-01, 304 iterations, 0.00 seconds (0.00 work units)

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0    0.12698    0   59 1306.00000    0.12698   100%     -    0s
H    0     0                    1168.0000000    0.12698   100%     -    0s
H    0     0                    1142.0000000    0.12698   100%     -    0s
H    0     0                     866.2222222    0.12698   100%     -    0s
H    0     0                     586.9999998    0.12698   100%     -    0s
     0     0    1.00000    0   74  587.00000    1.00000   100%     -    0s
H    0     0                     400.9999998    1.00000   100%     -    0s
H    0     0                      80.0000000    1.00000  98.7%     -    0s
     0     0    1.00000    0   72   80.00000    1.00000  98.7%     -    0s
H    0     0                      79.0000000    1.00000  98.7%     -    0s
     0     0    1.00000    0   70   79.00000    1.00000  98.7%     -    0s
H    0     0                      10.0000000    1.00000  90.0%     -    0s
H    0     0                       5.0000000    1.00000  80.0%     -    0s
     0     0    1.00000    0   16    5.00000    1.00000  80.0%     -    0s
     0     0    1.00000    0   16    5.00000    1.00000  80.0%     -    0s
     0     0    1.00000    0   16    5.00000    1.00000  80.0%     -    0s
     0     2    1.00000    0   16    5.00000    1.00000  80.0%     -    0s
H   36    40                       4.0000000    1.00000  75.0%   3.1    0s
H   79   161                       2.0000000    1.00000  50.0%   2.4    0s
H  117   161                       1.0000000    1.00000  0.00%   2.2    0s

Cutting planes:
  Gomory: 20
  MIR: 10

Explored 172 nodes (1178 simplex iterations) in 0.31 seconds (0.15 work units)
Thread count was 8 (of 8 available processors)

Solution count 10: 1 2 4 ... 866.222

Optimal solution found (tolerance 1.00e-04)
Best objective 1.000000000000e+00, best bound 1.000000000000e+00, gap 0.0000%

IIS vs. Feasibility Relaxation

  • IIS:
Minimize
 
Subject To
 c79: 93 x1 - 189 x2 + 10 x3 >= 12
 c156: x1 - x2 + x3 <= 0
Bounds
 x2 free
Generals
 x1 x2 x3
End
  • Solution values from feasibility relaxation: x1 = 1, x2 = 0, x3 = 0
  • Value of constraint slack variable: ArtN_c156 = 1

How to identify sources of difficulty?

Solver log for model glass4

Optimize a model with 396 rows, 322 columns and 1815 nonzeros
Model fingerprint: 0x18b19fdf
Variable types: 20 continuous, 302 integer (0 binary)
Coefficient statistics:
  Matrix range     [1e+00, 8e+06]
  Objective range  [1e+00, 1e+06]
  Bounds range     [1e+00, 8e+02]
  RHS range        [1e+00, 8e+06]
Presolve removed 6 rows and 6 columns
Presolve time: 0.01s
Presolved: 390 rows, 316 columns, 1803 nonzeros
Variable types: 19 continuous, 297 integer (297 binary)
Found heuristic solution: objective 3.133356e+09

Root relaxation: objective 8.000024e+08, 72 iterations, 0.00 seconds (0.00 work units)

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0 8.0000e+08    0   72 3.1334e+09 8.0000e+08  74.5%     -    0s
H    0     0                    2.200019e+09 8.0000e+08  63.6%     -    0s
H    0     0                    2.000016e+09 8.0000e+08  60.0%     -    0s
     0     0 8.0000e+08    0   72 2.0000e+09 8.0000e+08  60.0%     -    0s
H    0     0                    1.930017e+09 8.0000e+08  58.5%     -    0s
     0     0 8.0000e+08    0   72 1.9300e+09 8.0000e+08  58.5%     -    0s
     0     0 8.0000e+08    0   83 1.9300e+09 8.0000e+08  58.5%     -    0s
     0     0 8.0000e+08    0   78 1.9300e+09 8.0000e+08  58.5%     -    0s
     0     2 8.0000e+08    0   78 1.9300e+09 8.0000e+08  58.5%     -    0s
H   95   111                    1.816682e+09 8.0000e+08  56.0%   7.6    0s
H  414   410                    1.750015e+09 8.0000e+08  54.3%   4.5    0s
H 1334   991                    1.733348e+09 8.0000e+08  53.8%   5.5    0s
H 1367   965                    1.700014e+09 8.0000e+08  52.9%   5.4    0s
*18952 13298              62    1.700014e+09 8.0000e+08  52.9%   4.3    2s
*21945 13292              48    1.700014e+09 8.0001e+08  52.9%   4.4    2s
*21946 13278              48    1.700014e+09 8.0001e+08  52.9%   4.4    2s
H23596 14591                    1.700013e+09 8.0001e+08  52.9%   4.4    2s
H27260 14584                    1.688903e+09 8.0001e+08  52.6%   4.3    2s
H30543 15189                    1.644458e+09 8.6667e+08  47.3%   4.2    3s
H30547 14430                    1.644458e+09 8.6667e+08  47.3%   4.2    3s
H30550 13710                    1.644458e+09 8.6667e+08  47.3%   4.2    4s
H30555 13027                    1.600013e+09 8.6667e+08  45.8%   4.2    4s
 31146 13383 1.0500e+09   62   42 1.6000e+09 9.0000e+08  43.8%   4.4    5s
 95341 33758 1.6000e+09   76   13 1.6000e+09 1.0000e+09  37.5%   6.0   10s
 187457 77051 1.5000e+09   60   37 1.6000e+09 1.0000e+09  37.5%   5.9   15s
*263218 109630              73    1.600013e+09 1.0000e+09  37.5%   5.7   19s
 266480 111130 infeasible   55      1.6000e+09 1.0000e+09  37.5%   5.7   20s
*270267 80393              71    1.550013e+09 1.0000e+09  35.5%   5.7   20s
H271848 77647                    1.533346e+09 1.0000e+09  34.8%   5.7   20s
*292350 79627              76    1.500013e+09 1.0000e+09  33.3%   5.7   22s
H293401 79715                    1.500013e+09 1.0000e+09  33.3%   5.7   22s
H293417 57180                    1.475014e+09 1.0000e+09  32.2%   5.7   22s
H299029 55344                    1.460014e+09 1.0000e+09  31.5%   5.7   23s
 312999 55932 1.4600e+09   52   35 1.4600e+09 1.0072e+09  31.0%   5.7   25s
*331032 52143              75    1.400013e+09 1.1000e+09  21.4%   5.9   27s
 350453 53694 infeasible   54      1.4000e+09 1.1000e+09  21.4%   6.0   30s
 384990 51565 1.4000e+09   50   28 1.4000e+09 1.2000e+09  14.3%   6.2   35s
 413646 58415 1.3000e+09   68   27 1.4000e+09 1.2000e+09  14.3%   6.3   40s
H413897 11468                    1.200013e+09 1.2000e+09  0.00%   6.3   40s

Cutting planes:
  Learned: 1
  Gomory: 10
  Cover: 3
  Implied bound: 11
  Projected implied bound: 1
  MIR: 12
  Flow cover: 43
  RLT: 2
  Relax-and-lift: 10

Explored 413963 nodes (2629055 simplex iterations) in 40.32 seconds (23.11 work units)
Thread count was 8 (of 8 available processors)

Solution count 10: 1.20001e+09 1.40001e+09 1.46001e+09 ... 1.60001e+09

Optimal solution found (tolerance 1.00e-04)
Best objective 1.200012600000e+09, best bound 1.200006650000e+09, gap 0.0005%

Solution progress

Bad solution, good bound

  • Bound is strong and converges quickly
  • Solutions improve slowly \(\rightarrow\)
    • Solver parameters focusing on heuristics
    • Custom heuristics

Good solution, bad bound

  • Solution improves quickly due to internal/user heuristics
  • Bound is weak and increases slowly \(\rightarrow\)
    • Solver parameters focusing on bound
    • More cuts
    • More preprocessing
    • Reformulate model

Tailing off

  • Solution and bound improve quickly but take long to close the gap
  • Think about termination criteria:
    • How precise is data?
    • How precise is model?

Gurobi LogTools

  • Python package to parse and analyze solver logs
  • pip install gurobi-logtools
import gurobi_logtools as glt
import plotly.graph_objects as go

results = glt.parse(["*.log"])
log = results.progress("nodelog")

fig = go.Figure()
fig.add_trace(go.Scatter(x=log["Time"], y=log["Incumbent"], name="Upper Bound"))
fig.add_trace(go.Scatter(x=log["Time"], y=log["BestBd"], name="Lower Bound"))
fig.update_xaxes(title_text="Time")
fig.update_yaxes(title_text="Objective value")
fig.show()

Gurobi LogTools

Example: glass4

Tune Solver Parameters

Example: glass4

Set parameter Heuristics to value 0.5
Set parameter Cuts to value 0
...
    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0 8.0000e+08    0   72 3.1334e+09 8.0000e+08  74.5%     -    0s
H    0     0                    2.200019e+09 8.0000e+08  63.6%     -    0s
H    0     0                    2.000016e+09 8.0000e+08  60.0%     -    0s
     0     0 8.0000e+08    0   72 2.0000e+09 8.0000e+08  60.0%     -    0s
     0     2 8.0000e+08    0   72 2.0000e+09 8.0000e+08  60.0%     -    0s
H  171   176                    2.000016e+09 8.0000e+08  60.0%   2.2    0s
H  173   176                    1.966684e+09 8.0000e+08  59.3%   2.2    0s
H  203   315                    1.966684e+09 8.0000e+08  59.3%   2.2    0s
H  321   445                    1.900017e+09 8.0000e+08  57.9%   2.3    0s
H  395   445                    1.855571e+09 8.0000e+08  56.9%   2.3    0s
H  598   578                    1.800015e+09 8.0000e+08  55.6%   2.4    0s
H  680   921                    1.794460e+09 8.0000e+08  55.4%   2.6    0s
* 1478  1114              47    1.750016e+09 8.0000e+08  54.3%   3.0    0s
* 1488  1067              47    1.750016e+09 8.0000e+08  54.3%   3.0    0s
* 2065  1434              62    1.733351e+09 8.0000e+08  53.8%   3.2    0s
H 3993  2572                    1.733350e+09 8.0000e+08  53.8%   3.0    0s
H 4189  2540                    1.700017e+09 8.0000e+08  52.9%   3.0    0s
H 4967  3141                    1.700014e+09 8.0000e+08  52.9%   3.0    1s
H 5379  3014                    1.510013e+09 8.0000e+08  47.0%   3.1    1s
H 5578  2957                    1.500013e+09 8.0000e+08  46.7%   3.1    1s
H 5816  3122                    1.500013e+09 8.0000e+08  46.7%   3.2    1s
H 5818  2900                    1.475013e+09 8.0000e+08  45.8%   3.2    1s
H20108 11407                    1.475012e+09 8.0000e+08  45.8%   4.2    4s
 20707 11751 1.0000e+09   44   40 1.4750e+09 8.0000e+08  45.8%   4.5    5s
 38874 21313 1.0500e+09   56   33 1.4750e+09 8.0000e+08  45.8%   4.8   10s
*43317 22917              61    1.466682e+09 8.0000e+08  45.5%   5.5   14s
 44236 23753 1.1000e+09   39   44 1.4667e+09 8.0000e+08  45.5%   5.7   15s
H44555 22869                    1.400014e+09 8.0000e+08  42.9%   5.7   15s
H46604 23057                    1.375015e+09 8.0000e+08  41.8%   6.0   15s
*53603 23808              37    1.366680e+09 8.0000e+08  41.5%   5.7   16s
H54046 20392                    1.200013e+09 8.0000e+08  33.3%   5.8   17s

Explored 62989 nodes (369883 simplex iterations) in 18.50 seconds (12.03 work units)
Thread count was 8 (of 8 available processors)

Solution count 10: 1.20001e+09 1.36668e+09 1.37501e+09 ... 1.51001e+09

Optimal solution found (tolerance 1.00e-04)
Best objective 1.200012600000e+09, best bound 1.200012600000e+09, gap 0.0000%

Geometry of MIP

  • 2 integer variables x and y
  • 5 linear constraints define gray area \(=\) continuous or LP relaxation
  • Feasible solutions are integer points inside the gray area
  • Solvers heavily rely on LP relaxation, e.g., in bounds, heuristics, branching

Geometry of MIP

  • Tighten the LP relaxation by adding constraints \(=\) Cut
  • Set of feasible solutions stays the same
  • Gray area gets smaller
  • Variable bounds are also cuts

Geometry of MIP

  • Dark gray area \(=\) ideal formulation / convex hull
  • Solver cuts and presolve try to approach convex hull
  • Why ideal? MIP reduces to LP

Geometry of MIP

  • Objective function vector \(c\)

Geometry of MIP

  • Potential initial solution
  • Optimal integer solution
  • Optimal LP relaxation solution

Geometry of MIP

  • Potential initial solution
  • Optimal integer solution
  • Optimal LP relaxation solution
  • Gaps are highly dependent on objective function
  • Even small root LP gaps can hide bad formulations

Why is this important?

  • Integer problems can have many formulations for the same set of solutions, which one should you choose?
  • Tighter formulations often lead to better performance, even if the model has more variables and/or constraints
  • Art of Modeling
    • Identify and understand strength
    • Find cuts
    • Reformulate

Example: Facility Location

Input data:

  • \(S\): Set of supermarkets
  • \(W\): Set of candidate warehouse locations
  • \(f_{w}\): Cost for building warehouse \(w\)
  • \(c_{ws}\): Cost of supplying supermarket \(s\) from warehouse \(w\)

Choose a subset of warehouses

  • to supply all supermarkets
  • with minimal total building and supply costs

Facility Location Model

  • Decision Variables:
    • \(y_{w} \in \{0, 1\}\): Build warehouse \(w\) (\(y_w = 1\)) or not (\(y_w = 0\))
    • \(0 \leq x_{ws} \leq 1\): Relative demand of supermarket \(s\) served by warehouse \(w\)
  • Objective: Minimize total warehouse building and supply costs \[\text{min} \quad \sum_{w \in W} f_{w} \cdot y_{w} + \sum_{w \in W} \sum_{s \in S} c_{ws} \cdot x_{ws}\]
  • Each supermarket must be fully supplied. \[\sum_{w \in W} x_{ws} = 1 \quad \forall s \in S\]
  • Warehouse \(w\) has to be opened if at least one supermarket is supplied. \[\sum_{s \in S} x_{ws} \leq |S| \cdot y_{w} \quad \forall w \in W\]

Solve it!

200 warehouse locations, 2000 supermarkets

Optimize a model with 2200 rows, 400200 columns and 800200 nonzeros
Model fingerprint: 0x889c7bde
Variable types: 400000 continuous, 200 integer (200 binary)
Coefficient statistics:
  Matrix range     [1e+00, 2e+03]
  Objective range  [1e+00, 4e+03]
  Bounds range     [1e+00, 1e+00]
  RHS range        [1e+00, 1e+00]
Found heuristic solution: objective 1657011.0000
Presolve time: 0.89s
Presolved: 2200 rows, 400200 columns, 800200 nonzeros
Variable types: 400000 continuous, 200 integer (200 binary)

Root relaxation: objective 7.496728e+04, 0 iterations, 0.57 seconds (0.48 work units)

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

H    0     0                    655075.00000 74967.2750  88.6%     -    2s
H    0     0                    653009.00000 74967.2750  88.5%     -    2s
H    0     0                    649303.00000 74967.2750  88.5%     -    2s
H    0     0                    315273.00000 74967.2750  76.2%     -    4s
     0     0 90077.2092    0  189 315273.000 90077.2092  71.4%     -    7s
H    0     0                    309486.00000 113880.544  63.2%     -   12s
     0     0 113880.637    0  199 309486.000 113880.637  63.2%     -   12s
     0     0 113880.637    0  199 309486.000 113880.637  63.2%     -   13s
H    0     0                    288245.00000 113880.637  60.5%     -   14s
     0     0 127672.131    0  198 288245.000 127672.131  55.7%     -   24s
     0     0 141821.354    0  200 288245.000 141821.354  50.8%     -   38s
     0     0 141821.894    0  200 288245.000 141821.894  50.8%     -   38s
H    0     0                    287028.02305 152967.493  46.7%     -   54s
H    0     0                    286382.72460 152967.493  46.6%     -   54s
H    0     0                    284520.36419 152967.493  46.2%     -   54s
H    0     0                    283286.52103 152967.493  46.0%     -   54s
H    0     0                    279335.29108 152967.493  45.2%     -   54s
H    0     0                    278389.32228 152967.493  45.1%     -   54s
H    0     0                    276709.11508 152967.493  44.7%     -   54s
H    0     0                    274845.59080 152967.493  44.3%     -   55s
H    0     0                    272863.07200 152967.493  43.9%     -   55s
H    0     0                    270067.36859 152967.493  43.4%     -   55s
H    0     0                    269215.59182 152967.493  43.2%     -   55s
H    0     0                    267058.50246 152967.493  42.7%     -   55s
H    0     0                    264503.60754 152967.493  42.2%     -   55s
H    0     0                    262566.00000 152967.493  41.7%     -   55s
H    0     0                    260417.00000 152967.493  41.3%     -   55s
H    0     0                    259862.00000 152967.493  41.1%     -   55s
     0     0 152967.493    0  198 259862.000 152967.493  41.1%     -   55s
H    0     0                    247309.00000 164695.350  33.4%     -   77s
     0     0 164695.350    0  200 247309.000 164695.350  33.4%     -   77s
     0     0 164697.220    0  200 247309.000 164697.220  33.4%     -   78s
     0     0 174779.528    0  196 247309.000 174779.528  29.3%     -   97s
H    0     0                    244829.39820 184077.164  24.8%     -  147s
H    0     0                    243771.44447 184077.164  24.5%     -  147s
H    0     0                    241969.66983 184077.164  23.9%     -  147s
H    0     0                    241173.06143 184077.164  23.7%     -  147s
H    0     0                    238695.59670 184077.164  22.9%     -  148s
H    0     0                    238519.36803 184077.164  22.8%     -  148s
H    0     0                    235385.31861 184077.164  21.8%     -  148s
H    0     0                    233305.45931 184077.164  21.1%     -  148s
H    0     0                    230649.58224 184077.164  20.2%     -  148s
H    0     0                    226799.00000 184077.164  18.8%     -  148s
     0     0 184077.164    0  196 226799.000 184077.164  18.8%     -  148s
     0     0 184114.003    0  196 226799.000 184114.003  18.8%     -  149s
     0     0 192114.343    0  194 226799.000 192114.343  15.3%     -  176s
H    0     0                    219877.00000 195184.674  11.2%     -  219s
     0     0 195184.674    0  196 219877.000 195184.674  11.2%     -  219s
     0     0 195926.373    0  196 219877.000 195926.373  10.9%     -  222s
     0     0 196165.395    0  196 219877.000 196165.395  10.8%     -  223s
     0     0 196225.826    0  196 219877.000 196225.826  10.8%     -  225s
     0     0 196225.826    0  196 219877.000 196225.826  10.8%     -  227s
     0     2 196225.826    0  196 219877.000 196225.826  10.8%     -  235s
    23    28 196769.371    7  189 219877.000 196769.371  10.5%  25.9  240s
    63    68 197474.225   17  181 219877.000 197225.191  10.3%  19.4  245s
    98   107 198364.408   25  173 219877.000 197225.191  10.3%  18.5  250s
   143   155 199324.970   36  162 219877.000 197225.191  10.3%  20.1  255s
   183   194 200229.907   47  151 219877.000 197225.191  10.3%  21.8  260s
   223   233 201567.765   56  142 219877.000 197225.191  10.3%  23.1  266s
   259   271 202726.045   64  134 219877.000 197225.191  10.3%  25.0  271s
   298   316 204081.770   72  127 219877.000 197225.191  10.3%  27.1  276s
   330   345 206740.858   79  120 219877.000 197225.191  10.3%  28.3  280s
   358   380 206797.231   84  115 219877.000 197225.191  10.3%  29.6  285s
   404   439 208523.098   94  104 219877.000 197225.191  10.3%  30.7  290s
   473   503 210330.037  107   91 219877.000 197225.191  10.3%  30.4  296s
   533   541 214736.349  123   72 219877.000 197225.191  10.3%  32.7  302s
   561   574 215748.307  129   65 219877.000 197225.191  10.3%  33.9  305s
   644   631     cutoff  168      219877.000 197231.207  10.3%  33.5  312s
   672   651 197448.104    8  188 219877.000 197448.104  10.2%  33.5  315s
   718   697 199094.241   22  176 219877.000 197475.343  10.2%  32.9  321s
H  733   688                    219526.00000 197475.343  10.0%  32.8  321s
   738   710 200028.714   28  170 219526.000 197475.343  10.0%  32.8  325s
H  760   714                    218836.00000 197475.343  9.76%  33.0  329s
H  771   713                    218823.00000 197475.343  9.76%  33.3  329s
   776   734 201927.105   38  159 218823.000 197475.343  9.76%  33.4  333s
   797   763 202687.326   42  155 218823.000 197475.343  9.76%  33.8  337s
   826   791 203413.089   47  150 218823.000 197475.343  9.76%  33.5  341s
   854   816 203917.562   51  147 218823.000 197475.343  9.76%  33.6  345s
   879   839 204990.022   60  138 218823.000 197475.343  9.76%  34.0  350s
   902   876 207260.562   66  132 218823.000 197475.343  9.76%  34.6  355s
   939   911 208445.648   79  119 218823.000 197475.343  9.76%  34.7  360s
   974   952 209719.263   88  110 218823.000 197475.343  9.76%  35.0  365s
  1015   999 211292.539   95  102 218823.000 197475.343  9.76%  35.4  372s
  1063  1000 213064.979   75  196 218823.000 197475.343  9.76%  36.1  399s
* 1064   950              12    215114.00000 215114.000  0.00%  36.0  407s

Explored 1064 nodes (62733 simplex iterations) in 407.36 seconds (209.74 work units)
Thread count was 8 (of 8 available processors)

Solution count 10: 215114 218823 218836 ... 238519

Optimal solution found (tolerance 1.00e-04)
Best objective 2.151140000000e+05, best bound 2.151140000000e+05, gap 0.0000%

Model Reformulation

  • Warehouse \(w\) has to be opened if at least one supermarket is supplied. \[\sum_{s \in S} x_{ws} \leq |S| \cdot y_{w} \quad \forall w \in W\]
  • In most solutions, the left-hand side will be much smaller than \(|S| \cdot y_w\)
  • Constraint Disaggregation: If any single supermarket is supplied by a warehouse, the warehouse has to be opened. \[x_{ws} \leq y_{w} \quad \forall w \in W, \forall s \in S\]

Solve it again!

Optimize a model with 402000 rows, 400200 columns and 1200000 nonzeros
Model fingerprint: 0xab88c055
Variable types: 400000 continuous, 200 integer (200 binary)
Coefficient statistics:
  Matrix range     [1e+00, 1e+00]
  Objective range  [1e+00, 4e+03]
  Bounds range     [1e+00, 1e+00]
  RHS range        [1e+00, 1e+00]
Found heuristic solution: objective 1657011.0000
Presolve time: 2.35s
Presolved: 402000 rows, 400200 columns, 1200000 nonzeros
Variable types: 400000 continuous, 200 integer (200 binary)
Deterministic concurrent LP optimizer: primal simplex, dual simplex, and barrier
Showing barrier log only...

Root barrier log...

Ordering time: 0.18s

Barrier statistics:
 Dense cols : 200
 AA' NZ     : 8.000e+05
 Factor NZ  : 1.645e+06 (roughly 340 MB of memory)
 Factor Ops : 9.667e+07 (less than 1 second per iteration)
 Threads    : 2

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0   2.21451667e+07  7.45226501e+05  2.00e+01 2.66e+02  7.12e+01     5s
   1   2.83097541e+06 -7.72382509e+05  2.48e+00 7.53e+01  7.12e+00     6s
   2   6.94893623e+05 -1.44052241e+05  6.66e-15 1.63e-01  7.08e-01     6s
   3   6.19409426e+05  3.44940542e+04  9.10e-15 1.61e-01  4.92e-01     7s
   4   5.64068664e+05  9.87373612e+04  1.38e-14 1.04e-01  3.91e-01     7s
   5   5.49580322e+05  1.16302608e+05  1.33e-14 6.42e-02  3.64e-01     8s
   6   5.25675690e+05  1.29417444e+05  1.60e-14 6.21e-02  3.33e-01     8s
   7   4.94701173e+05  1.38858058e+05  7.55e-15 6.10e-02  2.99e-01     9s
   8   4.45389485e+05  1.76495697e+05  7.55e-15 8.92e-03  2.26e-01    10s
   9   4.01222789e+05  1.97074403e+05  1.34e-14 1.80e-02  1.71e-01    10s
  10   3.72653325e+05  2.05428659e+05  1.43e-14 1.57e-02  1.40e-01    11s
  11   3.45938327e+05  2.09878961e+05  1.19e-14 1.31e-02  1.14e-01    12s
  12   3.15836621e+05  2.11984365e+05  4.77e-15 9.89e-03  8.72e-02    13s
  13   2.95957845e+05  2.13062637e+05  7.55e-15 7.99e-03  6.97e-02    13s
  14   2.79458149e+05  2.13924452e+05  2.55e-14 5.93e-03  5.51e-02    14s

Barrier performed 14 iterations in 13.93 seconds (4.14 work units)
Barrier solve interrupted - model solved by another algorithm

Concurrent spin time: 1.23s (can be avoided by choosing Method=3)

Solved with dual simplex

Root relaxation: objective 2.151140e+05, 13089 iterations, 11.13 seconds (2.90 work units)

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

*    0     0               0    215114.00000 215114.000  0.00%     -   14s

Explored 1 nodes (13089 simplex iterations) in 14.05 seconds (4.26 work units)
Thread count was 8 (of 8 available processors)

Solution count 2: 215114 1.65701e+06 

Optimal solution found (tolerance 1.00e-04)
Best objective 2.151140000000e+05, best bound 2.151140000000e+05, gap 0.0000%

Solver Heuristics

  • Solver includes more than 30 different general-purpose heuristics
  • Main concept:
    1. Fix or round values for (some) integer variables based on
      • Relaxation solutions
      • Available integer solutions
    2. Solve sub-MIP for the rest
  • Sometimes relaxations are hard to solve or do not give much information \(\rightarrow\) NoRel heuristic
    • Often works well for large binary models
    • Activate via parameter NoRelHeurTime or NoRelHeurWork

NoRel Heuristic

Example: glass4

Set parameter Heuristics to value 0.5
Set parameter NoRelHeurTime to value 5
Set parameter Cuts to value 0
...
Optimize a model with 396 rows, 322 columns and 1815 nonzeros
Model fingerprint: 0x18b19fdf
Variable types: 20 continuous, 302 integer (0 binary)
Coefficient statistics:
  Matrix range     [1e+00, 8e+06]
  Objective range  [1e+00, 1e+06]
  Bounds range     [1e+00, 8e+02]
  RHS range        [1e+00, 8e+06]
Presolve removed 6 rows and 6 columns
Presolve time: 0.00s
Presolved: 390 rows, 316 columns, 1803 nonzeros
Variable types: 19 continuous, 297 integer (297 binary)
Found heuristic solution: objective 3.133356e+09
Starting NoRel heuristic
Found heuristic solution: objective 3.033354e+09
Found heuristic solution: objective 2.633356e+09
Found heuristic solution: objective 2.633356e+09
Found heuristic solution: objective 2.633355e+09
Found heuristic solution: objective 2.533354e+09
Found heuristic solution: objective 2.333352e+09
Found heuristic solution: objective 2.300021e+09
Found heuristic solution: objective 2.200019e+09
Found heuristic solution: objective 2.125017e+09
Found heuristic solution: objective 2.100018e+09
Found heuristic solution: objective 2.000018e+09
Found heuristic solution: objective 2.000017e+09
Found heuristic solution: objective 2.000017e+09
Found heuristic solution: objective 1.983351e+09
Found heuristic solution: objective 1.950017e+09
Found heuristic solution: objective 1.933351e+09
Found heuristic solution: objective 1.933350e+09
Found heuristic solution: objective 1.933350e+09
Found heuristic solution: objective 1.850015e+09
Found heuristic solution: objective 1.800015e+09
Found heuristic solution: objective 1.700015e+09
Found heuristic solution: objective 1.700015e+09
Found heuristic solution: objective 1.700014e+09
Found heuristic solution: objective 1.700014e+09
Found heuristic solution: objective 1.690015e+09
Found heuristic solution: objective 1.650015e+09
Found heuristic solution: objective 1.650014e+09
Found heuristic solution: objective 1.616682e+09
Found heuristic solution: objective 1.600013e+09
Found heuristic solution: objective 1.600013e+09
Found heuristic solution: objective 1.600013e+09
Found heuristic solution: objective 1.500013e+09
Found heuristic solution: objective 1.500013e+09
Found heuristic solution: objective 1.475013e+09
Elapsed time for NoRel heuristic: 5s (best bound 8.00002e+08)

Root simplex log...

Iteration    Objective       Primal Inf.    Dual Inf.      Time
       0    8.0000240e+08   1.843200e+04   0.000000e+00      5s
      72    8.0000240e+08   0.000000e+00   0.000000e+00      5s

Root relaxation: objective 8.000024e+08, 72 iterations, 0.00 seconds (0.00 work units)

    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0 8.0000e+08    0   72 1.4750e+09 8.0000e+08  45.8%     -    5s
     0     0 8.0000e+08    0   72 1.4750e+09 8.0000e+08  45.8%     -    5s
     0     2 8.0000e+08    0   72 1.4750e+09 8.0000e+08  45.8%     -    5s
 12212  5296 1.1000e+09   47   46 1.4750e+09 8.0000e+08  45.8%   5.9   10s
*12457  5028              66    1.400013e+09 8.0000e+08  42.9%   6.1   10s
H12698  3108                    1.200013e+09 8.0000e+08  33.3%   6.2   10s

Explored 17122 nodes (115780 simplex iterations) in 11.99 seconds (6.38 work units)
Thread count was 8 (of 8 available processors)

Solution count 10: 1.20001e+09 1.40001e+09 1.47501e+09 ... 1.65001e+09

Optimal solution found (tolerance 1.00e-04)
Best objective 1.200012600000e+09, best bound 1.200003200000e+09, gap 0.0008%

But sometimes …

  • Solver cannot find any solution
  • Solution quality is not sufficient
  • Solution improvement too slow
  • Model is too large

What can we do?

  • Make model easier to be feasible
    • Relax constraints
    • Use penalties for hard constraints
  • Construct initial solution and hand it over as MIP start
    • Can be complete or partial
    • No need for high solution quality
  • Improve solutions in callbacks
    • Round LP solutions with problem-specific knowledge
    • Improve integer solutions with local-search techniques

Example: Rolling Horizon MIP Heuristic

Example: Local Search MIP Heuristic

Summary: How to handle challenging models?

  • Modeling phase
    • Clean up input data
    • Understand model strength (bounds, Big-M, cuts)
    • Reformulate model
  • Solution phase
    • Analyze solver logs \(\rightarrow\) Gurobi Experts
    • Run custom heuristics
    • Analyze model structure \(\rightarrow\) Gurobi Experts
    • Tune solver parameters \(\rightarrow\) Gurobi Experts

©️ Gurobi Optimization – gurobi.github.io/slides/