The results below are organized as follows:

  • each table displays the cumulative results for the given task under different time limits
  • table values are average normalized scores across all evaluated problems as outlined in Evaluation Criteria
  • clicking on a solver name links to the results of the solver on the individual problem instances

PR

Solver 20sec 1200sec 3600sec
lbp-pr 90.3 89.9 90.2
AbstractionSampling 77.4 89.8 91.4
uai14-pr 61.7 96.8 96.7
ibia-pr 53.6 96.6 97.1
baseline 0.0 0.0 0.0

Side-by-Side Comparisons

Solvers

  • lbp-pr: loopy belief propagation
  • AbstractionSampling: Importance Sampling scheme in AND/OR trees
  • uai14-pr: PR solver submission from UAI 2014
  • ibia-pr: PR solver submission
  • baseline: weak baseline

MAR

Solver 20sec 1200sec 3600sec
uai14-mar 58.4 80.5 81.9
ibia-mar 46.5 78.2 78.6
lbp-mar 58.6 58.4 58.4
baseline 0.0 0.0 0.0

Side-by-Side Comparisons

Solvers

  • uai14-mar: MAR solver submission from UAI 2014
  • ibia-mar: MAR solver submission
  • lbp-mar: loopy belief propogation
  • baseline: weak baseline

MPE

Solver 20sec 1200sec 3600sec
toulbar2-vns-mpe 84.2 89.0 89.2
toulbar2-vacint-mpe 79.1 89.6 91.4
daoopt-mpe 75.4 89.5 92.1
daoopt-lh-mpe 73.2 85.1 90.7
toulbar2-ipr-mpe 63.4 88.9 89.0
baseline 0.0 0.0 0.0

Side-by-Side Comparisons

Solvers

  • toulbar2-vns-mpe: A variable neighborhood search method of former UAI competition winner Toulbar2
  • toulbar2-vacint-mpe: A heuristic search method of former UAI competition winner Toulbar2
  • daoopt-mpe: uai 2012, 2014, and 2016 participant by Lars Otten
  • daoopt-lh-mpe: DAOOPT scheme with lookahead by William Lam
  • toulbar2-ipr-mpe: An incremental precision-based search method of former UAI competition winner Toulbar2
  • baseline: weak baseline

MMAP

Solver 20sec 1200sec 3600sec
toulbar2-vns-mmap 70.4 72.1 72.2
toulbar2-ipr-mmap 67.7 72.7 73.3
daoopt-mmap 68.5 72.1 71.7
toulbar2-vacint-mmap 68.9 71.3 71.4
daoopt-lh-mmap 64.5 66.4 67.6
braobb-mmap 34.0 41.4 44.1
uai14-mmap 28.9 33.8 35.4
lbp-mmap 28.3 29.4 29.6
baseline 0.0 0.0 0.0

Side-by-Side Comparisons

Solvers

  • toulbar2-vns-mmap: A variable neighborhood search method of former UAI competition winner Toulbar2
  • toulbar2-ipr-mmap: An incremental precision-based search method of former UAI competition winner Toulbar2
  • daoopt-mmap: uai 2012, 2014, and 2016 participant by Lars Otten
  • toulbar2-vacint-mmap: A heuristic search method of former UAI competition winner Toulbar2
  • daoopt-lh-mmap: DAOOPT scheme with lookahead by William Lam
  • braobb-mmap: Breadth rotating depth-first AND/OR branch and bound based scheme for MMAP
  • uai14-mmap: MMAP solver submission from UAI 2014
  • lbp-mmap: loopy belief propagation modified for solving MMAP
  • baseline: weak baseline

Updated: