The results below are organized as follows:

  • each table displays the cumulative results for the given task under different time limits
  • table values are average normalized scores across all evaluated problems as outlined in Evaluation Criteria
  • clicking on a solver name links to the results of the solver on the individual problem instances

PR

Solver 20sec 1200sec 3600sec
ibia-pr 99.9 100.0 100.0
uai14-pr 97.2 100.0 100.0
Abstraction-Sampling 99.7 97.0 97.7
lbp 8.6 88.0 88.0
wmbsearch-aobfs-pr 29.3 57.1 57.1

Side-by-Side Comparisons

Solvers

  • ibia-pr: PR solver submission
  • uai14-pr: PR solver submission from UAI 2014
  • Abstraction-Sampling: Importance Sampling scheme in AND/OR trees
  • lbp: loopy belief propogation
  • wmbsearch-aobfs-pr: AND/OR best-first search scheme using the weighted mini-bucket guiding heuristic

MAR

Solver 20sec 1200sec 3600sec
ibia-mar 71.1 76.4 76.4
uai14-mar 61.6 76.5 76.5
lbp 3.3 54.2 54.2

Side-by-Side Comparisons

Solvers

  • ibia-mar: MAR solver submission
  • uai14-mar: MAR solver submission from UAI 2014
  • lbp: loopy belief propogation

MPE

Solver 20sec 1200sec 3600sec
toulbar2-vns-mpe 98.5 99.2 99.3
toulbar2-ipr-mpe 97.2 99.4 99.5
toulbar2-vacint-mpe 98.3 98.8 98.9
daoopt 97.3 98.5 98.9
daoopt-lh-mpe 94.1 96.3 96.2
daoopt-test 54.5 65.6 67.3
uai14-mpe 43.7 49.8 51.8

Side-by-Side Comparisons

Solvers

  • toulbar2-vns-mpe: A variable neighborhood search method of former UAI competition winner Toulbar2
  • toulbar2-ipr-mpe: An incremental precision-based search method of former UAI competition winner Toulbar2
  • toulbar2-vacint-mpe: A heuristic search method of former UAI competition winner Toulbar2
  • daoopt: uai 2012, 2014, and 2016 participant by Lars Otten
  • daoopt-lh-mpe: daoopt with look ahead heuristic
  • daoopt-test: daoopt with less competitive settings
  • uai14-mpe: MPE solver submission from UAI 2014

MMAP

Solver 20sec 1200sec 3600sec
toulbar2-vacint-mmap 98.5 98.4 98.4
toulbar2-vns-mmap 98.0 98.5 98.6
toulbar2-ipr-mmap 97.5 98.7 98.7
daoopt 96.9 97.9 98.0
daoopt-lh-mmap 95.5 96.3 97.1
merlin-mmap 35.5 45.0 52.0
uai14-mmap 40.9 41.4 41.5
lbp-mmap 0.6 50.6 54.5

Side-by-Side Comparisons

Solvers

  • toulbar2-vacint-mmap: A heuristic search method of former UAI competition winner Toulbar2
  • toulbar2-vns-mmap: A variable neighborhood search method of former UAI competition winner Toulbar2
  • toulbar2-ipr-mmap: An incremental precision-based search method of former UAI competition winner Toulbar2
  • daoopt: uai 2012, 2014, and 2016 participant by Lars Otten
  • daoopt-lh-mmap: daoopt with look ahead heuristic
  • merlin-mmap: Breadth rotating depth-first AND/OR branch and bound based scheme for MMAP
  • uai14-mmap: MMAP solver submission from UAI 2014
  • lbp-mmap: loopy belief propagation modified for solving MMAP

Updated: