In this chapter, we present kobe, a constraint benchmarking tools suite for various solvers including GeCode, Chuffed and AbSolute. The goals of this benchmarking tools suite are:

1. Reproducibility of the research results.
2. Report of the full results whereas in research papers it is often summarized.
3. Automate benchmarking of different solvers on different problems.
4. Analysis of the benchmarking results.

This work is on-going and replicability problems might still be present. In case of problems, please do not hesitate to contact us on the issues tracker or by email.

First, make sure you followed the instruction of the general getting started section. If so, you can install kobe through opam:

opam install kobe

You will have three new binaries in your path: kobegen, kobe and kobeview.

This gives you a list of the valid values for some entries of the benchmarking configuration file (explained below).

• Families of problems supported (directory in the JSON entry problem_sets.path): rcpsp, rcpsp-max.
• Solvers kind (solvers_kind): see function run_bench in kobe.ml.
• AbSolute domains (solvers_kind.AbSolute.domains): see function bench_absolute in kobe.ml.
• Box splitting strategies (solvers_kind.AbSolute.domains.strategies): see function make_box_strategy in kobe.ml.
• Octagon splitting strategies (solvers_kind.AbSolute.domains.strategies): see function make_octagon_strategy in kobe.ml.