The International Satisfiability Modulo Theories (SMT) Competition.
What the results of single-query SMT-COMP 2020 would have looked like with divisions split as shown below.
Summary of all competition results for the Single Query Track.
Results are given ranked by performance for each scoring scheme
(best solver is given as left-most solver).
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | 2019-Par4n, CVC4, Bitwuzla-fixedn, z3n, UltimateEliminator+MathSAT, Bitwuzla |
Parallel Performance | CVC4 | 2019-Par4n, CVC4, Bitwuzla-fixedn, z3n, UltimateEliminator+MathSAT, Bitwuzla |
SAT Performance | CVC4 | 2019-Par4n, Bitwuzla-fixedn, z3n, CVC4, UltimateEliminator+MathSAT, Bitwuzla |
UNSAT Performance | CVC4 | 2019-Par4n, CVC4, Bitwuzla-fixedn, z3n, UltimateEliminator+MathSAT, Bitwuzla |
24s Performance | CVC4 | 2019-Par4n, Bitwuzla-fixedn, z3n, CVC4, UltimateEliminator+MathSAT, Bitwuzla |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | z3n, CVC4, Yices2, Yices2-fixedn, 2019-Yices 2.6.2n, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n |
Parallel Performance | CVC4 | z3n, CVC4, Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n |
SAT Performance | CVC4 | z3n, CVC4, Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, 2018-Yicesn, 2018-CVC4n, Alt-Ergo |
UNSAT Performance | Yices2 | Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, z3n, CVC4, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n |
24s Performance | Yices2 | Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, z3n, CVC4, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | Yices2 | Yices2-fixedn, Yices2, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, z3n, CVC4, veriT, 2019-Yices 2.6.2n, Alt-Ergo, 2019-SMTInterpoln, 2018-Yicesn |
Parallel Performance | Yices2 | Yices2-fixedn, Yices2, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, z3n, CVC4, veriT, 2019-Yices 2.6.2n, Alt-Ergo, 2019-SMTInterpoln, 2018-Yicesn |
SAT Performance | Yices2 | Yices2, Yices2-fixedn, SMTInterpol, SMTInterpol-fixedn, CVC4, MathSAT5n, z3n, veriT, 2019-Yices 2.6.2n, 2019-SMTInterpoln, 2018-Yicesn, Alt-Ergo |
UNSAT Performance | Yices2 | Yices2-fixedn, Yices2, z3n, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, CVC4, 2019-Yices 2.6.2n, Alt-Ergo, veriT, 2019-SMTInterpoln, 2018-Yicesn |
24s Performance | Yices2 | Yices2, Yices2-fixedn, z3n, MathSAT5n, SMTInterpol-fixedn, SMTInterpol, CVC4, veriT, 2019-Yices 2.6.2n, 2019-SMTInterpoln, Alt-Ergo, 2018-Yicesn |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, MathSAT5n, 2019-CVC4n, z3n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog |
Parallel Performance | CVC4 | CVC4, MathSAT5n, 2019-CVC4n, z3n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog |
SAT Performance | CVC4 | CVC4, 2019-CVC4n, MathSAT5n, z3n, Yices2, Yices2-fixedn, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog, Alt-Ergo |
UNSAT Performance | CVC4 | MathSAT5n, CVC4, z3n, 2019-CVC4n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-MathSAT-defaultn, 2019-Par4n, veriT+raSAT+Redlog |
24s Performance | CVC4 | CVC4, MathSAT5n, 2019-CVC4n, z3n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | 2019-Par4n, CVC4, z3n, Yices2, Yices2-fixedn, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n |
Parallel Performance | CVC4 | 2019-Par4n, CVC4, z3n, Yices2, Yices2-fixedn, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n |
SAT Performance | CVC4 | 2019-Par4n, z3n, CVC4, Yices2, Yices2-fixedn, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n |
UNSAT Performance | SMTInterpol | 2019-Par4n, SMTInterpol, SMTInterpol-fixedn, CVC4, Yices2, Yices2-fixedn, MathSAT5n, z3n, veriT, 2019-Z3n |
24s Performance | Yices2 | 2019-Par4n, Yices2, Yices2-fixedn, CVC4, z3n, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | Yices2 | Yices2, Yices2-fixedn, CVC4, z3n, veriT, MathSAT5n, SMTInterpol-fixedn, SMTInterpol, OpenSMT, 2019-SPASS-SATTn, 2019-Par4n, 2019-Yices 2.6.2n |
Parallel Performance | Yices2 | Yices2, Yices2-fixedn, CVC4, z3n, veriT, MathSAT5n, SMTInterpol-fixedn, SMTInterpol, 2019-Par4n, OpenSMT, 2019-SPASS-SATTn, 2019-Yices 2.6.2n |
SAT Performance | Yices2 | Yices2-fixedn, Yices2, CVC4, z3n, veriT, SMTInterpol-fixedn, MathSAT5n, SMTInterpol, 2019-Par4n, 2019-SPASS-SATTn, OpenSMT, 2019-Yices 2.6.2n |
UNSAT Performance | CVC4 | CVC4, Yices2, Yices2-fixedn, z3n, veriT, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, 2019-Par4n, OpenSMT, 2019-SPASS-SATTn, 2019-Yices 2.6.2n |
24s Performance | Yices2 | Yices2-fixedn, Yices2, z3n, veriT, CVC4, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, 2019-SPASS-SATTn, OpenSMT, 2019-Par4n, 2019-Yices 2.6.2n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | Yices2 | Yices2, Yices2-fixedn, Bitwuzla, Bitwuzla-fixedn, MathSAT5n, CVC4, z3n, 2019-Boolectorn, 2019-Par4n, 2019-Yices 2.6.2n |
Parallel Performance | Yices2 | Yices2, Yices2-fixedn, Bitwuzla, Bitwuzla-fixedn, MathSAT5n, CVC4, z3n, 2019-Boolectorn, 2019-Par4n, 2019-Yices 2.6.2n |
SAT Performance | Bitwuzla | Bitwuzla-fixedn, Bitwuzla, Yices2, Yices2-fixedn, CVC4, MathSAT5n, z3n, 2019-Boolectorn, 2019-Par4n, 2019-Yices 2.6.2n |
UNSAT Performance | Yices2 | Yices2, Yices2-fixedn, Bitwuzla, Bitwuzla-fixedn, MathSAT5n, CVC4, z3n, 2019-Par4n, 2019-Boolectorn, 2019-Yices 2.6.2n |
24s Performance | Bitwuzla | Bitwuzla, Bitwuzla-fixedn, Yices2, Yices2-fixedn, CVC4, MathSAT5n, z3n, 2019-Par4n, 2019-Boolectorn, 2019-Yices 2.6.2n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, MathSAT5n, COLIBRI, Bitwuzla-fixedn, Bitwuzla, 2019-Par4n, z3n |
Parallel Performance | CVC4 | CVC4, MathSAT5n, COLIBRI, Bitwuzla-fixedn, Bitwuzla, 2019-Par4n, z3n |
SAT Performance | CVC4 | MathSAT5n, CVC4, COLIBRI, Bitwuzla, Bitwuzla-fixedn, z3n, 2019-Par4n |
UNSAT Performance | CVC4 | CVC4, MathSAT5n, Bitwuzla-fixedn, Bitwuzla, COLIBRI, 2019-Par4n, z3n |
24s Performance | CVC4 | CVC4, MathSAT5n, COLIBRI, Bitwuzla, Bitwuzla-fixedn, 2019-Par4n, z3n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | 2019-Par4n, CVC4, MathSAT5n, Yices2, Yices2-fixedn, z3n, AProVE, SMT-RAT, 2018-SMTRAT-Ratn |
Parallel Performance | CVC4 | 2019-Par4n, CVC4, MathSAT5n, Yices2, Yices2-fixedn, z3n, AProVE, SMT-RAT, 2018-SMTRAT-Ratn |
SAT Performance | CVC4 | 2019-Par4n, CVC4, MathSAT5n, z3n, Yices2, Yices2-fixedn, AProVE, SMT-RAT, 2018-SMTRAT-Ratn |
UNSAT Performance | Yices2 | 2019-Par4n, Yices2, Yices2-fixedn, MathSAT5n, CVC4, z3n, SMT-RAT, 2018-SMTRAT-Ratn, AProVE |
24s Performance | Yices2 | 2019-Par4n, Yices2, Yices2-fixedn, MathSAT5n, z3n, CVC4, AProVE, SMT-RAT, 2018-SMTRAT-Ratn |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | Yices2 | 2019-Par4n, Yices2-fixedn, Yices2, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC, CVC4, MathSAT5n |
Parallel Performance | Yices2 | 2019-Par4n, Yices2-fixedn, Yices2, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC, CVC4, MathSAT5n |
SAT Performance | Yices2 | 2019-Par4n, z3n, Yices2, Yices2-fixedn, SMT-RAT-MCSAT, SMT-RAT-CAlC, veriT+raSAT+Redlog, CVC4, MathSAT5n |
UNSAT Performance | Yices2 | 2019-Par4n, Yices2-fixedn, Yices2, CVC4, MathSAT5n, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC |
24s Performance | Yices2 | 2019-Par4n, Yices2, Yices2-fixedn, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC, MathSAT5n, CVC4 |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, Z3str4 |
Parallel Performance | CVC4 | CVC4, Z3str4 |
SAT Performance | CVC4 | CVC4, Z3str4 |
UNSAT Performance | CVC4 | CVC4, Z3str4 |
24s Performance | CVC4 | CVC4, Z3str4 |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | z3n, CVC4, 2019-Z3n, 2019-Par4n, UltimateEliminator+MathSAT, SMTInterpol-fixedn, SMTInterpol, veriT, veriT+viten, 2018-Z3n, Vampire |
Parallel Performance | CVC4 | z3n, CVC4, 2019-Z3n, 2019-Par4n, UltimateEliminator+MathSAT, SMTInterpol, SMTInterpol-fixedn, veriT, veriT+viten, 2018-Z3n, Vampire |
SAT Performance | CVC4 | z3n, 2019-Z3n, CVC4, 2019-Par4n, UltimateEliminator+MathSAT, SMTInterpol, SMTInterpol-fixedn, 2018-Z3n, veriT, veriT+viten, Vampire |
UNSAT Performance | CVC4 | z3n, CVC4, 2019-Z3n, 2019-Par4n, Vampire, UltimateEliminator+MathSAT, SMTInterpol-fixedn, SMTInterpol, veriT, veriT+viten, 2018-Z3n |
24s Performance | CVC4 | z3n, CVC4, 2019-Z3n, 2019-Par4n, UltimateEliminator+MathSAT, Vampire, SMTInterpol-fixedn, SMTInterpol, veriT, veriT+viten, 2018-Z3n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, 2019-CVC4n, Vampire, 2018-Vampiren, Alt-Ergo, veriT, veriT+viten, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol |
Parallel Performance | CVC4 | CVC4, 2019-CVC4n, Vampire, 2018-Vampiren, Alt-Ergo, veriT, veriT+viten, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol |
SAT Performance | CVC4 | CVC4, Vampire, 2019-CVC4n, 2018-Vampiren, z3n, SMTInterpol, SMTInterpol-fixedn, UltimateEliminator+MathSAT, veriT, Alt-Ergo, veriT+viten |
UNSAT Performance | CVC4 | CVC4, 2019-CVC4n, Vampire, Alt-Ergo, veriT, 2018-Vampiren, veriT+viten, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol |
24s Performance | Vampire | Vampire, 2019-CVC4n, CVC4, Alt-Ergo, 2018-Vampiren, veriT+viten, veriT, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, Alt-Ergo, z3n, veriT, veriT+viten, 2019-Par4n, SMTInterpol-fixedn, SMTInterpol, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT, Vampire |
Parallel Performance | CVC4 | CVC4, Alt-Ergo, z3n, veriT, veriT+viten, 2019-Par4n, SMTInterpol-fixedn, SMTInterpol, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT, Vampire |
SAT Performance | CVC4 | CVC4, 2018-CVC4n, z3n, SMTInterpol-fixedn, SMTInterpol, 2019-Par4n, UltimateEliminator+MathSAT, 2018-Z3n, 2019-CVC4n, veriT+viten, veriT, Alt-Ergo, Vampire |
UNSAT Performance | Vampire | Vampire, CVC4, Alt-Ergo, veriT, veriT+viten, z3n, 2019-Par4n, SMTInterpol-fixedn, SMTInterpol, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT |
24s Performance | CVC4 | CVC4, Alt-Ergo, z3n, veriT+viten, veriT, 2019-Par4n, SMTInterpol, SMTInterpol-fixedn, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT, Vampire |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, 2018-Z3n, 2019-Par4n, z3n, UltimateEliminator+MathSAT, 2018-CVC4n |
Parallel Performance | CVC4 | CVC4, 2018-Z3n, 2019-Par4n, z3n, UltimateEliminator+MathSAT, 2018-CVC4n |
SAT Performance | CVC4 | CVC4, z3n, 2019-Par4n, 2018-Z3n, 2018-CVC4n, UltimateEliminator+MathSAT |
UNSAT Performance | CVC4 | CVC4, 2019-Par4n, 2018-Z3n, z3n, UltimateEliminator+MathSAT, 2018-CVC4n |
24s Performance | CVC4 | CVC4, z3n, 2019-Par4n, 2018-Z3n, UltimateEliminator+MathSAT, 2018-CVC4n |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | CVC4, 2019-Par4n, Vampire, z3n, Alt-Ergo, 2018-Vampiren, UltimateEliminator+MathSAT, 2019-Vampiren |
Parallel Performance | CVC4 | CVC4, Vampire, 2019-Par4n, z3n, Alt-Ergo, UltimateEliminator+MathSAT, 2019-Vampiren, 2018-Vampiren |
SAT Performance | CVC4 | 2019-Par4n, CVC4, z3n, UltimateEliminator+MathSAT, 2019-Vampiren, Alt-Ergo, Vampire, 2018-Vampiren |
UNSAT Performance | CVC4 | CVC4, Vampire, 2018-Vampiren, 2019-Par4n, z3n, Alt-Ergo, UltimateEliminator+MathSAT, 2019-Vampiren |
24s Performance | CVC4 | CVC4, 2019-Par4n, z3n, Vampire, Alt-Ergo, 2018-Vampiren, UltimateEliminator+MathSAT, 2019-Vampiren |
Scoring Scheme | Winner | Ranking |
---|---|---|
Sequential Performance | CVC4 | 2019-Z3n, CVC4, z3n, UltimateEliminator+MathSAT |
Parallel Performance | CVC4 | 2019-Z3n, CVC4, z3n, UltimateEliminator+MathSAT |
SAT Performance | CVC4 | CVC4, 2019-Z3n, UltimateEliminator+MathSAT, z3n |
UNSAT Performance | CVC4 | 2019-Z3n, z3n, CVC4, UltimateEliminator+MathSAT |
24s Performance | CVC4 | 2019-Z3n, CVC4, z3n, UltimateEliminator+MathSAT |