SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

Participants
Results

SMT-COMP 2020 alternative results - Single Query Track (Summary)

What the results of single-query SMT-COMP 2020 would have looked like with divisions split as shown below.

Summary of all competition results for the Single Query Track.
Results are given ranked by performance for each scoring scheme (best solver is given as left-most solver).

BV

Scoring SchemeWinnerRanking
Sequential PerformanceCVC42019-Par4n, CVC4, Bitwuzla-fixedn, z3n, UltimateEliminator+MathSAT, Bitwuzla
Parallel PerformanceCVC42019-Par4n, CVC4, Bitwuzla-fixedn, z3n, UltimateEliminator+MathSAT, Bitwuzla
SAT PerformanceCVC42019-Par4n, Bitwuzla-fixedn, z3n, CVC4, UltimateEliminator+MathSAT, Bitwuzla
UNSAT PerformanceCVC42019-Par4n, CVC4, Bitwuzla-fixedn, z3n, UltimateEliminator+MathSAT, Bitwuzla
24s PerformanceCVC42019-Par4n, Bitwuzla-fixedn, z3n, CVC4, UltimateEliminator+MathSAT, Bitwuzla

QF_BV

Scoring SchemeWinnerRanking
Sequential PerformanceBitwuzlaBitwuzla-fixedn, Bitwuzla, Yices2-fixedn, 2019-Boolectorn, 2019-Poolectorn, STP + CMS, MinkeyRink-fixedn, CVC4, MathSAT5n, z3n, LazyBV2Int, MinkeyRink, STP + MergeSAT, Yices2
Parallel PerformanceBitwuzlaBitwuzla-fixedn, Bitwuzla, 2019-Poolectorn, Yices2-fixedn, 2019-Boolectorn, MinkeyRink-fixedn, STP + CMS, CVC4, MathSAT5n, z3n, LazyBV2Int, MinkeyRink, STP + MergeSAT, Yices2
SAT PerformanceBitwuzlaBitwuzla-fixedn, MinkeyRink-fixedn, Bitwuzla, 2019-Poolectorn, 2019-Boolectorn, Yices2-fixedn, STP + CMS, CVC4, STP + MergeSAT, z3n, MathSAT5n, LazyBV2Int, MinkeyRink, Yices2
UNSAT PerformanceBitwuzlaBitwuzla-fixedn, Bitwuzla, Yices2-fixedn, 2019-Poolectorn, 2019-Boolectorn, STP + CMS, Yices2, MinkeyRink-fixedn, CVC4, MathSAT5n, z3n, LazyBV2Int, MinkeyRink, STP + MergeSAT
24s PerformanceBitwuzla2019-Poolectorn, MinkeyRink-fixedn, Bitwuzla, Bitwuzla-fixedn, Yices2-fixedn, 2019-Boolectorn, STP + CMS, STP + MergeSAT, CVC4, MathSAT5n, z3n, LazyBV2Int, MinkeyRink, Yices2

QF_Equality (QF_UF, QF_AX, QF_DT)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4z3n, CVC4, Yices2, Yices2-fixedn, 2019-Yices 2.6.2n, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n
Parallel PerformanceCVC4z3n, CVC4, Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n
SAT PerformanceCVC4z3n, CVC4, Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, 2018-Yicesn, 2018-CVC4n, Alt-Ergo
UNSAT PerformanceYices2Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, z3n, CVC4, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n
24s PerformanceYices2Yices2, 2019-Yices 2.6.2n, Yices2-fixedn, z3n, CVC4, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, 2019-Par4n, veriT, OpenSMT, Alt-Ergo, 2018-Yicesn, 2018-CVC4n

QF_Equality+LinearArith (QF_ALIA, QF_AUFLIA, QF_UFLIA, QF_UFLRA, QF_UFIDL)

Scoring SchemeWinnerRanking
Sequential PerformanceYices2Yices2-fixedn, Yices2, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, z3n, CVC4, veriT, 2019-Yices 2.6.2n, Alt-Ergo, 2019-SMTInterpoln, 2018-Yicesn
Parallel PerformanceYices2Yices2-fixedn, Yices2, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, z3n, CVC4, veriT, 2019-Yices 2.6.2n, Alt-Ergo, 2019-SMTInterpoln, 2018-Yicesn
SAT PerformanceYices2Yices2, Yices2-fixedn, SMTInterpol, SMTInterpol-fixedn, CVC4, MathSAT5n, z3n, veriT, 2019-Yices 2.6.2n, 2019-SMTInterpoln, 2018-Yicesn, Alt-Ergo
UNSAT PerformanceYices2Yices2-fixedn, Yices2, z3n, SMTInterpol, SMTInterpol-fixedn, MathSAT5n, CVC4, 2019-Yices 2.6.2n, Alt-Ergo, veriT, 2019-SMTInterpoln, 2018-Yicesn
24s PerformanceYices2Yices2, Yices2-fixedn, z3n, MathSAT5n, SMTInterpol-fixedn, SMTInterpol, CVC4, veriT, 2019-Yices 2.6.2n, 2019-SMTInterpoln, Alt-Ergo, 2018-Yicesn

QF_Equality+NonLinearArith (QF_UFNRA, QF_UFNIA, QF_ANIA, QF_AUFNIA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, MathSAT5n, 2019-CVC4n, z3n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog
Parallel PerformanceCVC4CVC4, MathSAT5n, 2019-CVC4n, z3n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog
SAT PerformanceCVC4CVC4, 2019-CVC4n, MathSAT5n, z3n, Yices2, Yices2-fixedn, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog, Alt-Ergo
UNSAT PerformanceCVC4MathSAT5n, CVC4, z3n, 2019-CVC4n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-MathSAT-defaultn, 2019-Par4n, veriT+raSAT+Redlog
24s PerformanceCVC4CVC4, MathSAT5n, 2019-CVC4n, z3n, Yices2, Yices2-fixedn, Alt-Ergo, 2019-Par4n, 2019-MathSAT-defaultn, veriT+raSAT+Redlog

QF_LinearIntArith (QF_LIA, QF_LIRA, QF_IDL)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC42019-Par4n, CVC4, z3n, Yices2, Yices2-fixedn, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n
Parallel PerformanceCVC42019-Par4n, CVC4, z3n, Yices2, Yices2-fixedn, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n
SAT PerformanceCVC42019-Par4n, z3n, CVC4, Yices2, Yices2-fixedn, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n
UNSAT PerformanceSMTInterpol2019-Par4n, SMTInterpol, SMTInterpol-fixedn, CVC4, Yices2, Yices2-fixedn, MathSAT5n, z3n, veriT, 2019-Z3n
24s PerformanceYices22019-Par4n, Yices2, Yices2-fixedn, CVC4, z3n, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, veriT, 2019-Z3n

QF_LinearRealArith (QF_LRA, QF_RDL)

Scoring SchemeWinnerRanking
Sequential PerformanceYices2Yices2, Yices2-fixedn, CVC4, z3n, veriT, MathSAT5n, SMTInterpol-fixedn, SMTInterpol, OpenSMT, 2019-SPASS-SATTn, 2019-Par4n, 2019-Yices 2.6.2n
Parallel PerformanceYices2Yices2, Yices2-fixedn, CVC4, z3n, veriT, MathSAT5n, SMTInterpol-fixedn, SMTInterpol, 2019-Par4n, OpenSMT, 2019-SPASS-SATTn, 2019-Yices 2.6.2n
SAT PerformanceYices2Yices2-fixedn, Yices2, CVC4, z3n, veriT, SMTInterpol-fixedn, MathSAT5n, SMTInterpol, 2019-Par4n, 2019-SPASS-SATTn, OpenSMT, 2019-Yices 2.6.2n
UNSAT PerformanceCVC4CVC4, Yices2, Yices2-fixedn, z3n, veriT, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, 2019-Par4n, OpenSMT, 2019-SPASS-SATTn, 2019-Yices 2.6.2n
24s PerformanceYices2Yices2-fixedn, Yices2, z3n, veriT, CVC4, MathSAT5n, SMTInterpol, SMTInterpol-fixedn, 2019-SPASS-SATTn, OpenSMT, 2019-Par4n, 2019-Yices 2.6.2n

QF_Equality+BVArith (QF_ABV, QF_UFBV, QF_AUFBV)

Scoring SchemeWinnerRanking
Sequential PerformanceYices2Yices2, Yices2-fixedn, Bitwuzla, Bitwuzla-fixedn, MathSAT5n, CVC4, z3n, 2019-Boolectorn, 2019-Par4n, 2019-Yices 2.6.2n
Parallel PerformanceYices2Yices2, Yices2-fixedn, Bitwuzla, Bitwuzla-fixedn, MathSAT5n, CVC4, z3n, 2019-Boolectorn, 2019-Par4n, 2019-Yices 2.6.2n
SAT PerformanceBitwuzlaBitwuzla-fixedn, Bitwuzla, Yices2, Yices2-fixedn, CVC4, MathSAT5n, z3n, 2019-Boolectorn, 2019-Par4n, 2019-Yices 2.6.2n
UNSAT PerformanceYices2Yices2, Yices2-fixedn, Bitwuzla, Bitwuzla-fixedn, MathSAT5n, CVC4, z3n, 2019-Par4n, 2019-Boolectorn, 2019-Yices 2.6.2n
24s PerformanceBitwuzlaBitwuzla, Bitwuzla-fixedn, Yices2, Yices2-fixedn, CVC4, MathSAT5n, z3n, 2019-Par4n, 2019-Boolectorn, 2019-Yices 2.6.2n

QF_FPArith (QF_FP, QF_UFFP, QF_FPLRA, QF_BVFP, QF_ABVFP, QF_BVFPLRA, QF_ABVFPLRA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, MathSAT5n, COLIBRI, Bitwuzla-fixedn, Bitwuzla, 2019-Par4n, z3n
Parallel PerformanceCVC4CVC4, MathSAT5n, COLIBRI, Bitwuzla-fixedn, Bitwuzla, 2019-Par4n, z3n
SAT PerformanceCVC4MathSAT5n, CVC4, COLIBRI, Bitwuzla, Bitwuzla-fixedn, z3n, 2019-Par4n
UNSAT PerformanceCVC4CVC4, MathSAT5n, Bitwuzla-fixedn, Bitwuzla, COLIBRI, 2019-Par4n, z3n
24s PerformanceCVC4CVC4, MathSAT5n, COLIBRI, Bitwuzla, Bitwuzla-fixedn, 2019-Par4n, z3n

QF_NonLinearIntArith (QF_NIA, QF_NIRA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC42019-Par4n, CVC4, MathSAT5n, Yices2, Yices2-fixedn, z3n, AProVE, SMT-RAT, 2018-SMTRAT-Ratn
Parallel PerformanceCVC42019-Par4n, CVC4, MathSAT5n, Yices2, Yices2-fixedn, z3n, AProVE, SMT-RAT, 2018-SMTRAT-Ratn
SAT PerformanceCVC42019-Par4n, CVC4, MathSAT5n, z3n, Yices2, Yices2-fixedn, AProVE, SMT-RAT, 2018-SMTRAT-Ratn
UNSAT PerformanceYices22019-Par4n, Yices2, Yices2-fixedn, MathSAT5n, CVC4, z3n, SMT-RAT, 2018-SMTRAT-Ratn, AProVE
24s PerformanceYices22019-Par4n, Yices2, Yices2-fixedn, MathSAT5n, z3n, CVC4, AProVE, SMT-RAT, 2018-SMTRAT-Ratn

QF_NRA

Scoring SchemeWinnerRanking
Sequential PerformanceYices22019-Par4n, Yices2-fixedn, Yices2, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC, CVC4, MathSAT5n
Parallel PerformanceYices22019-Par4n, Yices2-fixedn, Yices2, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC, CVC4, MathSAT5n
SAT PerformanceYices22019-Par4n, z3n, Yices2, Yices2-fixedn, SMT-RAT-MCSAT, SMT-RAT-CAlC, veriT+raSAT+Redlog, CVC4, MathSAT5n
UNSAT PerformanceYices22019-Par4n, Yices2-fixedn, Yices2, CVC4, MathSAT5n, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC
24s PerformanceYices22019-Par4n, Yices2, Yices2-fixedn, z3n, SMT-RAT-MCSAT, veriT+raSAT+Redlog, SMT-RAT-CAlC, MathSAT5n, CVC4

QF_Strings (QF_S, QF_SLIA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, Z3str4
Parallel PerformanceCVC4CVC4, Z3str4
SAT PerformanceCVC4CVC4, Z3str4
UNSAT PerformanceCVC4CVC4, Z3str4
24s PerformanceCVC4CVC4, Z3str4

Arith (LRA, LIA, NIA, NRA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4z3n, CVC4, 2019-Z3n, 2019-Par4n, UltimateEliminator+MathSAT, SMTInterpol-fixedn, SMTInterpol, veriT, veriT+viten, 2018-Z3n, Vampire
Parallel PerformanceCVC4z3n, CVC4, 2019-Z3n, 2019-Par4n, UltimateEliminator+MathSAT, SMTInterpol, SMTInterpol-fixedn, veriT, veriT+viten, 2018-Z3n, Vampire
SAT PerformanceCVC4z3n, 2019-Z3n, CVC4, 2019-Par4n, UltimateEliminator+MathSAT, SMTInterpol, SMTInterpol-fixedn, 2018-Z3n, veriT, veriT+viten, Vampire
UNSAT PerformanceCVC4z3n, CVC4, 2019-Z3n, 2019-Par4n, Vampire, UltimateEliminator+MathSAT, SMTInterpol-fixedn, SMTInterpol, veriT, veriT+viten, 2018-Z3n
24s PerformanceCVC4z3n, CVC4, 2019-Z3n, 2019-Par4n, UltimateEliminator+MathSAT, Vampire, SMTInterpol-fixedn, SMTInterpol, veriT, veriT+viten, 2018-Z3n

Equality (UF, UFDT)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, 2019-CVC4n, Vampire, 2018-Vampiren, Alt-Ergo, veriT, veriT+viten, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol
Parallel PerformanceCVC4CVC4, 2019-CVC4n, Vampire, 2018-Vampiren, Alt-Ergo, veriT, veriT+viten, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol
SAT PerformanceCVC4CVC4, Vampire, 2019-CVC4n, 2018-Vampiren, z3n, SMTInterpol, SMTInterpol-fixedn, UltimateEliminator+MathSAT, veriT, Alt-Ergo, veriT+viten
UNSAT PerformanceCVC4CVC4, 2019-CVC4n, Vampire, Alt-Ergo, veriT, 2018-Vampiren, veriT+viten, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol
24s PerformanceVampireVampire, 2019-CVC4n, CVC4, Alt-Ergo, 2018-Vampiren, veriT+viten, veriT, z3n, SMTInterpol-fixedn, UltimateEliminator+MathSAT, SMTInterpol

Equality+LinearArith (ALIA, AUFLIA, UFLIA, UFIDL, AUFLIRA, UFLRA, UFDTLIA, UFDTLIRA, AUFDTLIA, AUFDTLIRA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, Alt-Ergo, z3n, veriT, veriT+viten, 2019-Par4n, SMTInterpol-fixedn, SMTInterpol, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT, Vampire
Parallel PerformanceCVC4CVC4, Alt-Ergo, z3n, veriT, veriT+viten, 2019-Par4n, SMTInterpol-fixedn, SMTInterpol, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT, Vampire
SAT PerformanceCVC4CVC4, 2018-CVC4n, z3n, SMTInterpol-fixedn, SMTInterpol, 2019-Par4n, UltimateEliminator+MathSAT, 2018-Z3n, 2019-CVC4n, veriT+viten, veriT, Alt-Ergo, Vampire
UNSAT PerformanceVampireVampire, CVC4, Alt-Ergo, veriT, veriT+viten, z3n, 2019-Par4n, SMTInterpol-fixedn, SMTInterpol, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT
24s PerformanceCVC4CVC4, Alt-Ergo, z3n, veriT+viten, veriT, 2019-Par4n, SMTInterpol, SMTInterpol-fixedn, 2018-CVC4n, 2019-CVC4n, 2018-Z3n, UltimateEliminator+MathSAT, Vampire

Equality+MachineArith (AUFFPDTLIRA, UFFPDTLIRA, UFFPDTNIRA, ABVFP, ABVFPLRA, UFBV, AUFBVDTLIA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, 2018-Z3n, 2019-Par4n, z3n, UltimateEliminator+MathSAT, 2018-CVC4n
Parallel PerformanceCVC4CVC4, 2018-Z3n, 2019-Par4n, z3n, UltimateEliminator+MathSAT, 2018-CVC4n
SAT PerformanceCVC4CVC4, z3n, 2019-Par4n, 2018-Z3n, 2018-CVC4n, UltimateEliminator+MathSAT
UNSAT PerformanceCVC4CVC4, 2019-Par4n, 2018-Z3n, z3n, UltimateEliminator+MathSAT, 2018-CVC4n
24s PerformanceCVC4CVC4, z3n, 2019-Par4n, 2018-Z3n, UltimateEliminator+MathSAT, 2018-CVC4n

Equality+NonLinearArith (AUFDTNIRA, UFDTNIA, UFDTNIRA, AUFNIA, AUFNIRA, UFNIA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC4CVC4, 2019-Par4n, Vampire, z3n, Alt-Ergo, 2018-Vampiren, UltimateEliminator+MathSAT, 2019-Vampiren
Parallel PerformanceCVC4CVC4, Vampire, 2019-Par4n, z3n, Alt-Ergo, UltimateEliminator+MathSAT, 2019-Vampiren, 2018-Vampiren
SAT PerformanceCVC42019-Par4n, CVC4, z3n, UltimateEliminator+MathSAT, 2019-Vampiren, Alt-Ergo, Vampire, 2018-Vampiren
UNSAT PerformanceCVC4CVC4, Vampire, 2018-Vampiren, 2019-Par4n, z3n, Alt-Ergo, UltimateEliminator+MathSAT, 2019-Vampiren
24s PerformanceCVC4CVC4, 2019-Par4n, z3n, Vampire, Alt-Ergo, 2018-Vampiren, UltimateEliminator+MathSAT, 2019-Vampiren

FPArith (BVFP, FP, BVFPLRA, FPLRA)

Scoring SchemeWinnerRanking
Sequential PerformanceCVC42019-Z3n, CVC4, z3n, UltimateEliminator+MathSAT
Parallel PerformanceCVC42019-Z3n, CVC4, z3n, UltimateEliminator+MathSAT
SAT PerformanceCVC4CVC4, 2019-Z3n, UltimateEliminator+MathSAT, z3n
UNSAT PerformanceCVC42019-Z3n, z3n, CVC4, UltimateEliminator+MathSAT
24s PerformanceCVC42019-Z3n, CVC4, z3n, UltimateEliminator+MathSAT