LTL Instruction Following on LetterWorld Finite-horizon
100Success Rate (SR)DEEPLTL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DEEPLTLTask Formula=(F((a ∨ c ∨ j) ∧ F b)) ∧ (F(c ∧ F d)) ∧ F k, |Q| (number of LDBA states)=5, |δ| (number of transitions)=102026.02 | 100 | — | |
| SEMLTLTask Formula=(F((a ∨ c ∨ j) ∧ F b)) ∧ (F(c ∧ F d)) ∧ F k, |Q| (number of LDBA states)=5, |δ| (number of transitions)=102026.02 | 100 | 3.04 | |
| DEEPLTLTask Formula=F(a ∧ (¬b U c)) ∧ F d, |Q| (number of LDBA states)=9, |δ| (number of transitions)=292026.02 | 100 | — | |
| SEMLTLTask Formula=F(a ∧ (¬b U c)) ∧ F d, |Q| (number of LDBA states)=9, |δ| (number of transitions)=292026.02 | 100 | 4.2 | |
| SEMLTLTask Formula=(F d) ∧ (¬f U (d ∧ F b)), |Q| (number of LDBA states)=14, |δ| (number of transitions)=392026.02 | 96 | 5.23 | |
| DEEPLTLTask Formula=¬a U (b ∧ (¬c U (d ∧ (¬e U f)))), |Q| (number of LDBA states)=6, |δ| (number of transitions)=152026.02 | 95 | — | |
| SEMLTLTask Formula=¬a U (b ∧ (¬c U (d ∧ (¬e U f)))), |Q| (number of LDBA states)=6, |δ| (number of transitions)=152026.02 | 95 | 4.01 | |
| DEEPLTLTask Formula=(F d) ∧ (¬f U (d ∧ F b)), |Q| (number of LDBA states)=14, |δ| (number of transitions)=392026.02 | 95 | — | |
| LTL2ACTIONTask Formula=(F d) ∧ (¬f U (d ∧ F b)), |Q| (number of LDBA states)=14, |δ| (number of transitions)=392026.02 | 84 | — | |
| LTL2ACTIONTask Formula=¬a U (b ∧ (¬c U (d ∧ (¬e U f)))), |Q| (number of LDBA states)=6, |δ| (number of transitions)=152026.02 | 73 | — | |
| LTL2ACTIONTask Formula=F(a ∧ (¬b U c)) ∧ F d, |Q| (number of LDBA states)=9, |δ| (number of transitions)=292026.02 | 62 | — | |
| LTL2ACTIONTask Formula=(F((a ∨ c ∨ j) ∧ F b)) ∧ (F(c ∧ F d)) ∧ F k, |Q| (number of LDBA states)=5, |δ| (number of transitions)=102026.02 | 48 | — |