# Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots

Source: [Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots](<https://www.sota2.com/research/paper/68fa72621e209131cea561ac>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T04:59:10.530Z

Authors: Junling Wang, Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Mrinmaya Sachan

Published: 2024-03-05T20:12:05.000Z

## Abstract

Educational chatbots are a promising tool for assisting student learning. However, the development of effective chatbots in education has been challenging, as high-quality data is seldom available in this domain. In this paper, we propose a framework for generating synthetic teacher-student interactions grounded in a set of textbooks. Our approaches capture one aspect of learning interactions where curious students with partial knowledge interactively ask a teacher questions about the material in the textbook. We highlight various quality criteria that such dialogues should fulfill and compare several approaches relying on either prompting or fine-tuning large language models. We use synthetic dialogues to train educational chatbots and show benefits of further fine-tuning in different educational domains. However, human evaluation shows that our best data synthesis method still suffers from hallucinations and tends to reiterate information from previous conversations. Our findings offer insights for future efforts in synthesizing conversational data that strikes a balance between size and quality. We will open-source our data and code.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Dialogue Generation on CoQA (MCTest)](<https://www.sota2.com/research/sota/dialogue-generation-on-coqa-mctest.md>): Dialogue Generation; CoQA (MCTest); BLEU: 26.3.

- [Dialogue Generation on CoQA CNN](<https://www.sota2.com/research/sota/dialogue-generation-on-coqa-cnn.md>): Dialogue Generation; CoQA CNN; BLEU: 15.11.

- [Dialogue Generation on NCTE](<https://www.sota2.com/research/sota/dialogue-generation-on-ncte.md>): Dialogue Generation; NCTE; BLEU: 8.99.

- [Educational Dialogue Generation on Synthesized Educational Dialogues Human Evaluation](<https://www.sota2.com/research/sota/educational-dialogue-generation-on-synthesized-educational-dialogues-human-evaluation.md>): Educational Dialogue Generation; Synthesized Educational Dialogues Human Evaluation; Answer Relevance: 97.

- [Dialogue Quality Evaluation on Dialogue Inpainting](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-dialogue-inpainting.md>): Dialogue Quality Evaluation; Dialogue Inpainting; BF1 (qt, at): 0.52.

- [Dialogue Quality Evaluation on Persona Sing. Inst.](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-persona-sing-inst.md>): Dialogue Quality Evaluation; Persona Sing. Inst.; BF1 (qt, at): 0.58.

- [Dialogue Quality Evaluation on Persona Low Info](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-persona-low-info.md>): Dialogue Quality Evaluation; Persona Low Info; BF1 (qt, at): 61.

- [Dialogue Quality Evaluation on Persona Med. Info](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-persona-med-info.md>): Dialogue Quality Evaluation; Persona Med. Info; BF1 (qt, at): 61.

- [Dialogue Quality Evaluation on Persona High Info](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-persona-high-info.md>): Dialogue Quality Evaluation; Persona High Info; BF1 (qt, at): 0.62.

- [Educational Feedback Generation on SEFL (test)](<https://www.sota2.com/research/sota/educational-feedback-generation-on-sefl-test.md>): Educational Feedback Generation; SEFL (test); Win Rate (vs GPT-4o): 43.

- [Dialogue Quality Evaluation on SimSeek](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-simseek.md>): Dialogue Quality Evaluation; SimSeek; BF1 (qt, at): not recorded.

- [Dialogue Quality Evaluation on MathDial](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-mathdial.md>): Dialogue Quality Evaluation; MathDial; BF1 (qt, at): not recorded.

- [Dialogue Quality Evaluation on QuAC](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-quac.md>): Dialogue Quality Evaluation; QuAC; BF1 (qt, at): not recorded.

- [Dialogue Quality Evaluation on NCTE](<https://www.sota2.com/research/sota/dialogue-quality-evaluation-on-ncte.md>): Dialogue Quality Evaluation; NCTE; BF1 (qt, at): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.03307>)

[PDF](<https://arxiv.org/pdf/2403.03307.pdf>)

[Implementation](<https://github.com/eth-lre/book2dial>)
