Abstract:
Automatic question generation can help teachers to save the time necessary for constructing examination papers. Several approaches were proposed to automatically generate...Show MoreMetadata
Abstract:
Automatic question generation can help teachers to save the time necessary for constructing examination papers. Several approaches were proposed to automatically generate multiple-choice questions for vocalbuary assessment or grammar exercises. However, most of these studies focused on generating questions in English with a certain similarity strategy. This paper presents a mixed similarity strategy which generates Chinese multiple choice distractors with a statistical regression model including orthographic, phonological and semantic features, i.e., features that were shown in previous psycholinguistics studies to contribute to character recognition. In a first experiment, we evaluated the predictive power of the proposed features in measuring Chinese character similarity. One of the significant experimental results showed that the combination of the four proposed categories of features (structure, semantic radical, stroke and meaning) accounts for 62.5 percent of the variance in the human judgments of character similarity. In the second experiment, a user study was conducted to evaluate the quality of system-generated questions using a test item analysis method. Two hundred ninety-six Chinese primary school students (10-11-year-old) participated in this study. We have compared the mixed strategy with another three common distractor generation strategies, orthographic strategy, semantic strategy, and phonological strategy. One of important findings suggested that the mixed strategy significantly outperformed other three strategies in terms of the distractor usefulness and has a highest discrimination power among four strategies.
Published in: IEEE Transactions on Learning Technologies ( Volume: 11, Issue: 2, 01 April-June 2018)
Funding Agency:

Southwest University, Chongqing, Sichuan, CN
Ming Liu received the PhD degree in artificial intelligence in
education from the School of Electrical and Information Engineering, The University of Sydney, Australia, in 2012. He
is associate professor in the School of Computer and Information Science, Southwest University, China. His main
research interests include question generation, learning analytics, and intelligent tutoring system. He participa...Show More
Ming Liu received the PhD degree in artificial intelligence in
education from the School of Electrical and Information Engineering, The University of Sydney, Australia, in 2012. He
is associate professor in the School of Computer and Information Science, Southwest University, China. His main
research interests include question generation, learning analytics, and intelligent tutoring system. He participa...View more

University of Memphis School of Law, Memphis, TN, US
Vasile Rus received the bachelor's degree in computer science
from the Technical University of Cluj-Napoca, in June 1997 with a diploma Thesis entitledDistributed and
Collaborative Configuration Management, masterpieced while at LSR Laboratory, INPG, Grenoble, France. He
received the MSc degree in computer science and the doctor of philosophy in computer science degrees
fromSouthern Methodist Universit...Show More
Vasile Rus received the bachelor's degree in computer science
from the Technical University of Cluj-Napoca, in June 1997 with a diploma Thesis entitledDistributed and
Collaborative Configuration Management, masterpieced while at LSR Laboratory, INPG, Grenoble, France. He
received the MSc degree in computer science and the doctor of philosophy in computer science degrees
fromSouthern Methodist Universit...View more

Chongqing University, Chongqing, Sichuan, CN
Li Liu received the PhD degree in computer science from the
Universit Paris-sud XI, in 2008. He is an associate professor with Chongqing University. He is also serving as
a senior research fellow of the School of Computing, the National University of Singapore. He was as an associate
professor with Lanzhou University, China. His research interests are in pattern recognition, data analysis, and their
ap...Show More
Li Liu received the PhD degree in computer science from the
Universit Paris-sud XI, in 2008. He is an associate professor with Chongqing University. He is also serving as
a senior research fellow of the School of Computing, the National University of Singapore. He was as an associate
professor with Lanzhou University, China. His research interests are in pattern recognition, data analysis, and their
ap...View more

Southwest University, Chongqing, Sichuan, CN
Ming Liu received the PhD degree in artificial intelligence in
education from the School of Electrical and Information Engineering, The University of Sydney, Australia, in 2012. He
is associate professor in the School of Computer and Information Science, Southwest University, China. His main
research interests include question generation, learning analytics, and intelligent tutoring system. He participated
in national and international projects funded by ARC Linkage (Australia), Young and Well CRC, the Office of Teaching
and Learning, Google, and the Chinese National Fund. He is an author of over 20 publications in prestigious
conferences and journals, such as Intelligent Tutoring Systems, the IEEE Transactions on
Learning Technologies, the Journal of Educational Technology and Society.
Ming Liu received the PhD degree in artificial intelligence in
education from the School of Electrical and Information Engineering, The University of Sydney, Australia, in 2012. He
is associate professor in the School of Computer and Information Science, Southwest University, China. His main
research interests include question generation, learning analytics, and intelligent tutoring system. He participated
in national and international projects funded by ARC Linkage (Australia), Young and Well CRC, the Office of Teaching
and Learning, Google, and the Chinese National Fund. He is an author of over 20 publications in prestigious
conferences and journals, such as Intelligent Tutoring Systems, the IEEE Transactions on
Learning Technologies, the Journal of Educational Technology and Society.View more

University of Memphis School of Law, Memphis, TN, US
Vasile Rus received the bachelor's degree in computer science
from the Technical University of Cluj-Napoca, in June 1997 with a diploma Thesis entitledDistributed and
Collaborative Configuration Management, masterpieced while at LSR Laboratory, INPG, Grenoble, France. He
received the MSc degree in computer science and the doctor of philosophy in computer science degrees
fromSouthern Methodist University, Dallas, Texas, in May 1999 and May 2002, respectively. He is professor with
the University of Memphis. His research interests include intelligent systems, software engineering, text and data
mining, artificial intelligence, natural language processing, knowledge representation based on natural language, and
syntax-based semantics with applications to autotutoring, textual entailment, question answering, and other
applications where Semantics is a must. He has been involved in research and development projects in the areas of
computational linguistics and information retrieval for more than 15 years and in open-ended student answer assessment
and intelligent tutoring systems for more than 10 years. He has won the first two Question Answering competition
organized by the National Institute for Science and Technology (NIST); recently, his team won the English Semantic
Similarity challenge organized by the leading forum on semantic evaluations SemEval; has received 4 Best Paper
Awards; produced more than 100 peer-reviewed publications; and currently serves as an associate editor of the
International Journal on Tools with Artificial Intelligence and Program Committee member of the International
Conference on Artificial Intelligence in Education (AIED 2015).
Vasile Rus received the bachelor's degree in computer science
from the Technical University of Cluj-Napoca, in June 1997 with a diploma Thesis entitledDistributed and
Collaborative Configuration Management, masterpieced while at LSR Laboratory, INPG, Grenoble, France. He
received the MSc degree in computer science and the doctor of philosophy in computer science degrees
fromSouthern Methodist University, Dallas, Texas, in May 1999 and May 2002, respectively. He is professor with
the University of Memphis. His research interests include intelligent systems, software engineering, text and data
mining, artificial intelligence, natural language processing, knowledge representation based on natural language, and
syntax-based semantics with applications to autotutoring, textual entailment, question answering, and other
applications where Semantics is a must. He has been involved in research and development projects in the areas of
computational linguistics and information retrieval for more than 15 years and in open-ended student answer assessment
and intelligent tutoring systems for more than 10 years. He has won the first two Question Answering competition
organized by the National Institute for Science and Technology (NIST); recently, his team won the English Semantic
Similarity challenge organized by the leading forum on semantic evaluations SemEval; has received 4 Best Paper
Awards; produced more than 100 peer-reviewed publications; and currently serves as an associate editor of the
International Journal on Tools with Artificial Intelligence and Program Committee member of the International
Conference on Artificial Intelligence in Education (AIED 2015).View more

Chongqing University, Chongqing, Sichuan, CN
Li Liu received the PhD degree in computer science from the
Universit Paris-sud XI, in 2008. He is an associate professor with Chongqing University. He is also serving as
a senior research fellow of the School of Computing, the National University of Singapore. He was as an associate
professor with Lanzhou University, China. His research interests are in pattern recognition, data analysis, and their
applications on human behaviors. He aims to contribute in interdisciplinary research of computer science and human
related disciplines. He has published widely in conferences and journals with more than 50 peer-reviewed publications.
He has been the Principal Investigator of several funded projects from government and industry.
Li Liu received the PhD degree in computer science from the
Universit Paris-sud XI, in 2008. He is an associate professor with Chongqing University. He is also serving as
a senior research fellow of the School of Computing, the National University of Singapore. He was as an associate
professor with Lanzhou University, China. His research interests are in pattern recognition, data analysis, and their
applications on human behaviors. He aims to contribute in interdisciplinary research of computer science and human
related disciplines. He has published widely in conferences and journals with more than 50 peer-reviewed publications.
He has been the Principal Investigator of several funded projects from government and industry.View more