The Riiid AIEd Challenge was started last year on the Kaggle platform – an ‘online community of data scientists and machine learning practitioners’ – to find algorithms for ‘Knowledge Tracing’, the modelling of student knowledge over time, and predict future student performance.
“We now have a benchmark that we can try to use in figuring out what we do next year”
Data used by more than 3,000 teams came from Riiid’s ‘EdNet’, a collection of more than 130 million interactions between more than 780,000 students, a resource that the company describes as “the world’s largest hierarchical education dataset”.
Supported by corporate partners including Ad Astra, Global ATP, Dxtera Institute, Edmentum, Kaplan, Reboot Representation, SocialTech.ai and The Learning Agency, and backed by $100,000 of prize money, the contest attracted 3,395 teams from 90 countries.
Academics Paul Kim, chief technology officer and assistant dean of the Graduate School of Education at Stanford University, and educational data mining expert Neil Heffernan also supported the competition.
The winners are outlined below (full name, affiliation, country, team or solo):
- 1st place: Kaggle handle KEETAR (Seungkee Jeon, Samsung Electronics, South Korea, Solo).
- 2nd place: Kaggle handle MAMAS (Takashi Oya, Waseda Research Institute for Science and Engineering, Japan, Solo).
- 3rd place: Kaggle handle NOISY STUDENTS (Javier Martin & Andres Torrubia, Regional Department of Education of Valencia & Institute of Artificial Intelligence, Spain, 2-Person Team).
- 4th place: Kaggle handle EMMY (Duc Kinh Le Tran, Axance Technology, France, Solo).
- 5th place: Kaggle handle GFHS + GCY95GCY + MERCURIALZHANG + YANG (Chengwei Zhang + Yangzhou Jiang + Wei Zhang + Chengyu Gu, Shanghai Jiao Tong University, China, 4-Person Team).
The top three prize-winning teams presented their models virtually at a conference held by the Association for the Advancement of Artifical Intelligence in February, at a session hosted and organised by researchers at Riiid entitled ‘Imagining Post-COVID Education with AI’.
Riiid said that the competition was very successful in raising awareness of education outcomes among top AI researchers.
A survey of entrants revealed that “while more than 50% of respondents stated they had a very little knowledge of possible applications of AIEd before the challenge, 81% believed that the competition made each participant interested in the field of AIEd”.
Riiid made its impact on the education sector in 2017 when it launched Santa, its first AI tutor solution based on deep learning algorithms. Santa is used by students preparing for the English language competency exam TOIEC.
At a briefing session before the winners’ presentation, Jim Larimore, chief officer for Equity in Learning at Riiid and challenge chair, explained some more of the thinking behind the competition and the long-term aim.
“What we wanted to do was to tap the global talent pool, much of which is locked up in big companies like Google and IBM, and we wanted to draw them into the field of education… We wanted to accelerate the pace of attention to education.
“So, we now have a benchmark that we can try to use in figuring out what we do next year, because this was our inaugural challenge and we’ll have additional challenges as we go forward.”
