09 Nov 2019

Algorithms winning Artificial Intelligence Journey competition get B for USE exam

50 people (98 teams) from 14 countries took part in competition The goal was to design an AI solution that would pass a USE test in the Russian language Prize pool worth RUB3 mln Award ceremony held during AI Journey, an international conference organized by Sberbank and other leading companies

The award ceremony honoring the winners of the competition to create AI has been held during the Artificial Intelligence Journey conference and featured Alexander Vedyakhin, First Deputy Chairman of the Executive Board of Sberbank.

Backed by Sberbank and other leading companies, this largest AI forum in Eastern Europe is now being held at Expocentre, Moscow. As many as 350 people from 14 countries took part in the competition in 98 teams to generate over 2,300 solutions. They attempted to approach the creation of Artificial General Intelligence, or AGI, and algorithms capable of learning and passing exam tests of different complexity. The online challenge was available to solo contenders and teams, which were asked to design an AI solution that would pass a Unified State Exam in the Russian language.

Alexander Vedyakhin, First Deputy Chairman of the Executive Board, Sberbank, “For our competitions we’ve been picking challenges that amaze ourselves, not to mention others, for the fourth year running. The Russian language is a difficult academic subject and its test features open questions to understand the meaning of a text and requires writing a paper. For the competition, special exams had been designed. The papers were then graded by professional teachers and they couldn’t always tell if the exam had been taken by a robot or a human.”

The event was a Robot College Student Test that evaluated how well participants’ algorithms could answer questions in a specific field, having limited information access. Sberbank analyzed the international experience and formulated the challenge as follows, “create an algorithm that can answer questions from a high school curriculum using information from open sources.” The challenge included tests of knowledge, analytical skills, and the ability to write an essay on a piece of text. The papers that passed an automatic evaluation as plagiarism-free and making sense were graded under a standard methodology applied to high school graduates, which included viewpoint, reasoning, logic, language, punctuation, etc. The papers were also graded by real teachers and teaching method professionals from Moscow and Novosibirsk.

Such kind of a competition was held in Russia for the first time. The total prize pool amounted to RUB3 mln, of which the winner received RUB1 mln, the silver medalist got RUB500,000, and the one ranking third got RUB300,000. Cash prizes were awarded to the teams that took the first ten places, as well as to the winners in the special categories, “Best Essays” and “Best Tests”.

Here’s the Top 3:
1st place — Kyubik (Nikolai Zinov, Daniil Anastasyev)
2nd place — Bilbo Bagging (Alexander Anisimov, Herman Novikov, Vladimir Chernykh, Oleg Alenkin)
3rd place — Magic City (Arina Streltsova, Roman Pyankov, Sergei Arefyev).

Their AI solutions got B’s for the USE test.