Initial situation

From textbook to measurable learning experience

Complai: As a leading provider of online courses and certifications for continuing medical education, our client had a lot of experience and domain knowledge in dealing with doctors and other medical staff.

Both residents who were completing their mandatory time in the hospital and seasoned professionals who needed to recertify in order to continue practicing had the same problem: not a lack of information, but a lack of time to consume it and a lack of certainty that all the key points had been understood and learned.

In their stressful everyday working lives, hardly anyone found time to read textbooks or run through sample exams. Between two surgeries, there was just enough time for a coffee and maybe a short technical article on the cell phone.

In 2020, we at NEOMO had started researching how Large Language Models could improve the way we consume information and acquire knowledge.

Together with our client, we were able to put these technologies to the test in practice.

Image

Objective

Complai: Continuing education in the medical field must entail consuming teaching content in 5-minute chunks and at the same time always keeping an eye on how well you have mastered the material.

The logical conclusion for our client was to make the extraordinarily extensive subject knowledge of the various specializations accessible in the form of "test-enhanced learning", i.e. to create individual multiple-choice questions from long texts including images and diagrams, which at the same time served to test knowledge but also - through detailed explanations of the correct answer with references to the original text and further articles - to impart knowledge.

Intelligent algorithms had to help with both the creation and the adaptation of these questionnaires to the respective learner and thus break down the learning experience into small, self- contained units.

Image

Complai: Automatic selection of the "learning path" and flexible measurement of progress

Complai: From its previous work, our client had already created a repertoire of 3000 questions with answers and explanations that completely covered a specific medical specialty. This meant that anyone who could answer 80% of these questions correctly was very likely "ready for the exam".

Of course, advanced experts who just need a "refresher" do not want to answer countless "beginner questions".

NEOMO therefore first tackled the task of finding an optimal learning path for each user and how to score the answers as "fairly" as possible.

We designed an engine that automatically assessed the difficulty of the questions based on past answers. In combination with another clever algorithm, each new learner could be assigned a suitable proficiency level within a few steps, so that they automatically skipped almost all questions "below their level". The client's requirement here was to always keep the learner at the edge of his comfort zone so that he made rapid progress but did not grow frustrated.

As the scoring requirements for trainee doctors on the one hand and recertifiers on the other were very different, we designed several different scoring modules - including one based on state-space algorithms - and a "Measurement Decision Theory" component.

Result

Complai: The final result was a powerful authoring platform that automated the tedious grunt work of importing and splitting content into different formats. Instructional designers can now focus on selecting the best sources and optimizing the learning experience.

This platform has opened up several new audiences for our client, as it has not only transformed the specialized training of medical professionals, but also many forms of corporate training, for example: every HR document and every recorded training session - e.g. on "Sexual Harassment" or "Privacy" - can be transformed into interactive and measurable courses.

As of today, more than 10,000 users work on the client's platform and benefit from an interactive and engaging learning experience.

Image
Avatar

Jørgen Kadal, CEO, ComlAi

"As a compliance-focused company we are obviously under intense scrutiny in regards to security, data protection and reliability. So we could not just put arbitrary AI components into our product which might expose customer data to third parties or start hallucinating at any time. NEOMO helped us navigate the somewhat treacherous territory of current AI solutions with our specific use case and our specific parameters in mind. Their deep knowledge about the properties and implications of all types of large language models - both technological and legal - allowed us to build a solution that brings AI to our customer at maximum efficiency and reliability."

Image

Start a conversation with us.

We are always interested in a good conversation, whether with a specific reason or simply to exchange ideas.

You can reach us via the contact form or you can book a meeting directly.