AI in Medical Training

AI for Writing MCQs and OSCE Stations: A Safe Workflow for Medical Educators

How medical educators in Malaysia can use AI to draft single-best-answer questions and OSCE stations, and the expert review that has to follow every draft.

  • By
  • 4 min read
  • Facts checked
Short answer

AI can draft single-best-answer questions, OSCE station scripts and marking sheets in minutes, but the drafts carry typical flaws: implausible distractors, cues to the right answer, outdated guidelines and questions that test recall instead of reasoning. Use AI after you have a blueprint, never paste real patient cases, and let a subject expert review and edit every item before it reaches a student.

Part of the guide: AI in healthcare and medicine

On this page
  1. Where AI helps in assessment, and where it does not
  2. Step 1: blueprint before you prompt
  3. Step 2: check the draft for the usual flaws
  4. Step 3: drafting an OSCE station
  5. Step 4: review, record, re-use
  6. What this saves, honestly
  7. Learn the workflow with your faculty
  8. Frequently asked questions

Writing assessment items is one of the slowest jobs in medical education. A good single-best-answer question can take an experienced examiner half an hour. An OSCE station with a candidate brief, a simulated patient script and a marking sheet takes a morning. AI shortens the drafting. It does not shorten the thinking, and that is where most mistakes happen.

Checked on 7 November 2026. This article is about teaching and assessment design. It is not medical advice.

Where AI helps in assessment, and where it does not

Task AI is useful for The examiner still owns
Single-best-answer (SBA) questions Drafting stems, five options, an explanation Clinical accuracy, the right answer, distractor quality
Extended matching questions Generating option lists and parallel stems Whether the theme tests reasoning
OSCE stations Candidate brief, patient script, examiner notes The standard, local protocols, the pass mark
Marking sheets Checklist items and global rating anchors Weighting and what counts as unsafe
Feedback after exams Grouping common errors from anonymised results What to teach differently

The pattern is the same everywhere: AI produces volume, the expert produces validity.

Step 1: blueprint before you prompt

Most bad AI questions come from a bad request. "Write ten MCQs on cardiology" returns ten recall questions about textbook facts. Start from the blueprint you already use: the learning outcome, the level of the candidate and the skill you want to test.

A useful prompt names five things:

  1. The candidate: final-year medical student, house officer, emergency medicine trainee.
  2. The outcome: for example, "recognise inferior STEMI on a 12-lead ECG and choose the first management step".
  3. The skill: diagnosis, next step, interpretation or prioritisation, not recall.
  4. The format: one clinical vignette, a lead-in question, five homogeneous options.
  5. The standard: ask the model to follow an item-writing guide such as the NBME Item-Writing Guide, and to explain why each wrong option is wrong.

Never paste a real patient case, a released past paper with copyright restrictions, or unreleased exam items. Constructed cases are safer and usually teach better. Our article on patient data and AI in Malaysia explains where the line is.

Step 2: check the draft for the usual flaws

AI-written items fail in predictable ways. Train your reviewers to look for these first:

  • Implausible distractors. One option is obviously wrong, so the question becomes a four-option guess.
  • Cueing. The right answer is the longest, the most specific, or repeats a word from the stem.
  • Absolute words. "Always" and "never" in wrong options give the answer away.
  • Outdated or foreign practice. The model defaults to US or UK guidance. Check against the guideline your candidates are taught, including Malaysian Clinical Practice Guidelines where they exist.
  • Two defensible answers. Fluent explanations can hide that a second option is also correct.
  • Recall dressed as reasoning. A long vignette that only asks for a fact.

A simple routine: answer the question yourself before reading the AI's key. If you hesitate, so will a good candidate.

Step 3: drafting an OSCE station

AI is especially useful for the parts of a station that take time to write but carry little risk: the candidate instructions, the simulated patient's background, the answers the patient gives if asked, and a first marking checklist.

A workable order:

  1. Write the station objective and the examiner's "must do" items yourself.
  2. Ask AI for a candidate brief of no more than 80 words.
  3. Ask for a simulated patient script with opening line, history details released only when asked, emotional cues and one challenging question.
  4. Ask for a checklist and a global rating scale with anchors, mapped to your objective.
  5. Run the station once with a colleague before any candidate sees it.

The MMC guideline on AI in medical practice warns against over-reliance and skills degradation (clauses 3(10) and 3(11)). In assessment that means one thing: the examiner decides what competence looks like. Our summary of the MMC guideline covers the clauses in full.

Step 4: review, record, re-use

Treat AI items like any other draft in your question bank:

  • Named reviewer. One subject expert signs off each item.
  • Version note. Record that AI assisted the draft and which tool was used. Some boards will ask.
  • Post-exam analysis. Difficulty and discrimination statistics show quickly whether AI-assisted items behave like faculty-written ones.
  • Security. Unreleased items stay out of consumer accounts that may store or train on content.

What this saves, honestly

Faculty who adopt this workflow usually report that drafting gets much faster while review takes about as long as before. The gain is that experts spend their time judging items instead of typing them. If your review step gets shorter too, something is being missed.

Learn the workflow with your faculty

Assessment with guardrails is module three of our course AI in Medical & Clinical Education, taught by two emergency physicians with national exam-board experience. Participants draft SBA questions and an OSCE station with AI during the course and review them together. For the wider picture, see our guide to AI in healthcare and medicine and seven uses of AI in medical education. Hospitals and private colleges that are registered employers can claim the course from HRD Corp.

Frequently asked questions

Can ChatGPT write good MCQs for medical exams?

It can write usable first drafts. Studies that compared AI-drafted and faculty-written questions found the drafts a reasonable starting point but more prone to item-writing flaws. Treat every AI question as a draft that a subject expert must review.

Is it allowed to use AI for exam questions in Malaysia?

No Malaysian rule forbids it, but your faculty or examination board may have its own policy on AI and on exam security. Check that first, and never put unreleased exam items into a tool that may store or train on them.

Which AI tool is best for writing exam questions?

General models such as ChatGPT, Claude and Gemini all draft questions well when the prompt includes a blueprint, a learning objective and an item-writing standard. The method matters more than the brand. Use the account your institution has approved.

Can AI mark OSCE stations?

Not on its own. AI can draft a marking sheet and a global rating guide, and some platforms analyse recorded stations, but the examiner's judgement stays the standard.