A practical guide to writing quiz questions
Most quiz questions test whether someone remembers a fact. Few people test to see if they truly understand. This is a deep-rooted problem in corporate training and education as well. Although assessment appears thorough on paper, it is unclear whether learners will be able to apply what they have learned. As a result, pass rates are inflated and a false sense of confidence is created in the effectiveness of the training program.
The good news is that you don’t need a degree in psychometrics to write better questions. You need to be more intentional about what each question is designed to measure and avoid some common traps that make questions easier to guess than to actually answer.
Creating quizzes to measure understanding
Start with what you want to measure, not what you want to hear
Before creating a single question, define the learning objective that the question addresses. This sounds obvious, but in reality, most quiz writers start with the content (“We talked about phishing in module 3”) and then write questions about the details (“What percentage of breaches involve phishing?”). Those tests aren’t about understanding, they’re about remembering.
A better approach is to ask yourself, “What should my learners be able to do after completing this module?” If the objective is “Identify suspicious emails,” the question should present a scenario (an email with subtle red flags) and ask the learner to evaluate it. This question tests skill, not statistics.
Avoid questions that can be answered without learning
Some question formats are so common that learners develop strategies to guess correctly without knowing the content. Note the following pattern.
“All of the above” is correct.
When learners see “all of the above,” they usually quickly realize that it is correct. If you must use it, make sure you know it may be wrong. One answer is noticeably longer or more detailed than the other.
Quiz creators tend to put effort into expressing correct answers in order to make them stand out. Keep all options about the same length and level of detail. Negatively worded questions such as “Which of the following is not true of you?”
These test your attentiveness more than your knowledge. If you need to assess what is not, reframe it as a positive question with a scenario. True/false questions on sensitive topics.
In binary options, the learner has a 50% chance of guessing correctly. Use True/False for truly clear facts, and use multiple choice or short answers when judgment is required.
Write a plausible disruptor
The wrong answer in a multiple-choice question is just as important as the right answer. If the distraction is clearly wrong, the question becomes easier, no matter how good the stem is.
Effective distractions are born from real misunderstandings. If you’ve been training people on a topic for a while, you probably already know a common mistake: it’s a distraction. For example, a data privacy quiz asks the question, “When can customer data be shared with third parties?” You should include options that reflect the actual misconceptions, such as “when a customer makes a purchase” or “when a third party signs a general NDA.” They feel plausible because they reflect errors in actual reasoning.
If you don’t know the common misconceptions yet, take the quiz once and see which incorrect answers are chosen most often. That data shows where the confusion exists.
Usage scenarios and application questions
The easiest way to move from recall to understanding is to put the learner in a situation. Instead of asking, “What is the first step in the incident response process?”, describe the specific incident and ask what should happen next. Learners need to be aware of the situation, remember the process, and apply it. This is close to what you actually need to do at work.
Scenario-based questions take more time to create, but provide much more useful data about learner readiness. Learners who can recite the five steps of incident response but cannot identify which steps apply to a particular situation have not really learned the material.
Add explanation to all answers
One of the most underutilized features in quiz design is the description field. If a learner gets an answer wrong, simply showing them the correct answer won’t help them understand why they got it wrong. Explanations that easily address the misconceptions behind each distraction turn assessments into learning moments.
This is especially useful for self-paced training where you don’t have an instructor to provide context. Even if the answer is correct, a short explanation will strengthen your reasoning. “Correct. The indicators indicate an active violation, not just a policy violation, and I’ll escalate it to the security team.” It’s worth more than a green checkmark.
keep language simple and precise
Ambiguous expressions are the enemy of good reviews. If a learner gets a question wrong because they interpret it differently than intended, they have failed the question and are not the learner.
Avoid double negatives, ambiguous qualifiers such as “sometimes” and “usually,” and jargon not explicitly taught in training. Each question must have only one defensible correct answer. If you find yourself writing a long rationale for why one option is “more correct” than another, you need to rewrite your question.
Have someone outside the subject review your question before publishing. Capture ambiguities invisible to the author with fresh eyes.
Measure, then improve
Writing good questions requires iterative work. Once a significant number of learners have completed the quiz, review the data. A question that almost everyone gets the right answer to is not necessarily a good one. It may be too simple. Questions that are evenly spread across all options are likely to be poorly worded rather than truly difficult.
Look at the identification index. Do high-achieving learners get this question correct more often than low-achieving learners? If not, the question is not measuring what you think of it.
The goal is not to make the quiz difficult. This makes your learners more honest about what they actually know, allowing you to focus your training efforts on the parts that matter most.
