Two Choices Do NOT Equal a 50-50 Chance

“Whoa!” you say. “Everyone knows that you have a 50-50 chance with two choices.  What planet is this crazy lady from? ”

You are not alone in this thinking.  I hear over and over that we have to give MANY choices per question…and have gotten caught up thinking this way myself in the past. We think many choices will prove our kids aren’t simply guessing. When our kids deal with motor impairments, though, too many choices can be an access nightmare. What to do, what to do?

The fact is, with testing, selecting from two choices can give statistically irrefutable results showing mastery of information. Let’s look at how this can be…

First let’s look at what makes a 50-50 chance

We’ll start by backing up to a situation where two choices DO have a 50-50 chance of being correct. Take a coin and predict whether it will land with heads or tails up. Now toss it one time and let it land. At this moment, you DO have a 50-50 chance that it will show heads. Or tails. Agreed?

You could repeat this same random coin toss ten times and about five of those tosses would be heads.

Try it a hundred times and you should get about 50 tosses coming up tails.

Why is this so? Why do half the tosses give heads and half give tails?

Because the only factor at play is luck. All the other factors are equal for each side of the coin on every toss…the balance of the coin, the number of rotations in a flip, the speed of travel... These weigh equally on the outcome. They won’t change the probability of a guess based on luck.

If a student is genuinely guessing at answers, they are strictly shooting based on luck. You could expect about the same results as the coin toss. They would score about 50% correct on the answers of a true/false test, a sorts assessment with two columns, or another binary choice test.

When odds are no longer random

When all the factors to support student success are in place, luck doesn’t particularly influence the outcome. If the child knows the answer, has the appropriate access methods, and is caught on a day when health and sensory issues are met, then a correct answer likely has nothing to do with luck.

The factors at play—knowledge, access, and health issues—change the game completely. Testing is unrelated to the coin toss, even when there are only two possible responses.

When a child is able to score 5 correct responses on a 5-point true/false test, the statistical likelihood that this was achieved by pure guessing is about 3%.

But 3% still leaves a tiny possibility the child was guessing. Three chances in 100. How do we reduce that so we can know beyond a doubt that the child was demonstrating knowledge rather than luck?

It doesn’t take very many corrects to get pretty strong data

Five isn’t many questions for a test, but it IS a good number for a child whose ability to attend to a task is impaired by issues of access, attention, health, or sensory processing. If you can cycle through a longer test in short 5-question chunks, you can get some highly valid information in relatively short time.

5/5 correct can only be achieved by guessing 3% of the time, as we said earlier.

5/5 done twice (totaling 10/10) gives <.1% chance--less than 1 in 1000--of guessing as the means for achieving the score. That’s good enough for me (side note: this would not be adequate if we are talking about skills needed to fly a commercial airliner or perform neurosurgery, but for academic skills for most kids, it works).

5/5 done three times (or 15/15 total) only gives a .00003% chance of guessing…and that is statistically improbable. Most definitely.

The great news is that a child can still demonstrate high levels of understanding even if they can’t get a perfect score every time.

Let‘s say they can manage to get 4/5 correct on a group of True/False test questions. There is a 16% margin for guessing, which is too high to put our money on.

If they can get 4/5 again on a second try (a total of 8/10), the likelihood of arriving at this score by guessing drops to 4%. Still a little high.

The third time they score 4/5 (raising the total to 12/15), the probability of guessing drops to 1%. That is 1 in 100, so getting smaller!

The fourth time they manage 4/5 (or 16/20 now), the likelihood they are guessing drops to only .4%, or 4 in 1000. For basic academics, this satisfies my confidence they are exercising skill rather than luck.

Great news for our kids with motor impairments

This shows that kids with severe motor impairments who can only select between two responses really can demonstrate to us that their answers are intentional.

Obviously, having the motor control to indicate an answer from THREE choices is terrific for giving strong proof that your answers are not guesses. It lets you get to that point faster.

For example, in a 5-question quiz where each question has THREE responses, a perfect score of 5/5 only has a .004% chance of being achieved by guessing. Everyone can agree that this score is pretty unlikely to result from luck!

But when motor skills dictate that TWO choices are best, our kids can still demonstrate their proficiency without the fear of that guessing has influenced their scores. It just takes longer.

Also, it takes longer if they can’t achieve a perfect score every time. But it can be done. That, friends, takes a huge burden of pressure off both the student and his teacher!

A tool to simplify this statistical computation

Back in my college statistics class, we had to crunch the numbers to rule out the probability of guessing. On scratch paper. By hand. It was grueling!

Now, we have a fantastic FREE tool online that can do this for us. In  a split-second!

www.stattrek.com/online-calculator/binomial.aspx

It’s so easy to use!

1.       On the first line, type in the "chance" of getting one problem correct, expressed as a decimal. In a 2-choice problem, the chance is .5. In a 3-choice problem, the chance is .33. For a problem with 4 choices, the chance is .25. Kids with motor issues should not be given more than 4 selections per question, so I won’t go farther.

2.       On the second line, tell how many questions on the test.

3.       On the third line, write how many corrects the student scored.

4.       The fourth line will tell you the probability that the student could have arrived at their score by guessing. The tinier the number, the less chance there is that the student guessed.

Hidden messages:  guessing and intentional misses

Scores that are not perfect (or close to perfect) hide important information for us to uncover.

Those that fall around the 50% chance of being achieved simply by luck were probably the result of guessing. So why is the student guessing? Do they not care? That’s OFTEN the case! Do they not understand? Very possible, but the fact that they haven’t established any kind of generalization, even an incorrect one, means they have not caught ANY of your instruction.

Scores that show a statistical probability that answers are actual errors (say, 0-1/10) rather than guessing are also strongly telling. When you type the figures into Stattrek.com, you will see that the likelihood of guessing only errors is a slim as guessing only corrects. There is a message in a page full of errors. Does the student understand the teaching wrongly, applying the concepts backward? It’s very possible. Is the student trying to tell you she is bored? Hmmm…we need to pay attention to this.

Are you feeling more confident now in providing tests that ask students to select from just two choices? They can be devised to give you very accurate feedback about the child’s understanding.

The key is in the number of repetitions, but even that can be surprisingly small. Once you start crunching numbers, you can see that tests don’t need to be pages and pages of questions. Go small, add more if needed, and check back with Stattrek.com often to see if you’ve eliminated the probability of guessing. You saw from our examples how a perfect score on only 15 true/false questions is statistically irrefutable as being skill rather than luck!

Let me know if you still think I’m a crazy lady from another planet or if this makes sense. Just leave a comment in the box below…I read them all!

* * * * * * * * * *

You might also enjoy reading:

How Do We Reduce the Stress of Testing?

The Best Test for Kids with Motor and Sensory Disabilities