Two Choices Do NOT Equal a 50-50 Chance

“Whoa!” you say. “Everyone knows that you have a 50-50 chance with two choices.  What planet is this crazy lady from? ”

You are not alone in this thinking.  I hear over and over that we have to give MANY choices per question…and have gotten caught up thinking this way myself in the past. We think many choices will prove our kids aren’t simply guessing. When our kids deal with motor impairments, though, too many choices can be an access nightmare. What to do, what to do?

The fact is, with testing, selecting from two choices can give statistically irrefutable results showing mastery of information. Let’s look at how this can be…

First let’s look at what makes a 50-50 chance

We’ll start by backing up to a situation where two choices DO have a 50-50 chance of being correct. Take a coin and predict whether it will land with heads or tails up. Now toss it one time and let it land. At this moment, you DO have a 50-50 chance that it will show heads. Or tails. Agreed?

You could repeat this same random coin toss ten times and about five of those tosses would be heads.

Try it a hundred times and you should get about 50 tosses coming up tails.

Why is this so? Why do half the tosses give heads and half give tails?

Because the only factor at play is luck. All the other factors are equal for each side of the coin on every toss…the balance of the coin, the number of rotations in a flip, the speed of travel... These weigh equally on the outcome. They won’t change the probability of a guess based on luck.

If a student is genuinely guessing at answers, they are strictly shooting based on luck. You could expect about the same results as the coin toss. They would score about 50% correct on the answers of a true/false test, a sorts assessment with two columns, or another binary choice test.

When odds are no longer random

When all the factors to support student success are in place, luck doesn’t particularly influence the outcome. If the child knows the answer, has the appropriate access methods, and is caught on a day when health and sensory issues are met, then a correct answer likely has nothing to do with luck.

The factors at play—knowledge, access, and health issues—change the game completely. Testing is unrelated to the coin toss, even when there are only two possible responses.

When a child is able to score 5 correct responses on a 5-point true/false test, the statistical likelihood that this was achieved by pure guessing is about 3%.

But 3% still leaves a tiny possibility the child was guessing. Three chances in 100. How do we reduce that so we can know beyond a doubt that the child was demonstrating knowledge rather than luck?

It doesn’t take very many corrects to get pretty strong data

Five isn’t many questions for a test, but it IS a good number for a child whose ability to attend to a task is impaired by issues of access, attention, health, or sensory processing. If you can cycle through a longer test in short 5-question chunks, you can get some highly valid information in relatively short time.

5/5 correct can only be achieved by guessing 3% of the time, as we said earlier.

5/5 done twice (totaling 10/10) gives <.1% chance--less than 1 in 1000--of guessing as the means for achieving the score. That’s good enough for me (side note: this would not be adequate if we are talking about skills needed to fly a commercial airliner or perform neurosurgery, but for academic skills for most kids, it works).

5/5 done three times (or 15/15 total) only gives a .00003% chance of guessing…and that is statistically improbable. Most definitely.

The great news is that a child can still demonstrate high levels of understanding even if they can’t get a perfect score every time.

Let‘s say they can manage to get 4/5 correct on a group of True/False test questions. There is a 16% margin for guessing, which is too high to put our money on.

If they can get 4/5 again on a second try (a total of 8/10), the likelihood of arriving at this score by guessing drops to 4%. Still a little high.

The third time they score 4/5 (raising the total to 12/15), the probability of guessing drops to 1%. That is 1 in 100, so getting smaller!

The fourth time they manage 4/5 (or 16/20 now), the likelihood they are guessing drops to only .4%, or 4 in 1000. For basic academics, this satisfies my confidence they are exercising skill rather than luck.

Great news for our kids with motor impairments

This shows that kids with severe motor impairments who can only select between two responses really can demonstrate to us that their answers are intentional.

Obviously, having the motor control to indicate an answer from THREE choices is terrific for giving strong proof that your answers are not guesses. It lets you get to that point faster.

For example, in a 5-question quiz where each question has THREE responses, a perfect score of 5/5 only has a .004% chance of being achieved by guessing. Everyone can agree that this score is pretty unlikely to result from luck!

But when motor skills dictate that TWO choices are best, our kids can still demonstrate their proficiency without the fear of that guessing has influenced their scores. It just takes longer.

Also, it takes longer if they can’t achieve a perfect score every time. But it can be done. That, friends, takes a huge burden of pressure off both the student and his teacher!

A tool to simplify this statistical computation

Back in my college statistics class, we had to crunch the numbers to rule out the probability of guessing. On scratch paper. By hand. It was grueling!

Now, we have a fantastic FREE tool online that can do this for us. In  a split-second!

www.stattrek.com/online-calculator/binomial.aspx

It’s so easy to use!

1.       On the first line, type in the "chance" of getting one problem correct, expressed as a decimal. In a 2-choice problem, the chance is .5. In a 3-choice problem, the chance is .33. For a problem with 4 choices, the chance is .25. Kids with motor issues should not be given more than 4 selections per question, so I won’t go farther.

2.       On the second line, tell how many questions on the test.

3.       On the third line, write how many corrects the student scored.

4.       The fourth line will tell you the probability that the student could have arrived at their score by guessing. The tinier the number, the less chance there is that the student guessed.

Hidden messages:  guessing and intentional misses

Scores that are not perfect (or close to perfect) hide important information for us to uncover.

Those that fall around the 50% chance of being achieved simply by luck were probably the result of guessing. So why is the student guessing? Do they not care? That’s OFTEN the case! Do they not understand? Very possible, but the fact that they haven’t established any kind of generalization, even an incorrect one, means they have not caught ANY of your instruction.

Scores that show a statistical probability that answers are actual errors (say, 0-1/10) rather than guessing are also strongly telling. When you type the figures into Stattrek.com, you will see that the likelihood of guessing only errors is a slim as guessing only corrects. There is a message in a page full of errors. Does the student understand the teaching wrongly, applying the concepts backward? It’s very possible. Is the student trying to tell you she is bored? Hmmm…we need to pay attention to this.

Are you feeling more confident now in providing tests that ask students to select from just two choices? They can be devised to give you very accurate feedback about the child’s understanding.

The key is in the number of repetitions, but even that can be surprisingly small. Once you start crunching numbers, you can see that tests don’t need to be pages and pages of questions. Go small, add more if needed, and check back with Stattrek.com often to see if you’ve eliminated the probability of guessing. You saw from our examples how a perfect score on only 15 true/false questions is statistically irrefutable as being skill rather than luck!

Let me know if you still think I’m a crazy lady from another planet or if this makes sense. Just leave a comment in the box below…I read them all!

* * * * * * * * * *

You might also enjoy reading:

How Do We Reduce the Stress of Testing?

The Best Test for Kids with Motor and Sensory Disabilities 

The Best Test for Kids with Motor and Sensory Disabilities

Even kids with severe disabilities need to demonstrate their learning through tests. As we look at creating low-stress tests to accommodate motor and sensory disabilities, what type of tests should we use? Does one style work better than others for these students?  Let’s talk more about the response style that has the best chance for supporting student success.

* * * * * * *

You want your kids to score well on tests.

As a teacher, you work tirelessly to help your students learn content and critical skills. You want to be able to prove to the world just how successful your kids are despite their special needs! That means they need to score as high as possible on classroom tests. We all know that tests play a big role in shaping a child’s educational future. We want our kids to succeed!

Yet no matter how strongly dedicated teachers and parents are, physical challenges still interfere with our students’ abilities to show us what they know. We can’t make these physical problems go away; we have to find a way to work within the same limits our kids face every day.

One standard testing format outshines all the others for accommodating physical disabilities.

To help our kids perform their best on tests, we need to reduce the physical demands of answering test questions. Of the five standard response styles typically used in classroom assessments, one stands far ahead of the others in being accessible to students with severely limited motor skills.

·         Essay—This style of response requires that students generate a lengthy answer, letter-by-letter or word-by-word. For most students with severe motor impairments, essay responses are far too motor-intensive to be practical. Avoid essay questions as a method of testing.

·         Short answer—Students are expected to generate shorter answers, ranging in length from single words or phrases to a sentence or two. When motor disabilities challenge a student, even short answer responses may be too great a demand on physical output. Rather than expecting students to generate answers, short answer questions may be adapted to allow students to choose from a list of words, lessening the motor demand considerably.

·         Multiple choice—This format challenges students to select a response from a list of potential answers. Multiple choice tests are not as easily adapted as one might assume. If we ask students to hold 4-5 possible responses in their heads while they indicate the letter matching the correct response, we ask them to perform a huge memory task. This is a different skill than recalling the information we are trying to draw from our students. If you must use multiple choice tests, ALWAYS assign reminders for each answer choice directly on cards or switches rather than simply labeling them A,B, C, and D.

·         Matching—Matching tests typically ask students to pair items from one column with items in a second column. We might ask them to draw a line to connect the items or copy the letters or numbers that precede each item. Either of these response modes presents a great motor and visual challenge. Instead, print the choices off on colored paper (one color for the prompts and one for the matching choices) and cut them into cards. Students can arrange the matches in pairs on the table, reducing the motor and sensory complexity of this task.

·         True/false, yes/no (aka binary choice)—For kids with motor planning or sensory challenges, this testing strategy far outshines the other four we’ve talked about so far. With only TWO possible responses, this style makes the fewest demands on motor planning. It sharpens cognitive focus by reducing distractions. Think about all the distractions a child has to filter out with a matching or multiple choice task; having only two possible selections reduces this problem greatly. This format makes it easy to present one question at a time, whether by masking off a single question on a page or reading it out loud to the student.

Binary choice tests are bigger than just "true/false."

Test questions that can be answered by choosing from two responses are technically known as “binary choice.” The formats you are most familiar with are probably true/false and yes/no. Questions answered with these right/wrong responses have to be worded with tremendous care, which is a skill in its own right. Poorly worded questions confuse or mislead students. When children lack the verbal skills to ask for clarification, ambiguous wording sets them up for failure. If you need help to write clear questions, refer back to most any Educational Measurement textbook. Ebel and Frisbie’s Essentials of Educational Measurement or Popham’s Modern Educational Measurement are both excellent resources.

The great news is that binary choice options for testing extend past just true/false and yes/no questions. Information can be categorized into columns to clearly demonstrate mastery of the skill being tested. This format has been used in the literacy word with a high degree of success since the 1980’s under the name of “word sorts.” Fortunately for our kids, this sorting format extends to most content areas, varied levels of complexity, and all ages of students. It can be adapted using a wider variety of access methods than most styles of testing.

Sorts used to categorize words starting with /m/

I’m a huge fan of sorts as both a practice and a testing activity. I’ve used them quite often in my classrooms and at home with my motor-challenged daughter. Sorts tend to break down skills into definable increments, making assessments easier to create. I find they direct my instruction leading up to the test so that I focus more intentionally on the objective. I’ve successfully used sorts as an assessment for pre-literate students with autism who needed to have other types of tests given orally—yet whose echolalia interfered with their ability to say anything but the last answer choice they heard. Once they understood the process of sorting answer cards under category headings, they could demonstrate the target skill independently without any verbal input from adults. Their output was now under their control and no longer hampered by echolalia. Recently, my daughter sorted her Agricultural Science vocabulary under the headings of “lactation” and “reproduction” during their dairy unit, showing that sorts are appropriate even through high school.

If you will be testing your students this week, how might you adapt the test to support the motor challenges of your students? Is there a way to convert some of the more complex response styles into simpler binary choice questions? Please leave a comment to let us know how this worked for you this week. Or, feel free to ask for suggestions on how you might accomplish this.

Next week, let’s explore the many ways sorts can be adapted to meet the diverse motor and sensory challenges our kids bring to testing.

* * * * * * * *

Did you catch last week's post on reducing the stress of testing for students? If not, be sure to check it out here!