The Problem With Measuring Math Ability

It’s All Just a Game

By reducing down to a single number or letter grade the complex set of skills, behaviors, and values required to do mathematics, a signaling game is created. By signaling game, I mean a situation where a signaler (the student) wants to share information (their math ability) to an uninformed party (admission’s office). There is something to gain for both the student and the admission’s office if the information about the student’s math ability is shared reliably. However, the interests of the student and the admission’s office do not entirely coincide.

The main reason this game exists is because there is no widely accepted, straightforward, and direct way to observe someone’s math ability like there is for someone’s height, weight, or even blood type. The admission’s office cannot simply ask the student, “What is your math ability?” or use some type of device to measure it. The admission’s office must try to deduce the student’s math ability based on actions of the student. To determine math ability, students need to produce signals in the form of standardized test scores (SAT, ACT, GRE, achievement tests, etc.), which they send to admission’s offices. The standardization of these signals makes deducing math ability a very simple process for admission’s offices.

It is important to separate the signal of one’s math ability from one’s actual math ability. They are not the same thing. The signal only represents one’s mathematical ability.

Now, two questions come to mind:

  1. How reliable are the signals students send about their math ability?
  2. What does it cost a students to send a strong signal about their math ability?

A Reliable Signal

Signals are used to convey information between living things all the time. Biologists consider signals to be reliable, or “honest”, only if they convey useful information to the receiver of the signal. So, to be reliable, a signal does not need to be perfectly informative eliminating all uncertainty. The reliable signal just needs to provide an advantage to the receiver that the receiver would not have without the signal. This means that ultimately, the value of the signal depends entirely on how much it helps the receiver with very little regard given what it costs the signaler to send it.

Admission’s offices base their decisions on the best available signal for deducing math ability: standardized test scores and grade point averages. They have used these signals for a number of years. The schools with these admission’s offices appear to still be doing very well. As a result, the only conclusion I see is that the student-produced signals representing their math ability are indeed reliable, honest signals.

The Interests of Admission’s Offices

We must take a moment to consider the interests of these admission’s offices, which are receiving the student-produced signals. Admission’s offices are looking for those who are most likely to succeed in their environment. They know that to be successful in the school’s environment, a student must produce more tests, grades, and other signals similar to the ones produced for the application. By observing the strong signals from a student in the form of a high SAT score or GPA, admission’s offices deduce that the student is likely to produce similarly strong signals while in their environment, which the school can take credit for building their reputation and welfare.

The main focus of admission’s offices is not to find weak signalers who have the potential to be strong signalers. Deducing who those students might be would be more complicated than finding already strong signalers. What’s more is that turning weak signalers into strong signalers requires much more effort from the school than maintaining strong signalers. The main function of admission’s offices and the schools who have them is about identifying and enrolling already strong signalers and enabling them to produce more strong signals while enrolled at their school. This is evident from many admission’s offices publishing the lowest acceptable test scores and GPAs to even be considered for acceptance into the school.

At What Cost to the Student

Back to biology. The handicap principle suggests that reliable signals must be costly to the signaler. They must cost the signaler something that could not be afforded by an individual with less of a particular trait. If we look at the classic example of peacocks, their extravagant tail feathers signal to peahens that, “Despite having this huge tail that slows me down when I am chased by predators, is a desirable home to insects and parasites, and is a burden in all other ways – I have survived. Therefore, you must see that I am more fit and a more attractive mate than other peacocks.”

Humans practice conspicuous consumption to the same end. Some humans want to be able to signal their wealth and attractiveness with expensive cars, watches, or clothing, which others without as much money cannot afford. Biology goes so far as to say that reliable signals such as a peacocks plumage or a Rolex watch are reliable signals because “inferior” signalers cannot afford to produce such wastefully extravagant signals. The value of a signal to the receiver is directly related to the cost to the signaler.

So, what is this particular trait that some math students have, which allows them to produce strong, wastefully extravagant signals that those without cannot? I don’t know, exactly, and I’m skeptical of anyone that says they do know. It likely has something to do with time, energy, and resources. Now, why some students have and apply the time, energy, and resources towards costly signals while others students do not is something those more politically minded than I can debate.

I think this is the wrong question to focus on. The question I’m interested in answering is how can we directly observe math ability in students instead of relying on them to produce wastefully extravagant signals. Imagine if we all walked around with our net worth visible for all to see. A signal of a fancy car or watch representing your wealth would no longer be meaningful since everyone could directly observe your wealth without having to rely on signals and deduction.

Directly Observing Math Ability

Look, all the peahen wants to know is if the peacock will be a good mate. The peahen doesn’t know about parasites, viruses, diseases, sperm count, flagellar motility, or anything that will directly tell her that the peacock will be a good mate. So, all the peahen can do is look at the peacock’s plumage (or lack thereof) and make deductions.

However, if we could teach the peahen all about diseases and parasites and pedigree and sperm and how to take a sample and use a microscope, the feathers would not be as big of a factor. Certainly, there would be a correlation between plumage size and the quality of a mate. After all, those extravagant tail feathers are a reliable signal. But, if I was a peahen and I had the choice, I’d look at the science and not the feathers. Not very romantic, I know.

Humans are not peafowl! As long as we are relying on wastefully extravagant signals, we are operating on imperfect information and fostering a system that benefits those with the most resources. Math ability should not be dependent on socioeconomic status. Instead of creating more math tests, grades, and signals that require more and more student resources to signal with, we should be creating ways to directly observe math ability.

Make no mistake, math ability does require time, energy, and resources to develop. However, all the time, energy, and resources that a student has should go into developing math ability and not towards developing math ability plus the ability to produce a strong signal about their math ability. Remember, these are not the same things.

What Are We Even Looking For?

In the end, the fundamental problem with measuring math ability is that we have no idea what we are even looking for. If every admission’s office had time to sit across from every applicant and asked them to demonstrate their math ability, what would the student do? What would the admission’s office like to see them do? Directly observing math ability requires a fundamental and widely accepted understanding of what math ability is, which amazingly is missing from society.

Based on the current standardized tests, it appears that speed is a large component of math ability. I don’t think many mathematicians would agree with that, yet articles like this are published quite frequently. Artists have portfolios or auditions, which are the result of years of work. Admission’s offices listen and look at each one. Writers create essays that must be read and critiqued and athletes have highlight reels, which are studied and evaluated. Yet, mathematicians have a single number from one test that took a few hours on a Saturday. Surely, math ability is more than this and surely we are smart enough to create ways to observe it.

What if admission’s offices accepted math audition tapes, math portfolios, or math highlight reels instead of a single score to a single test? How would this change the admission’s process? How would it change the years of math education leading up to the admission’s process? How would students demonstrate math ability?