Jacob Windsor taught a class on data surveillance where he wanted students to understand how improper use of data can distort the truth. He began his class by asking students how many of them had looked at a popular publication’s rankings of universities in their choice of a school to attend. Most of them had.
Next he asked them to write down in their notes what they thought were the components used in the ranking of universities. Then he asked students for their components and put these on the board. Students were then asked to rank the components most important to them. He then circled the components from their list that were actually used in the rankings in the publication. On a good semester, one or two of their components were circled. None of the students’ highest ranked components were used by the publication.
The reality is that rankings often reflect an intentional bias of the people making up the rankings. In the publication’s rankings of universities, the components had a very definite elitist bias.
Measurements need to reflect what really matters. If you are going to rank universities, you need to do extensive work in surveying what parents and students want from a university.
Next, you need to be sure that the measurements are true indicators of what they are actually meant to represent. A popular ranking of the social side of universities often confuses party atmosphere with campus spirit. Universities where students are spirited and engaged are often considered to be party schools.
Rankings should also be scaled in the right direction. A ranking provided by a national accounting firm measures state’s in their attractiveness to businesses. One component in their rankings is the state’s environmental regulatory practices. The more money the state spends on environmental management, the lower their ranking. For most responsible businesses, this ranking is clearly in the wrong direction.
The purpose of the ranking and the funding source behind it also needs to be considered. Rankings are often used for political purposes to shame legislative and executive branches of state governments. Both conservative and liberal groups do this.
If there is one generalization that could be made about rankings, it is this: “Don’t use them for anything other than amusement.” You need to know what’s important to you and do your own research in your evaluations of choices you need to make.
Think about the most recent ranking system you have seen. Do you know how the rankings were done? Just imagine how often we shape national or state policy based upon rankings? Just imagine how rankings give a distorted view of what the public really believes? Just imagine how rankings can be used to perpetuate privilege?
* * *
“If you get good ratings, they’ll cover you even if you have nothing to say.” —Donald Trump
(1)Ground Truth is what is actually happening in society rather than what is being said is happening.