Another Reason Why the *U.S. News and World Report* College Rankings Mean Nothing

Boyer Hall

It looks like U.S. News & World Report is now in the business of rating the best undergraduate teaching colleges and universities.  Over at Inside Higher Ed, Joshua Kim raises some disturbing points about the methodology by which the magazine ranks such schools.

Here is a taste:

Here is how I think the methodology for coming up with this list goes.

Step 1:  U.S. News sends out a peer assessment survey* to college presidents, provosts and admissions deans.

* Note:  I can’t find a copy of this peer assessment survey anywhere.  The latest data that I can find on the sample size is from the 2013 survey, where U.S. News reports that 842 “top college officials” were surveyed, with a response rate of 53 percent.

Step 2:  The surveyed presidents, provosts, and admissions deans are asked to nominate up to 10 schools (in their institutions category) that they judge to have ” an especially strong commitment to undergraduate teaching”.

Step 3:  The rankings are then derived “solely on the responses” of the peer assessment survey, with the ranking order calculated “based on the number of top 10 nominations they received”.  Only schools with 7 or more nominations amongst presidents, provosts and admissions deans as having “especially strong commitment to undergraduate teaching” are included in the list.

And again, so I understand this, the U.S. News rankings for Best Undergraduate Teaching are solely based on a reputational sorting, with that sorting provided only by presidents, provosts, and admissions deans who receive and return the peer assessment survey.

Read the entire piece here.  (HT: Chris Gehrz)

One thought on “Another Reason Why the *U.S. News and World Report* College Rankings Mean Nothing

  1. In my 23 years as a college president I regularly refused to complete this survey and usually sent a letter of concern about the obvious subjectivity of the methodology. If parents and students weren’t so swayed by the annual ratings this wouldn’t be such a problem.

    I realized something was amiss when I noted many colleges were sending mailings to all presidents touting their institution’s accomplishments–timed to arrive just before the annual survey was distributed. In my opinion, the process is stacked against smaller, less well known institutions, some of which are doing great undergraduate liberal arts education.


Comments are closed.