It may be a case of be careful what you wish for.
Seven months ago, dozens of elite law schools and medical schools announced that they were boycotting the U.S. News & World Report rankings and refusing to give the publication any data. The rankings, they said, were unreliable and skewed educational priorities.
Last week, U.S. News previewed its first rankings since the boycott — for the top dozen or so law and medical schools only — and now, it seems, many of these same schools care quite a lot about their portrayal in the publication’s pecking order.
In fact, their complaints about the methodology were so forceful that U.S. News announced on Wednesday that it had indefinitely postponed the ranking’s official publication.
“The level of interest in our rankings, including from those schools that decline to participate in our survey, has been beyond anything we have experienced in the past,” U.S. News wrote on its website, explaining why it was delaying the release.
Yale Law School, the instigator of the boycott, is among those that see the rankings as incorrigible. “What we are seeing unfold with U.S. News on a weekly basis is exactly why so many schools no longer participate,” said Debra Kroszner, an associate dean and chief of staff at the law school. ”It’s a deeply flawed system.”
This latest skirmish — which comes as students are committing themselves to schools, often with U.S. News as a guide — demonstrates that even a boycott enveloped in the ivy of Yale and Harvard may be no match for the influence of the U.S. News rankings system.
Yale exited in November, followed shortly thereafter by Harvard, Stanford, Georgetown, Columbia and the University of California, Berkeley, among others. Harvard was the first medical school to depart, followed by schools like Columbia and the University of Pennsylvania.
Facing a revolt, U.S. News went on a listening tour of more than 100 schools and conducted what it said was the most significant revision of its methodology ever. To fill in the missing data from boycotting schools, it used public numbers from sources like the American Bar Association.
When the rankings preview was released, not much changed. Yale Law School was still No. 1 (though now tied with Stanford). U.C.L.A’s law school bumped Georgetown out of the “Top 14.” Harvard Medical School dropped to No. 3 from No. 1 in the research ranking, replaced in the top spot by Johns Hopkins.
But boycotting schools were still upset over some of the data, especially the way that U.S. News counted after-graduation employment.
U.S. News had said that it would change its methodology and count students on fellowships as employed, with the caveat that the fellowships were long term and required passage of the bar exam (or, at the very least, that a law degree gave an advantage to the fellowships).
Factoring in the fellowships, Yale expected its employment rate to rise to nearly 100 percent from 90 percent. Instead, it dropped to 80 percent, at least from what Yale said it had gathered from hearing about the data through media reports. (Yale said it had not purchased access to the data or been in touch with U.S. News.)
“If this is the employment metric that they’re using for Yale Law School, it’s entirely incorrect and flatly inconsistent with the methodologies outlined on their website,” said Ms. Kroszner.
The University of California, Berkeley, had similar complaints, saying that students in its joint law and Ph.D. program, who take longer to graduate, were being counted as unemployed. The law school’s dean, Erwin Chemerinsky, said he had complained to U.S. News but not yet heard back.
Mr. Chemerinsky, however, batted back any idea that he cared about the ratings.
The problem is not that schools suddenly have become believers in the value of the rankings, he said. Rather they believe that if U.S. News is going to produce rankings regardless of a school’s cooperation, the data should at least be correct.
“I hope that by making this choice we have undermined the credibility of U.S. News, because it has far too much influence over education,” Mr. Chemerinsky said. “But I’m a realist. I know they’re doing rankings. I want to make sure that whatever the data is, it is done accurately.”
To some university officials, the dust-up reveals the hypocrisy of the high-minded schools.
Peter B. Rutledge, dean of the University of Georgia law school, which did not boycott the rankings, said that he thought the changes in methodology were a legitimate attempt to incorporate what U.S. News had learned from its listening tour. His school had one question about the data, and it was answered, he said.
“In my estimation, U.S. News has done its level best to engage deans in a dialogue,” he said. “The radical change in methodology was not something that U.S. News waved its magic wand and plucked out of a hat.”
Mr. Rutledge said that he was respecting the embargo and would not say whether Georgia, which last year placed 29th, rose or fell in the rankings.
To other observers, however, the haggling reveals the arbitrariness of the data that can be disrupted by a simple change in metrics.
Michael Thaddeus, a math professor at Columbia who has criticized the rankings for being too easily manipulated by the schools, said it did not inspire confidence that U.S. News was renegotiating rankings on the eve of their release.
“It’s sort of like the wizard of Oz saying, ‘Pay no attention to the man behind the curtain,’” Dr. Thaddeus said.
Although many organizations rank colleges and universities, U.S. News is probably the most prominent of them. Students across the country use its rankings as a guide to the most prestigious schools, and as a tool for deciding where to enroll. The rankings also affect how prospective employers evaluate graduates.
Schools invest time and money in enhancing the metrics that U.S. News values — for instance, admissions test scores, faculty-to-student ratios, class size and post-graduation employment.
Now it appears that the changes in some of those metrics have had unanticipated consequences for some of the elite schools that demanded them.
“When you think about everything else going on in the world, there’s a side of it that sort of looks like a tempest in a teapot,” Mr. Rutledge, the Georgia dean, said. “Then you realize that this is an industry where the incumbents have for 30 years built their model around a relatively predictable and unchanged regimen for how to produce a highly ranked law school.”
Paul Caron, dean of the Pepperdine University Caruso School of Law, which ranked 52nd last year, suggested that the word “boycott” in this context is a kind of gaslighting. In a recent headline on his blog, he noted that U.S. News had again delayed the release of its rankings because of inquiries, “including from schools that are ostensibly boycotting the rankings.”
