Overall Methodology Reception EU Research Headlines reported the
ARWU's work on 31 December 2003: "The universities were carefully evaluated using several indicators of research performance." A survey on higher education published by
The Economist in 2005 commented
ARWU as "the most widely used annual ranking of the world's research universities." In 2010,
The Chronicle of Higher Education called
ARWU "the best-known and most influential global ranking of universities" and Philip G. Altbach named ARWU's 'consistency, clarity of purpose, and transparency' as significant strengths.
University of Oxford Chancellor Chris Patten has said "the methodology looks fairly solid ... it looks like a pretty good stab at a fair comparison." While ARWU has originated in China, the ranking have been praised for being unbiased towards Asian institutions, especially
Chinese institutions.
Criticism The ranking has been criticised for "relying too much on award factors" thus undermining the importance of quality of instruction and
humanities. A 2007 paper published in the journal
Scientometrics found that the results from the Shanghai rankings could not be reproduced from raw data using the method described by Liu and Cheng. A 2013 paper in the same journal finally showed how the Shanghai ranking results could be reproduced. In a report from April 2009, J-C. Billaut, D. Bouyssou and Ph. Vincke analyse how the ARWU works, using their insights as specialists of
Multiple Criteria Decision Making (MCDM). Their main conclusions are that the criteria used are not relevant; that the aggregation methodology has a number of major problems; and that insufficient attention has been paid to fundamental choices of criteria. The
ARWU researchers themselves, N.C. Liu and Y. Cheng, think that the quality of universities cannot be precisely measured by mere numbers and any ranking must be controversial. They suggest that university and college rankings should be used with caution and their methodologies must be understood clearly before reporting or using the results.
ARWU has been criticised by the European Commission as well as some EU member states for "favour[ing] Anglo-Saxon higher education institutions". For instance,
ARWU is repeatedly criticised in France, where it triggers an annual controversy, focusing on its ill-adapted character to the French academic system and the unreasonable weight given to research often performed decades ago. It is also criticised in France for its use as a motivation for merging universities into larger ones. Several methods for ranking universities were analyzed. Many scholars proposed developing new methodologies for global university rankings, expressing concerns about bias against universities in the Arab region within current ranking systems. They emphasized the need to adjust the weighting of indicators to account for overlooked institutional differences. Indeed, a further criticism has been that the metrics used are not independent of university size, e.g. number of publications or award winners will mechanically add as universities are grouped, independently of research (or teaching) quality; thus a merger between two equally-ranked institutions will significantly increase the merged institutions' score and give it a higher ranking, without any change in quality. == Regional rankings ==