University differentiation in Europe : results from the AQUAMETH project
Although the role of universities in the knowledge society is increasingly significant, there remains a severe lack of systematic quantitative evidence at the micro-level, with virtually all policy discussion based on country level statistics or case studies. This project redressed the balance by examining original data from universities in six European countries – Italy, Norway, Portugal, Spain, Switzerland and the UK (the source of the 2007 book see below), enlarged afterwards to 6 new countries (including Australia which joined on a voluntary basis with their own funds). It has provided micro-based evidence on the evolution of the strategic profile of universities in terms of scientific research, contract research, education and the third mission.
The project started as a small exploratory effort considered as highly risky by the Scientific Council. Very soon (after the first presentation at PRIME conferences), new teams joined to build a consortium of 10 PRIME participants involved in the construction of university-based data bases, in the development of new econometric techniques and in their exploitation.
As such the construction of the databases was considered as a challenge, especially taking into account the descriptors asked for and collected at the level of individual universities (see box). Its success has been an important element of the selection of the consortium for the EUMIDA project which gathers data on some 2500 universities in 27 member states (the consortium is made of 5 core teams, all members of PRIME, surrounded by 27 national experts).
|AQUAMETH database : contents
General information : year of foundation ; city, province, region (NUTS) ; number and type of faculties/ schools/disciplines covered ; governance (public, private) ; type (university, technical college) ; other relevant historical information.
Revenues : Total revenues of the university, General budget of the university (in federal countries divided between national and regional appropriations) ; Tuition and Fees : Grants and contracts, if possible divided between government, international, private and private non-profit ; Other revenues.
Expenditures : Total expenditures (excluding investments and capital costs), Personnel expenditures, if possible divided between personnel categories ; Other expenditures.
Personnel : Total staff (FTE or headcount) ; Professors ; Other academic staff ; Technical and administrative staff.
Education production : Number of undergraduate students ; Number of undergraduate degrees ; Number of PhD students ; Number of PhD degrees.
Research and technology production : ISI publications, Technological production indicators.
A second major achievement has been the development of new econometric approaches based upon the work by Leopold Simar. The Aquameth project has for the first time applied to higher education and S&T a family of new nonparametric techniques, based on the integration of statistical tools with established frontier techniques. These new techniques not only permit the estimation of confidence intervals, but also solve for the main limitations of DEA (sensitivity to outliers, need for large samples of curse of dimensionality) : robust nonparametric techniques (order m), conditional robust non parametric analysis, and bootstrap.
These developments are so visible that it has raised a strong interest within the PhD community. The project has organised two successive summer schools, each with over 25 doctoral students, having to operate a strong selection between applicants (Pisa 11-17 July 2007, Pisa 7-11 July 2008).
They are also visible within the econometric community : After presentations that were granted the best paper award, Pisa was selected to organise the XI EWEPA Conference (European Workshop on Efficiency and Productivity Analysis) (23-26 June 2009).
Furthermore, in view of fostering their use at the level of university managers, these new methods have been incoporated in a software (in a beta version) developed by Philippe van den Eckaut, Harold Fried and collaborators, under a small contractual provision of the project. The software incorporates almost all new developments in nonparametric efficiency analysis and permits the comparison between the efficiency of a given university and a reference group, extracted from the overall dataset on the basis of a principle of stochastic dominance. In other words, each individual university can be compared with dominating universities (i.e. those that employ less inputs and produce the same level of output or more) and dominated universities (i.e. those that employ more inputs and produce the same level of output or less). These two groups are clearly comparable with the target unit, because they are structurally similar in the input structure, but perform differently. Therefore the benchmark is really informative and can support the collective learning of university managers.
A third achievement has been to demonstrate the richness of such an approach. This has been done through a highly visible book (see box below), numerous academic publications, and through active interaction with stakeholders. A paper was presented at the OECD blue sky conference (Ottawa, September 2006). Members of the Aquameth team have been invited to take part to the EUROSTAT Working Party on S&T Indicators and on Higher Education, on a systematic basis. Important exchanges have also taken place with IPTS and UNESCO. Altogether only in 2006-2007, members of the project presented results of it in 39 conferences, book chapers and articles. It has continued since and only a small selection is listed below. Furthermore the project published a synthesis of Aquameth approach and results (January 2009) : “A micro characterisation of the European university landscape : evidence from the Aquameth project”. It is one of the rare papers in social sciences signed by more than 10 authors (26 in total from 20 institutions !), the corresponding author being a young researcher, Cinzia Daraio. It presents key results from 488 universities coming from 11 countries. We cannot account here for the richness and variety of results. Probably one facinating result is, whatever criteria used, the very limited differentiation that describes the European landscape. The exceptions of the UK, and to a lesser extent Switzerland and the Netherlands, are associated to a few universities whose research productivity is at the upper extreme of the European distribution. Another striking phenomenon is that trade-offs between “activities” are important, and for instance ‘research intensive’ universities may not belong to the highest performers in term of undergraduate education. The national comparative treatment that remains at this stage is in great part due to the rather limited (though unique) set (less than 500 universities). The new EUMIDA project, with around 2500 universities, will enable deepen and characterise forms of differentiation, and analyse relative performance depending on intrinsic features of universities (as organisations).
|Universities and Strategic Knowledge Creation : Specialisation and Performance in Europe
Edited by Andrea Bonaccorsi and Cinzia Daraio. Edward Elgar, 2007, 492 pages
Ch. 1 Theoretical perspectives on university strategy, by Bonaccorsi A., Daraio C.
Ch. 2 Universities as strategic knowledge creators : Some preliminary evidence, by Bonaccorsi A., Daraio C.
PART I : Transversal Topics
Ch. 3 Changing patterns in funding of European higher education institutions, by Lepori B.,
Benninghoff M., Jongbloed B., Salerno C., Slipersaeter S.
Ch. 4 The third mission and the entrepreneurial university model, by Gulbrandsen M., Slipersaeter S.
Ch 5 Efficiency and productivity in European Universities : Exploring trade-offs in the
strategic profile, by Bonaccorsi A., Daraio C. and Simar L.
PART II. National patterns
Ch. 6 Patterns of diversity in the Swiss Higher Education System, by Lepori B.
Ch. 7 University autonomy and structural constraints in the Italian system, by Bonaccorsi
A., Daraio C.
Ch 8. Cost structure, economies of capacity utilization and scope in the Swiss university
Industry, by Filippini M. Lepori B.
Ch. 9 The UK knowledge production function, by Crespi G.
Ch. 10 The Portuguese Public University System : on the Road to Improvement, by Teixeira P., Cardoso MF., Sarrico C., Rosa MJ.
Ch 11 Expansion and reorganization in the Spanish higher education system, by Garcia
PART III. Methodological issues, Tools and Data Platform
Ch. 12 Indicators for the analysis of Higher Education Systems : some methodological
Reflections, by Bonaccorsi A., Daraio C, Lepori B.
Ch. 13 Variables for analyzing Higher Education Institutions in Europe, by Slipersaeter S.
Ch. 14 The University Benchmarker : An Interactive Computer Approach, by Bogetoft P., Fried HO., Vanden Eeckaut P.
|AQUAMETH : Selected publications and conferences
Bonaccorsi A., Daraio C., Geuna A. (2010) Universities in the new knowledge landscape. Tensions, challenges, change. Introduction to the Special Issue on Universities, Minerva.
Lepori, B., Probst,C. and Baschung,L. 2010. "Patterns of subject mix of higher education institutions : a first empirical analysis from the AQUAMETH database." Minerva.
Daraio C., Bonaccorsi A. et al. (29 authors) (2010) A micro characterization of the European university landscape : evidence from the Aquameth project. Forthcoming, Research Policy.
Bonaccorsi, A. and C. Daraio 2009. Characterizing the European university system : a preliminary classification using census microdata. Science and Public Policy, 36(10), 763-775.
Bonaccorsi A., Daraio C. (2008), The differentiation of the strategic profile of higher education institutions. New positioning indicators based on microdata. Scientometrics, 74 (1), 15-37.
Bonaccorsi A., Daraio C., Geuna A. (2009) Nobody is selling the Ivory Tower. An examination of trade-offs in university production. Paper presented to the XI EWEPA Conference (European Workshop on Efficiency and Productivity Analysis), Pisa., June 23-26.
Bonaccorsi, A., Daraio C., Lepori, B. e Slipersaeter S. (2007), Indicators on individual Higher Education Institutions. Addressing data problems and comparability issues, Research Evaluation.
Bonaccorsi, A., Daraio C. e Simar, L. (2006) Advanced indicators of productivity of universities. An application of Robust Nonparametric Methods to Italian data, Scientometrics, Vol. 66, No. 2, 389-410.
Bonaccorsi A., Daraio C. and Simar, L. (2007), “Scale, age, substitution effects and their interactions in academic production. A disaggregated analysis at discipline level”. Paper presented at the EWEPA X Lille, June 2007.
Bonaccorsi A. Daraio C. (2006), “Micro–level indicators of knowledge production : The AQUAMETH project on European universities”, presented at the Blu Sky II 2006 OECD Indicators Conference , Ottawa, September 2006.
Bonaccorsi A. Daraio C. (2006), “The differentiation of the strategic profile of higher education institutions. New indicators based on microdata”, presented at the 9th International Conference on S&T Indicators, Leuven, September 2006.
Bonaccorsi, A., Daraio C., Simar, L. (2006), “A Comparative Efficiency Analysis on the European University System. First evidence of a characterization at micro level based on nonparametric methods”, presented at the North American Productivity Workshop, New York, June 2006.
Bonaccorsi A. Daraio C. (2006), “Institutional Differentiation in the European University System. New Evidence from the AQUAMETH Project”, presented at the Atlanta Conference on Science and Technology Policy 2006, US-EU Policies for Research and Innovation”, Atlanta 18-20 May 2006.