Guest Article for the 10th Anniversary of TJHE

Tuning in Higher Education: Ten years on

Julia María González Ferreras and Robert Wagenaar[*]

doi: https://doi.org/10.18543/tjhe.2879

E-published: November 2023

Abstract: When the first issue of the Tuning Journal for Higher Education was published in November 2013, the Tuning initiative had become of global significance, running projects in all continents. These all focussed on curriculum reform backed up by an internationally defined approach based on the paradigm of outcome based, that is, student-centred and active learning, in addition to agreed reference points. Around the same time, the Tuning aims and objectives were extended. This reset followed societal and technical developments. More emphasis was put on social inclusions, the involvement of students, identifying relevant topics, and to the development of practical tools. Also, more attention has been given to staff development responding to the notion that the modernisation of the higher education sector as well as the reform of degree programmes proved to be rather slow and a bit disappointing. To boost the process, Tuning – very recently - developed general Qualifications Reference Frameworks for all learning cycles as well as a Guideline to use these for Quality Assurance. For five disciplinary fields it also developed transnational diagnostic tests as a means to identify shortcomings in degree programmes, pushing for change. As a result, Tuning has reinvented itself to stay highly relevant for the years to come.

Keywords: Tuning Project; Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe (CALOHEE); Qualifications (Reference) Frameworks; OECD Assessment of Higher Education Outcomes (AHELO); Joint Programmes; European Standards and Guidelines for Quality Assurance (ESG); practical tools; inclusiveness; paradigm change; role of academics.

I. Introduction

The decision to launch a Tuning Journal for Higher Education did not come out of the blue. November 2013 was exactly 13 years after the idea of launching a large-scale project to reform higher education in Europe was born. In these years, Tuning developed from a European initiative to a global one, involving all continents. It showed great interest in the reform agenda set by Tuning. Key being the Tuning model for the modernisation of higher degree programmes, based on the student-centred approach and backed-up by reference point documents for disciplinary areas. During the years, the Tuning approach developed further as a result of the many projects that were set up with the financial, moral and political support of the European Commission.

In November 2012 the Tuning World Conference took place in Brussels which was organised together with the Directorate General Education and Culture of the European Commission. The conference was attended by approximately 750 academics and representatives from all parts of the world. The initial idea of the Tuning Educational Structures in Europe project to create a platform for discussion among academics to enhance the quality and relevance of higher education by modernising degree programmes had turned into a reality. Having an applicable model in place is one thing, the World Conference confirmed that it would be very helpful to support this model by research for which a related academic journal would be useful. An academic peer-reviewed journal of high standard to offer researchers worldwide, a scholarly platform to publish on educational topics initiated by Tuning, or at least related to its aims and objectives. The composition of its Educational and Advisory Board should reflect its global outreach.

The foundation for such a Journal was created in the years just preceding its launch. Years which proved to be highly fruitful and successful to share and discuss the Tuning concepts with other world regions. Tuning projects were launched and implemented concerning Africa (2011-2013), Australia (2010-2011), Canada (2011-2012), Central Asia (2012-2016), Latin America (Tuning America Latina III) (2011-2013), Lithuania (2009-2012), Russia (2010-2013), Thailand (2011-2013) and the USA (2011-2012), the last one supported by the Obama government.[1] Implementing projects in different world regions also allowed for comparison.[2] On request of the European Commission, in 2012 a study started in close cooperation with the Ministry of Education of China.[3] It would cover a period of 7 years. In the same year the outcomes were published of the Sectoral Qualifications Project HUMART, Humanities and Performing and Creative Disciplines, co-financed by the European Commission (2010-2012).[4] It was a follow-up of a comparable project focusing on Social Sciences (2008-2010).[5] These two projects show that although going global, Tuning still kept seeing an important role for itself in Europe.[6] The year the first issue of the Tuning Journal was published, November 2013, was also the start of the Tuning Middle East and North Africa (T-MEDA) project (2013-2016).[7]

II. New initiatives and directions

All projects which started in 2013 and before had in common that they focussed on (1) curriculum reform based on the paradigm change from expert or staff driven education to student-centred and active learning – introduced Europe wide by Tuning in 2001 – and (2) the development of agreed reference points / standards for individual subject areas (disciplinary approach, including multi-disciplinary and interdisciplinary degree programmes). This was also the case for the regional projects Tuning Asia-South East (TA-SE) (2016-2019)[8] and Tuning India (2016-2018).[9]

However, the Tuning agenda was also reset from 2013 which was triggered by a number of experiences, resulting from the implementation of the projects mentioned, and observations made, based on global educational and societal developments. From the Sectoral Qualifications Projects, it was learned that the two competing overarching European qualifications frameworks, the Framework of Qualifications for the European Higher Education Area (Bologna Process) and the European Qualifications Framework for Lifelong Learning as well as the national qualifications frameworks for both and the Tuning Reference Points for subject areas, were not sufficiently aligned, to be effective in practice. Building on the European Commission Implementation Reports and EUA and ESU reports concerning the Bologna Process, it was noticed that the modernisation process of the EHEA made limited and uneven progress in many ‘Bologna’ countries.[10]

To obtain more insights about the level of implementation of the paradigm change to output-based or student-centred learning – one of the main objectives of Tuning and formally made an aim of the signatory countries of the Bologna Declaration from 2009 – Tuning implemented a study in the years 2013-2016. This Tuning EU-USA Study was co-financed by the European Commission and the Lumina Foundation. This Foundation based in Indianapolis had been a strong promotor of the Tuning philosophy in the USA for years, supporting in organising projects and offering financial support to a range of disciplinary fields, of which History, stands out.[11] The findings were published in a paper published in the Tuning Journal for Higher Education.[12] Although it was concluded in the paper that there was broad knowledge of the outcome-based approach among in particular higher education management and partly university staff, there was a disappointing level of implementation in many universities and departments which resulted in its title: A Long Way to Go ….

III. AHELO and CALOHEE

In these years, a learning experience was also the OECD Assessment of Higher Education Outcomes (AHELO) feasibility study.[13] This global multi-million project aimed to measure the level of learning of generic competences / transferable or soft skills, e.g. ‘critical thinking’, and of economic studies and civil engineering at the end of the first cycle at country level. In its design it was comparable to the PISA test for secondary education. Tuning obtained the assignment to develop the reference frameworks for both academic fields.[14] The outcomes of this study proved to be disappointing, due to its approach to focus mainly on subject specific competences and, in a separate strand, ‘critical thinking’. The critical thinking test applied, was based on a US one which proved not to be very appropriate for many other world regions. A possible follow-up proposed by the OECD obtained severe criticism from in particular UK and US academics, who made the argument that educational programmes were highly culture bound.[15] It was terminated as a result. Only a much smaller study was implemented focussing again on ‘creative and critical thinking skills’.[16]

The International Tuning Academy, established on the basis of the Tuning projects experience at Deusto and Groningen Universities in 2011 and 2012, was of a different opinion. An opinion discussed in much detail with the Directorate General for Education and Culture of the European Commission. The analysis of the Tuning directors was that the OECD project had its flaws and weaknesses but the basic idea to identify the strength or weaknesses of educational programmes and in summary national systems by applying a transnational test, could be an indicator and stimulus to promote reforms. Conditional would be that any test should take into account the variety of missions and profiles of degree programmes, different educational cultures and be focussed on a combination of high-level generic and subject specific competences, formulated as learning outcomes.

This philosophy resulted in the project Measuring and Comparing Achievement of Learning Outcomes in Higher Education in Europe (CALOHEE).[17] In terms of implementation three stages were identified. A first on defining qualifications reference frameworks and more detailed assessment / learning outcomes frameworks for initially five academic fields: civil engineering, history, nursing, physics and teacher education. Learning from the HUMART project these frameworks to be based on a combination/merger of the two overarching European Qualifications Frameworks. They were published in 2018.[18]

Comparable frameworks have been developed for an additional 6 academic fields since, that is creative and performing arts and design, business administration, ICT, international relations, medicine and occupational therapy, which will be published at the end of 2023. The initial group of disciplines moved on to the second stage, the design of comparable international diagnostic tests to be understood as a feasibility study. In 2023 the outcomes of this study were published. The main conclusion: it is indeed possible to develop fair and reliable tests or assessments which indicate the level of learning of high level subject specific and generic competences as an instrument for diagnosis of the quality of individual degree programmes and their relevance for society. Each of the disciplinary groups developed detailed examples of applicable tests.[19]

As part of the CALOHEE projects, two other initiatives were initiated to support the preparation process of state-of-the-art reference frameworks. The first one was defining separate frameworks for five burning societal topics: multicultural society, political awareness, ethics and values, information society and sustainable society/ies / climate change integrating UN Sustainability Goals in all educational programmes.[20]

A second initiative was developing Tuning-CALOHEE General Reference Frameworks for the short or associated degree and the first and second cycle degree – Bachelor and Master – and the doctorate.[21] These frameworks are based on the same ‘merger’ approach as the academic fields. Defined not to compete or replace the existing overarching frameworks, but as a state-of-the-art completion of the existing ones. In addition, also a detailed Guideline was developed to apply the CALOHEE general and subject-specific frameworks in the European Standards and Guidelines for Quality Assurance (ESG).[22] The Frameworks and the Guideline are expected to be of a great help for developing profiles of joint programmes and degrees as well as to serve as a foundation for robust cooperation in European Universities Alliances, enhancing quality, jointness and consistency of curricula. This is evidenced in the successful, inspirational and standard-setting Erasmus Mundus Joint Degree Programmes, Euroculture[23] and NOHA,[24] which are both based on the Tuning philosophy; joint degrees being perceived as the most profound experience of jointness and genuine European “Products” linked with quality and excellence.

The CALOHEE initiative should be perceived as a fundamental addition to its original aims and objectives. By making international comparative tests, it moved from offering references of quality to measuring whether the quality standards defined, are actually being met.[25] These initiatives should be understood against the notion that three key Bologna tools need urgent updating, taking societal and technology developments into account: European / national frameworks, ESG and the ECTS Users’ Guide 2015, now between 8 to almost 20 years of age.

The CALOHEE model and experience has been used as an important foundation for defining and designing new Erasmus+ Capacity Building Projects, e.g. Aprendizaje Centrado en el Estudiante en América del Sur (ACE) (2020-2024)[26] and Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Asea (CALOHEA) (2021-2024), in practice covering the ASEAN region.[27]

IV. Higher level of inclusiveness

Has the societal responsibly of the higher education sector being highlighted in the CALOHEE projects, the data about educational reforms not yielding sufficient results, inspired to additional action. The Bologna Follow-up Group (BFUG)[28] as well as Tuning noticed lack of progress of implementing the Bologna key commitments. One of the reasons: substantial lack of training of academic staff in the vast majority of countries to act successfully as 21^st century teachers. Tuning observed that higher education teachers in practice are ‘operating as pilots with the experience of passengers’. Therefore, it included substantial training of staff in all of its new initiatives. It also stipulated explicitly in the project designs that internationalisation and curriculum reform go hand in hand and should involve not only disciplinary experts, but also administrative and supporting staff and management, e.g. in ACE, CALOHEA and Resources for Internationalisation of Higher Education Institutions in India (RISHII) (2020-2024). Although student representatives had been present in nearly all Tuning projects, in the ACE one it was decided to involve substantial numbers of students, making them comparable in roles and contributions to members of staff. Process and outcomes showed this to be a very successful strategy for developing ownership and reform.

A second line has been the focus on social educational issues and practical tools; all focussing to empower the position of the learner. In the Latin America III project attention was given to the social dimension as part of it. An explicit first initiative regarding the social dimension was the Developing All-Round Education (DARE+) project (2014-2017) coordinated by the University of Granada but conceptually developed by Tuning Academy staff.[29] The aim: to identify and to find recognition for non-formal but in particular informal learning in a higher education context.

Another initiative worth mentioning in this respect is the project FORTH: Formation of Teachers in Challenged Areas (2018-2023).[30] The aim of this project to develop a Bachelor Minor and a Master Major for teacher education to empower primary and secondary teachers to operate more successfully in challenged (and remote) areas, has been fully met. The model developed allows for implementation in and adaptation to the needs in all world regions. The project involved the ministry of education, key stakeholder organisations, and five universities and fifteen schools spread well over the country. The Master Major obtained full accreditation of the Philippine authorities and its vice-president and minister of education paid tribute in a video message to those involved to make the initiative a success.[31]

V. New and better practical tools

Another new direction has been the development of helpful tools to be applied by practitioners. Already in 2010 such a tool was published in the A Tuning Guide to Formulating Degree Programme Profiles. Including Programme Competences and Programme Learning Outcomes, prepared in close cooperation with credential evaluators (ENIC-NARIC).[32] The instrument proved to be very useful and needed. At present, an update is prepared applying the CALOHEE frameworks and findings. Another project fitting this context, is the Knowledge Alliance project ‘Integrating Entrepreneurship and Work Experience in Higher Education (WEXHE)’ (2016-2020).[33] This project resulted in nine comprehensive information packages for organising work placements / internships, traineeships and entrepreneurship. Identifying three types for each of them. The WEXHE Guides are based on the Tuning-CALOHEE approach defining three levels of achievement of generic competences in statements / learning outcomes. The initiative has been continued in the project eWBL: electronic work-based learning which is coordinated by the Fachhochschule Münster involving a crucial role of the International Tuning Academy based in Groningen.[34]

Recently, a project, entitled Supporting Teachers Who Support Student Transition (START) (2022-2025), started that focusses on preparing academics teaching in the first year of the Bachelor better for their role in the educational process. A toolbox of effective strategies is developed by teachers, involving students, which intend to help freshmen with different backgrounds in higher education to learn more successfully.[35]

VI. Academic contribution

As said, the Tuning Journal of Higher Education was established to offer a platform for Tuning related research, although the Tuning approach also obtained a high level of visibility in academic publications of scholars not directly involved in the Tuning initiatives. By far most citations are related to the project implemented in Europe and Latin America.[36] To boost the prestige in academic circles it was thought helpful to publish not only project reports – some 100 so far[37] - but also present findings in academic volumes and scholarly papers published in prestigious international journals, besides the Tuning one. This strategy is partly visualised in the footnotes and biography of this paper. In addition, it was thought useful, may be even necessary, to outline the role of Tuning in the context of the development of the Bologna Process and that of the European Credit Transfer and Accumulation System.[38]

It was Tuning – on invitation of the European Commission and with support of the European University Association (EUA) - that transferred ECTS from a credit into a credit and accumulation one in the period 2001-2004.[39] Tuning experts played a central role again in the preparation of the ECTS Users’ Guides of 2009 and 2015. Nevertheless, despite the different strategies applied and the Tuning approach and toolbox developed, the implementation of the key instruments has been slow. At one moment the director for higher education sector of DG EAC, Adam Tyson, raised the question whether Tuning had been able to make a difference.[40] The response was formulated in an academic contribution entitled The Myth of Power: Governing Reform in the Bologna Process of Higher Education.[41] It makes the argument that change in higher education is a multi-level process and can only succeed when all levels involved, European/national policy makers, higher education management, academic staff, educational experts and students. In other words, Tuning is a factor among others.

VII. Contribution in the formation of academics and champions in higher education

Might the policy impact of Tuning initiatives having its limitations, the contribution of Tuning in the formation of academics and champions in Higher education reform in Europe and the world is surely most important, decisive and successful. Many hundreds of people involved in each project developed ideas, gave shape to detail, shared their contributions, convinced their authorities and colleagues both at their university and at country level to make change. They dreamt the programmes and shaped in their minds the new profiles, they owned the project and provided the passion that made it possible; they travelled many airports and gave their time freely with great generosity. They shared their vision, their knowledge, and their commitment attending meetings and writing summaries, coordinating ideas and creating new and challenging experiences for the students. These are the Tuning experts, those who made the projects possible, those who involved departments and faculties to embrace a new and more demanding way of thinking of indicators and profession for a global world looking for common points of reference.

Without these Tuning champions, the initiatives would have been sets of lifeless imaginations. Without their robust academic background and their capacity to share and to build, learning from each other´s cultural roots, Tuning would never have crossed frontiers and reached different regions of the world. They were the actors and the authors of a blow of inspiration that shook higher education during now close to a quarter of a century in a genuine search for transparency and internationalization for quality and relevance.

VIII. In conclusion

Without exaggeration, one can claim that overtime Tuning has developed into the largest and most influential initiative globally to reform higher education. At present nearly 130 countries have been / are involved in one or more of the Tuning projects. The initiative keeps enjoying prestige in all world regions due to the projects which were running in the years around 2013 and after. Operating in a worldwide context allowed not only for tailoring the approach to regional and national settings, but also to enrich continuously the models taking into account the findings of the different projects. These showed that besides difference - mainly related to different educational philosophies developed in Europe – there proved to be very much in common: the need to make degree programmes more relevant towards the needs to society and to take into account global technical and societal developments. Programmes proved to be very less culture bound than initially might have thought.

Noticing that the paradigm change towards outcome-based learning developed very slowly, forced to re-think the Tuning strategy to modernise higher education. This resulted in the CALOHEE projects, which as the original Tuning ones, proved to be agenda setting again not only for Europe but worldwide. Maybe the greatest contribution of Tuning has been during in particular the last decade in which it has been able to bring academics from different world regions together by offering a platform for cooperation and to give them a voice in the policy making theatre. The many Tuning initiatives have facilitated academics to discuss their experiences as teacher and develop strategies with peers to enhance the quality of learning and teaching. As a result, an informed group of thousand and more academics have been created who perceive(d) involvement in Tuning as a very helpful and positive experience.

Bibliography

ACE website: https://vrip.unmsm.edu.pe/aprendizaje-centrado-en-el-estudiante-de-america-del-sur/.

Altbach, Philip G. “AHELO: The Myth of Measurement and Comparability.” International Higher Education 82 (Fall 2015), 2-3. https://doi.org/10.6017/ihe.2015.82.8861.

American Historical Association website: https://www.historians.org/teaching-and-learning/tuning-the-history-discipline.

Pablo Beneitone, and Maria Yarosh, “Trans-regional generic-competences: The core of an internationalzed curriculum.” Research in Comparative and International Education 17, no. 3 (2022): 486-510. https://doi.org/10.1177/17454999221097026.

Birtwistle, Tim, Courtney Brown, and Robert Wagenaar, “A long way to go … A study on the implementation of the learning-outcomes based approach in the EU.” Tuning Journal for Higher Education 3, no. 2. (May 2016): 429-463.

BFUG, The Bologna Process Revisited: The future of the European Higher Education Area. Doc. Code: BFUG_LV_IS_43-4. Last modified 10.12.2014.

CALOHEA website. https://calohea.org.

CALOHEE Project website. https://www.calohee.eu.

DARE+ website. http://dareplus.eu/content/about-dare.

EMJMD Society, Politics, and Culture in a Global Context: Euroculture website: https://www.euroculturemaster.eu.

EMJMD in International Humanitarian Action (NOHA) website: https://www.nohanet.org.

eWBL website. https://www.ewbl-project.com/.

European Commission, ECTS Users’ Guide. European Credit Transfer and Accumulation System and the Diploma Supplement. Brussels, 14 August 2004.

European Commission, European Credit Transfer and Accumulation System. Key Features. Luxembourg: Office for Official Publications of the European Communities, 2004. ISBN 92-894- 4742-7.

European Commission et.al, The European Higher Education Area in 2012: Bologna process – Implementation report. Brussels, 2012.

European Commission/EACEA/Eurydice. The European Higher Education Area in 2015: Bologna Process Implementation Report. Luxembourg: Publications Office of the European Union, 2015.

European Students’ Union (ESU), Bologna With Student Eyes 2012. Brussels, 2012.

European Students’ Union (ESU), Bologna With Student Eyes 2015. Time to meet the expectations from 1999. Brussels, 2015.

International Tuning Academy website. https://tuningacademy.org/publications/.

Lokhoff, Jenneke, et al. Tuning Guide to Formulating Degree Programme Profiles. Including Programme Competences and Programme Learning Outcomes. Bilbao, Groningen, The Hague, 2010. https://tuningacademy.org/wp-content/uploads/2014/02/A-Guide-to-Formulating-DPP_EN.pdf.

OECD, Fostering and Assessing Students’ Creative and Critical Thinking Skills in Higher Education: https://www.oecd.org/education/ceri/fostering-assessing-students-creative-and-critical-thinking-skills-in-higher-education.htm.

OECD, Tuning-AHELO Conceptual Framework of Expected / Desired Learning Outcomes in Engineering. OECD Education Working Papers, no. 60, OECD Publishing, Paris 2011. https://www.oecd-ilibrary.org/education/a-tuning-ahelo-
conceptual-framework-of-expected-desired-learning-outcomes-in-engineering_
5kghtchn8mbn-en.

OECD, Tuning-AHELO Conceptual Framework of Expected and Desired Learning Outcomes in Economics. OECD Education Working Papers, no. 59, OECD Publishing, Paris, 2011. https://www.oecd-ilibrary.org/education/tuning-ahelo-conceptual-framework-of-expected-and-desired-learning-outcomes-in-economics_5kghtchwb3nn-en.

OECD website: https://www.oecd.org/education/skills-beyond-school/ahelo-main-study.htm.

Perusso, Andre and Robert Wagenaar , “Electronic work-based learning (eWBL): a framework for trainers in companies and higher education.” Studies in Higher Education, 8 Nov. 2023. https://doi.org/10.1080/03075079.2023.2280193

Perusso, André, and Robert Wagenaar, “The State of Work-based Learning Development in EU Higher Education: Learning from the WEXHE Project,” Studies in Higher Education 47, no.7 (2022). https://doi.org/10.1080/03075079.2021.1904233.

Project FORTH website. https://www.project-forth.org/.

START website. https://supportingstart.eu/#:~:text=START%20project%20aims%20to%20achieve,the%20communication%20between%20the%20higher.

Sursock, Andrée, Trends 2015: Learning and Teaching in European Universities. Brussels, European University Association, 2015.

Tuning Asia – South-East website. https://tuningasia-southeast.org.

Tuning India website. https://tuningindia.org.

Tuning Sectoral Qualifications Frameworks for Social Sciences. Final Report 2008 – 2010. Bilbao, 2010. https://tuningacademy.org/wp-content/uploads/2014/02/SQF_Social-Sciences_EN.pdf.

Tuning Sectoral Qualifications Frameworks for the Humanities and the Arts. Final Report 2010 – 2011. Bilbao, 2012. https://tuningacademy.org/wp-content/uploads/2014/02/SQF_HUMART_Final_Report_2010-2011.pdf.

Tuning T-Meda website. http://tuningmeda.org.

Van Damme, D., and D. Zahner, ed. Does Higher Education Teach Students to Think Critically?, OECD Publishing, Paris, 2022.  https://doi.org/10.1787/cc9fa6aa-en.

Wagenaar, Robert, A History of ECTS, 1989-2019. Developing a World Standard for Credit Transfer and Accumulation in Higher Education. Bilbao and Groningen: International Tuning Academy, 2019. ISBN 978-84-1325-042-7.

Wagenaar, Robert. “Evidencing Competence in a Challenging World. European Higher Education Initiatives to Define, Measure and Compare Learning.” In International Journal of Chinese Education (IJCE). SAGE. April 20, 2021. https://doi.org/10.1177/22125868211006928.

Wagenaar, R., Learning Outcomes a Fair Way to Measure Performance in Higher Education: the Tuning Approach. Outcomes of higher education: Quality relevance and impact. Paris: OECD, 8-10 September 2008. http://www.oecd.org/site/eduimhe08/41203784.pdf.

Wagenaar, Robert, Reform! TUNING the Modernisation Process of Higher Education in Europe. A Blueprint for Student-Centred Learning. Bilbao and Groningen, 2019. ISBN 978-84-1325-032-8.

Wagenaar, Robert, “Quality Efforts at the Disciplinary Level: Bologna’s Tuning process.” In Research Handbook on Quality, Performance and Accountability in Higher Education, edited by Ellen Hazelkorn, Ellen, Alexander McCormick, Andrew Gibson, and Hamish Coates, 275-289. Cheltenham: Edward Elgar Publishing Ltd., 2018.

Wagenaar, Robert, Arlene Gilpin and Pablo Beneitone, Tuning in China. An EU-China Feasibility Study into the Modernisation of Higher Education, Bilbao and Groningen, 2015. ISBN 978-84-15772-95-8.

Wagenaar, Robert. “The Myth of Power: Governing Reform in the Bologna Process of Higher Education.” In From Actors to Reforms in European (Higher) Education. Festschrift for Pavel Zgaga, edited by Manja Klemenčič, 45-63. Higher Education Dynamics Series. Series Editors: Peter Maassen and Johan Muller. Cham: Springer-Nature, 2022.

Wagenaar, Robert. “What Do We Know – What Should We Know? Measuring and Comparing Achievements of Learning in European Higher Education: Initiating the New CALOHEE Approach.” In Assessment of Learning Outcomes in Higher Education. Cross-National Comparisons and Perspectives, edited by Zlatkin-Troitschanskaia, Olga, Miriam Toepper, Hans Anand Pant, Corinna Lautenbach, and Christine Kuhn, 169-189. Cham: Springer International Publishing, 2018.

Yarosh, Maria, Anna Serbati, and Aidan Seery, ed. Developing Generic Competences Outside the University Classroom. Granada, 2016.

---