The e-portfolio in higher education: The case of a line of teaching innovation and complex change management

Alfredo Berbegal Vázquez, Abel Merino Orozco,
Ana Arraiz Pérez, and Fernando Sabirón Sierra[*]

doi: http://dx.doi.org/10.18543/tjhe-9(1)-2021pp29-64

Received: 27 April 2021
Accepted: 27 October 2021

Abstract: This work proposes a line of innovation to implement possible uses for the e-portfolio in higher education. We present an initial framework for analysis with attention to three main arguments: the validity of the interest of the e-portfolio for higher education in light of the current challenges posed by the knowledge society; the relevance of complex change management within organisations such as universities; and the identification of critical elements in the relevant literature concerning experiences similar to the case analysed here. The milestones for the line of innovation include six projects and three teacher training courses. Once data had been gathered in the respective assessment phases for each milestone by means of document analysis techniques (e-portfolios, teaching materials, usage statistics), questionnaires, discussion groups, in-depth interviews, and self-assessments (responsible academics, teachers, and students), a global analysis of the whole line was conducted from a complex approach to the problems of teaching change and innovation. A technological, political and cultural reading of innovation reveals emerging problems to reconsider: the attainment of deep learning; the standardisation of academic tutoring; the formation of learning and practice communities; the reconceptualisation of the e-portfolio as a personal learning environment; and the transformation of the university institution as a learning organisation. The premature condition of higher education to deal with the change in the teaching paradigm and the urgency to revisit its innovation policies to overcome it stand out among the critical conclusions of this study.

Keywords: e-portfolio; personal learning environment; teaching innovation; academic tutoring; change management.

I. Introduction

This article proposes a line of research for possible uses of the e-portfolio in higher education. We present a longitudinal case study on comprehensive and institution-wide implementation of an e-portfolio programme from 2011 to 2019. A review of the literature, analysing the line of teaching innovation as a horizon, forms the basis of the first part. The literature can provide us with a complex framework for current interest in teaching research and change management, especially concerning e-portfolio experiences at university. The main purpose of the article is discussed in the second part. We present three key branches of e-portfolio innovation developed in six projects: 1) exploration of learning-centred and skill-based assessment; 2) definition, optimisation and transfer of the Mahara e-portfolio solution; and 3) design of a policy to expand and disseminate the MaharaZar digital personal learning environment. For each branch, we apply analytical models of the revised framework, describe the principal projects involved, sum up partial results and highlight emerging problems and lessons learned. Finally, we end the article with a discussion on practical implications, limitations, future research and critical conclusions.

II. Literature review

Analysing the line of research first requires a theoretical review of three key questions: 1) Why innovate in higher education?; 2) How can innovation processes and change in complex organisations such as universities be undertaken?; and 3) What is the current situation at universities regarding e-portfolio innovation and use?

II.1. Why? Teaching innovation attractors in higher education

As argued in the systematic review by Cai,[1] universities promote better ways of doing the same things (sustaining innovation), although current knowledge society leads to radical changes in the traditional organisation of universities (disruptive innovation).[2] The objectives of innovation address the new preoccupations of the University-Market (employability of graduates, economic and territorial policies, links with companies, knowledge management). Although all academic fields are addressed, innovations in curriculum and in teaching-learning methods are more numerous: integration of educational technology,[3] adoption of student-centred learning,[4] authentic and formative assessment[5] and collaborative learning.[6] In the areas of research, management and governance, preferred changes suggest capitalising on universities’ knowledge and social responsibility and exploring other models of public management, organisational units and academic dynamics.[7] The literature distinguishes between outside-driven innovations (policies, incentives) and inside-driven innovations (professional initiatives);[8] it also analyses preferences and responsibilities in leadership, systematic planning, participative procedures and multi-directional communications. The institutionalisation stage is reached in the medium and long-term and requires longitudinal studies in both formal (structures, organisations, functions) and informal dimensions (values, attitudes, rules), each with their different rhythms and speeds. The success of institutionalisation depends on understanding the potential use of innovation, its compatibility with the sociocultural organisation in which it evolves and the agency or capacity to act of the stakeholders involved.[9]

In this respect, it is possible to define three main attractors in the field of teaching innovation required to adjust to the needs of modern societies: knowledge, language and activity.

A. Knowledge. University institutions have been buffeted by the transformations brought about by the knowledge society.[10] Higher education cannot remain indifferent to the decentralisation and dispersion of knowledge sources in the globalised world;[11] to the transformation of knowledge management services into knowledge economy services;[12] to the usefulness of new forms of knowledge, both performative and situated;[13] to the reconfiguration of interactive, portable and transferable knowledge; to the exponential growth of information and the vertiginous speed with which it becomes obsolete; to the substitution of expert dialogical validation for standardised assessment systems and flows;[14] and to the multi-systemic and continuous accountability used to guarantee institutional excellence.[15] The historical mission of higher education with respect to knowledge appears to have reached its end point, overcome by a crisis of identity in substantive (the loss of institutional status), ideological (lack of social legitimacy) and procedural (institutional vulnerability and increasing instrumentalisation) terms. However, the revision of the canonical knowledge of higher education and the fall of the monopoly it has traditionally held over knowledge management does not need to be read as the end of knowledge: its mission must be transformed in the era of supercomplexity. Universities have a duty to offer new references for understanding, to facilitate the interpretation and comprehension of emerging forms of knowledge and to facilitate means and resources for learning how to live in supercomplex societies.[16]

B. Language. The challenges of the digital university call for a conceptual restructuring of academia. Digitalisation comprises those cultural and information environments that condition the things we know and how we know them.[17] The ubiquity of the digital media radically resignifies the communication processes in post-industrial societies, thus generating profound tensions and forms of resistance within the organisational and professional cultures of university institutions.[18] Similarly, the exponential growth in social, informational, educational and training spaces relocates higher education on the continuum of lifelong learning. From the perspective of teaching-learning systems, educational technologies update the cognitivist and socio-constructivist focus from connectivism.[19] There is, indeed, a proliferation of alternative training designs to adjust to the new routines and needs of citizens (ad hoc, online programmes, in-service, blended, flipped classrooms, through learning communities, skill-based, MOOCs, etc.).[20] Training spaces are being substituted by user-centred digital environments, thus stimulating a complete restructuring of formal education and the integration of non-formal and informal education.

C. Activity. The ambiguity of the notion of skill generally invites either economicist–balancing human capital and productivity demands of the environment–or behaviourist and cognitivist acceptations–preferred in instructional designs. The most extensive, socio-constructivist, ethnomethodological acceptations of the activity theory[21] show the interdependence with respect to a context where activities are interpreted and assessed dynamically and with self-determination.[22] In the context of higher education, these acceptations entail the creation of opportunities for situated learning[23] that lend value to skills due their authenticity, their suitability and their relevance with respect to the contexts of professional coping and performance.[24] Translating training into a referential of skills represents a paradigmatic revolution[25] that addresses three key challenges: 1) the readjustment of academic-professional skills to confront inconsistent and supercomplex futures;[26] 2) the identification and appraisal of tacit knowledge and transversal life-skills;[27] and, 3) the harmonised recognition of skills in open, ubiquitous and transversal training spaces (virtual, cross-border, multi-agent campuses).

In light of the challenges of an epistemology of uncertainty, these attractors call for an exercise in social responsibility.[28] Professional identities, organisational and institutional cultures face ongoing dilemmas between responsible adaptation (third mission, triple helix) and perverse survival (industrial university, academic capitalism).[29] Innovation arises precisely in the middle of these internal contradictions of the University-Market, coexisting with other residual models such as the University-Academy and the University-State.

II.2. How? Complex change management

Some classical analyses that advance multidimensional interpretations of the processes of innovation[30] and the “perverse” nature of their planning[31] should also be considered.

In the first place, innovation problems do not respond solely to an instrumental ethic (means-ends) from regulatory rules (coordination) and expert leadership (competency) to address the interests of sponsors (governance, responsible academics). In contrast to this technological approach, the political perspective examines the commitments and conflicts of those involved that are resolved using a contractual ethic by means of rules of cooperation (competitiveness and negotiation) and focuses of influence (persuasion, stimulation, coercion). The cultural perspective acts through norms, values and beliefs of a community comprising different cultures and subcultures (conceptual antithesis, rivalry, complementarity, reciprocity) that relate through rules of cooperation (resistance, approval) and from a relativist ethic (a problematic agreement in itself).

In the second place, problems related to innovation cannot be domesticated. They are acquiescent and irreversible and they require ongoing and up-to-date solutions.[32] In contrast with the analytical model of innovation, which manages actions over a limited period of time and aims at achieving goals and solving problems of interest, innovation in open, social organisations requires interpretive models focused on processes and with broader time scales to redirect the solutions and their emerging effects.

These concerns have been analysed through the lens of complexity theories.[33] Complexity is an umbrella theory covering a diverse array of theories and approaches, with the common interest in the emergence of order in dynamic, non-linear,[34]^,[35] self-organising, socio-ecological[36] and adaptive complex systems.[37] Using these approaches, the nature of change management is cyclical, horizontal, divergent, democratic, disruptive, with indeterminate margins of tolerance and it acts as a catalyst for expansive changes, by reassessing questions such as the balanced distribution of power, a preferential focus on the stakeholders directly involved, strategies for lifelong learning and the commitment of the organisation to serving the community. Such considerations have consequences in the field of organisational development and, especially, in stances that consider organisations as learning systems. Learning organisations are the regulators of lifelong personal learning (reflective practice, adaptation of professional skills), of mental models (openings for change), of shared vision (individual and organisational harmonisation), of learning communities (social learning),[38] and systems thinking (pattern for change).[39] Likewise, these are sustained by forms of organisational learning such as exploitation of knowledge, problem solving, information exchange, forms of leadership and psychological and social processes of intuition, interpretation, integration and institutionalisation of changes.

When learning organisations fail to understand their own learning practices, a theoretical difficulty arises. In the analysis of the systems, the structures and the rules governing organisations–leaving out their practices and the agents that constitute them–there is a risk of subsuming individual and group capacity for action into an archetypal form of behaviour, a cognitive model or mechanism for transmitting the properties of the system. Learning practices are sociocultural practices that unite presupposed understanding, forms of practical knowledge, tacit knowledge and informal rules. Even from the pioneering work by Kurt Lewin, which explores the complexity of change through consultations and interventions by means of action research and training groups, the agency of the individuals involved is key to the analysis of generative changes.[40]^,[41]^,[42] The demand for change should also be questioned, which implies radicalising the agency of the members of the organisation and, therefore, prioritising the policy dimension. Disruptive changes (instituting) trigger critical resistance that reveals the arrangement of the cultural premises in terms of influence practices within a particular institutional configuration (instituted). The ensuing crisis acts as an institutional mirror to recognise oneself in–up until that point–latent beliefs, rules and values and, through this recognition, open a dialogical process for change institutionalisation (institutional analysis). Regarding sustaining innovations, certain micro-political practices allow for change and offer margins for action and opportunity conditions within the routines of organisations (internal analysis).[43] In any case, there is a strong transformational component acting on both group and individual dimensions of members in terms of self-determination (agency over the instituted and authorisation over the instituting).

Although the resulting theoretical frameworks differ, these approaches see change as a social practice and as an opportunity for learning that comprise an intimate interdependence between individual, group, organisational and environmental dimensions.

II.3. What? The e-portfolio: Catalyst for complex change in higher education

The e-portfolio is a tool that allows documenting and managing learning sources, resources, products and processes in the modern digital culture, in a deep and continuous way throughout the life cycle. Although this is perhaps a conciliatory definition, the reality is that the specialised literature experiences huge difficulties in specifying what an e-portfolio is and its applicability to higher education.[44]^,[45]^,[46] It represents a system of social learning with highly varying aim (reflective purposes), function and role (audiences), and structure and activity (artefacts)[47] and whose operational realisation, therefore, implies several practical dilemmas. Connectivism and hypertextuality grant the e-portfolio a specific character when compared to the traditional portfolio.[48]

Its first applications in a higher education setting took place in the 1980’s, in the context of initial teacher training. Towards the end of the 1990’s, a time with extensive academic output in the field of metacognition and educational technology, its use became widespread. Since 2000, the development of Web 2.0 and social media, the inconsistencies between learning and assessment arising in higher education plus the internationalisation of universities made the e-portfolio a more attractive solution.[49] However, although it represents a good alternative to respond to the attractors introduced, its application in higher education has been limited to theoretical research or to the development of digital platforms and environments. Moreover, the majority of this research was limited to defining the perceptions and attitudes of stakeholders during a relatively brief period of time rather than understanding and demonstrating the effective achievement of skills through comparative and longitudinal studies.[50]^,[51]^,[52] Judging by the literature reviewed, the current situation regarding the e-portfolio within the higher education setting is as follows:

•It is reduced to storing learning evidence, as a filing cabinet.[53] It is limited to acting as a form of submission that does not go beyond an isolated realisation of the task without exploring its inherent meaning nor connecting it with personal experience, prior knowledge or the real world. The digitalisation and adaptability of these tools can outshine their teaching potential.[54] On occasion, technology serves as the scapegoat for failed initiatives that are actually due to serious conceptual shortcomings related to the way in which learning portfolios have been implemented.[55]

•Deep learning is the exception. It is rare for content to be selected on the basis of audiences and purposes and processes of self-regulation and critical reflection are confused with descriptions or lists of completed tasks.[56]^,[57] The basic skills that allow documenting learning must be exercised explicitly and over relatively long periods of time.[58] The only way to transform these skills into thinking habits is over several academic years, taking on the threats to sustainability and initial resistance and insecurities.[59]^,[60] Consequently, there is a need for teacher and responsible academics training with a strategic focus on academic guidance and coordination.

•The e-portfolio is limited to academic life within universities and usually to specific subjects. Its use in later professional development and throughout life is not reached by trusting in the intrinsic motivation of graduates or in the inertia of a continuing and massive demand on the part of universities. It involves an online vision and a reconceptualisation of e-portfolios (focused on institutions, training services and users) as digital personal learning environment (focused on citizens and their professional career and personal lives);[61]^,[62]^,[63] evidence of this is the scarcity of e-portfolios by faculty members or those for university or professional guidance.

This diagnosis indicates that the potential of the e-portfolio has not been sufficiently explored, mainly due to a lack of knowledge of its possibilities of use by educational communities. In addition, evidence from research is somewhat hesitant and, as a consequence, the large-scale take-up of e-portfolios by universities is fraught with uncertainty.[64]^,[65]

Using its potential requires organisational learning in innovative collectives and complex changes within organisations as follows:

•a structural consensus concerning the paradigms of teaching-learning that demand the use of e-portfolios (shared visions);

•lifelong and individual learning by agents within the educational community through transfer of educational research and innovation in universities (professional development);[66]

•opening up towards deeper curriculum reforms, with new training modalities, harmonisation and interoperability with other universities and socioeconomic agents, standards of accreditation with respect to official audiences (mental model);[67]

•learning communities, professional learning and practice communities that share a transversal and coordinated vision of the e-portfolio, organised according to teaching activity spaces (social learning);[68] and

•political action at regional, national and international levels in order to advance towards personal learning spaces that are integrated between services, programmes and social agents that take part in the needs of citizens throughout their lives, thus building strong cohesion between higher education and university, professional and employment guidance (systems thinking with patterns for change).[69]

A comprehensive implementation of the e-portfolio seems to be a priority in higher education, thus bringing into question whether universities are providing the deep learning demands of the 21st Century and if this learning continues throughout life.

III. The case of a line of innovation in the e-portfolio

Taking place between 2011 and 2019 at the University of Zaragoza (Spain), the project comprised three key branches: 1) the development of learning-centred and skill-based assessment; 2) definition, optimisation and transfer of the Mahara e-portfolio solution; and 3) design of a policy for expansion and dissemination of the MaharaZar digital personal learning environment. The following figure gives a global view of the line.

What follows is a description of the nature, objectives, types, contexts and resources of the various milestones,[70] including the most relevant results and the emerging problems and learning lessons for each branch.

This is an evaluative research that attempts a comprehensive and institution-wide analysis of implementation of an e-portfolio programme. The information presented was provided by the research participants (students, staff, organisations, third parties or others), whose confidentiality has been adequately protected with descriptions and systematic interpretations. Any personal and professional reference is irrelevant. Data analysis, full results of the inquiry and critical conclusions focus exclusively on levels and structural dynamics of teaching innovation practices and institutional change management.

III.1. Branch 1. Development of learning-centred and skill-based assessment

This branch of the work appears in the six innovation projects and its adaptation to teaching contexts continues to the present time. It comprises three specific innovation projects in development between 2011- 2016.

III.1.1. Methodology

It comprises three specific innovation projects in development between 2011- 2016.

•First project (2011-2012). It aimed to tailor the skill-assessment portfolio for each of the subjects of the reference area of the promoting collective in the degrees in primary and early childhood education at the Faculty of Education (350 students and 5 lecturers). This adaptation process was analysed through the portfolios produced (evidence and good practices), guided self-assessment (perceptions and degree of self-regulation), discussion groups and in-depth interviews with students (explicit and shared needs) and revision tutoring and formative assessment (critical incidents). Areas related to pedagogy (tasks and resources, tutoring and mediation, authentic assessment), didactics (defining and communicating aims, artefacts and assessment criteria), technology (adaptation to the digital platform, Blackboard 9.1) and organisation (sustainability and teaching coordination) were analysed.

•Second project (2012-2013). It involved tailoring the skill-assessment portfolio by another academic collective–innovative and with background–for use in the undergraduate and master programmes at the Faculty of Business and Public Management in Huesca (300 students and 6 lecturers). Innovation was also maintained in the context of the innovation collective (400 students and 6 lecturers). There were both intragroup (portfolios produced, guided self-assessment, discussion groups and in-depth interviews) and intergroup (discussion groups of teaching teams) analyses. The dimensions examined included assessment sustainability and institutionalisation. The project culminated in the adoption of Mahara as digital solution and its installation in the university openly accessible to the whole educational community.

•Third project (2015-2016). It concerned opening up the e-portfolio assessment. Skill-assessment in other contexts–such as teacher training placements in external educational establishments and end-of-degree projects–was explored. It involved a modular, promotional, tutoring- and showcase-related vision of the e-portfolio. A total of 23 lecturers from 9 different departments and the coordinators of the degrees in primary and early childhood education participated in the project. The perceptions of faculty members were assessed through a questionnaire, with attention given to the type and degree of use of the e-portfolio, its potential to respond to academic demands, needs of initial and emerging training, and attitudes and skills promoted and needed for the integration of the regular teaching practice. Good practices (portfolios produced, derived training materials) were assessed and the perception of degree coordinators on the viability and continuation of the space and the possible strategic actions to promote regarding all the degrees were investigated.

III.1.2. Contexts, objectives, and resources

This branch of the work involves a massive transfer of critical knowledge generated in works prior to the line of innovation.[71]^,[72]^,[73] It attempts a progressive normalisation of the practices of learning-centred and skill-based assessment in specific and local teaching contexts. In this sense, it addresses the following generic objectives: continuous reorientation of the design and implementation of the skill-assessment portfolio for new scenarios; definition of interpretive keys for its consolidation; exploration of effective digital solutions; and opening up the portfolio to more understanding, stable and durable aims. Various types of innovation are developed: teaching models and supports, institutional digital alternatives and progressive regulation of aims and designs of tasks and activities. The first two projects take place in specific teaching contexts and focus on mainly pedagogical needs. At the time, the new curriculums of the degree courses had only been running for a short time and the culture of assessment was in a process of generalisation. The third project takes place when the discussion surrounding innovation in universities is reaching culmination. It aims to consolidate the change in the culture of assessment beyond the context of local innovation (departments and faculties). With respect to the resources generated, these can be categorised in three ways: pedagogical (spaces for innovation, instructional manuals and guides, reports, publications), technological (the Mahara digital solution and adaptations) and organisational (academic and teaching coordination).

III.1.3. Multidimensional results

•Technological dimension. Two critical questions stand out: assessment sustainability and portfolio digitalisation. On the one hand, an analysis model is generated that defines interpretive keys to improve sustainability. Considerations on design indicate a clear definition of the portfolio’s aim (priority core and professional skills) and an appropriate construction of tasks to produce significant, authentic, scaffold, modular learning with no dispersions regarding teaching interactions. Academic tutoring reveals itself as the central branch for assessment, enabling the understanding of the aim and meaning of tasks and creating opportunities to develop the core skills necessary to document learning (critical reflection, communication, self-regulation, collaboration). Academic guidance and tutoring generate the necessary support structures, considering content assimilation and consolidation times, with-out ever reaching a corrective nature, and offering opportunities for peer learning. On the other hand, three preferred criteria for digital solutions are defined: institutional control, communicative potential and personalisation. The aim is creating a space linked to the user, not to the institution, allowing the exploration of several purposes, audiences and artefacts with the prospect of building lifelong learning. Finally, Mahara is established as solution and, a formal and reasoned proposal is made for its pilot installation at the University of Zaragoza.

•Political dimension. The line is progressively entwined in a rhetoric of innovation. The results translate into arguments to persuade responsible academics of the need for an institutional digital medium and of its potential for promoting the change of the teaching-learning paradigm and the progressive adoption of the new assessment culture. Micro-policy change in teaching contexts defines resistance and obstacles that are not strictly pedagogical in nature and more connected with structural questions (degrees, departments, faculties). The agency of subjects and degree coordinators to establish synergies between local assessment experiences and to facilitate the adoption of innovations through training modules over more extended time cycles is emphasised.

•Cultural dimension. It is clear that this innovation has a strongly disruptive character. In this case, it is a matter of resistance and obstacles associated with students’ assessment cultures of origin and old teaching cultures–especially intense during the first years of the line of innovation. Despite being immersed in incentivising policies for teaching innovation, reaching the deeper meanings of paradigmatic change produces sets of conceptual antitheses within most collectives. There are clashes between beliefs and values associated with learning and assessment and also between professional development and concomitant identities. In the case of the teaching activity, the new mediating identities found in the literature (informer, coordinator, encourager, facilitator, model) are established. The first signs of the need for a global institutional stance arise from these cultural questions, incorporating the paradigmatic change as the strategic branch in the medium and long-term. In contrast, critical questions such as assessment sustainability are difficult to overcome.

III.1.4. Emerging problems and learning lessons

Three key issues are defined (see figure 1): deep learning, learning and/or practice communities and academic tutoring.[74] The difficulties faced by faculty members in accommodating continuous change in their teaching performance and those by students in attaining deep learning must be addressed. In this way, the space for innovation becomes an opportunity context for mutual learning in consolidated teaching teams and those for mentoring new teaching staff. Academic tutoring is indispensable for regulating the process and, in particular, for exercising the cross-sectional skills that allow students to document learning in their e-portfolios.

III.2. Branch 2. Shaping, optimising and transferring the Mahara e-portfolio Solution

III.2.1. Methodology

This branch has been in development since 2013 up to the present through three specific projects.

•First project (2013-2014). It was the onset of an iterative process in uses and practices of the Mahara digital resource. Prior to its implementation there was a period of wide-ranging exploration in order to understand how Mahara was used at other universities (Castellón, Barcelona, Bath, Glasgow, Alberta). The specific objective adapted the skill-assessment portfolio to the new space in three subjects linked to the innovation collective (100 students and 5 lecturers). The interface, the resulting e-portfolios and possible examples of “good practices” were analysed; discussion groups and in-depth interviews with students were also conducted. Working groups of students and lecturers were also convened within the digital environment itself. The dimensions assessed of the digital solution included stability (continuity of the environment during and after university studies), profitability (cost and synergies with other institutional platforms), versatility (formats and languages used in representing evidence and mediations), pedagogical understanding (adaptation with respect to prior portfolio experiences), sustainability (global workloads) and institutionalisation (change of assessment culture in the medium and long-term). Strategic actions for transfer to other subject areas were identified, so it became necessary to develop generic frameworks for support and guidance for future experiences.

•Second Project (2014-2015). It took place within the MaharaZar space, as the environment is termed within the University of Zaragoza. Pilot experiences continued in the innovation context (120 students and 5 lecturers), consolidating pedagogical adaptations and achieving technical stabilisation of the digital resource. Its use was also integrated into the development of other innovation projects concerning mentoring for new teachers and academic tutoring; in addition, some isolated cases of tutoring for teaching placements at external centres and supervision of end-of-degree projects were initiated. The reconceptualisation of the e-portfolio as a personal digital learning environment, which is key for the institutionalisation stage, appeared for the first time in this project (third branch of work).

•Third project (2015-2016). It contributed to opening up the assessment purpose, by transferring its use to other groups and exploring new areas within the faculty as an innovation unit. A transfer phase to common and transversal tasks and subjects in degrees in primary and early childhood education was outlined from the conceptualisation of personal learning environment. The digital space could confer coherence and a modular structure to what was originally fragmented as subjects from different academic years, thus taking the first steps towards extensive acceptance of the e-portfolio as a way of documenting learning in the form of an academic-professional biography line. The experience allowed the evaluation of the technological solution from the viewpoint of teaching staff outside the line of innovation.

III.2.2. Contexts, objectives, and resources

This branch of work introduces an assessment nature of the technological solution reached as an institutional resource: MaharaZar. The initial premise always assumes that digitalisation, despite its ability to transform processes of representation, integration and mediation of artefacts, must be understood as a facilitating means, without lessening the governing pedagogic sense. The central objective was to evaluate the new technological solution with respect to the already existing critical knowledge regarding the skill-assessment portfolio. Nevertheless, other more overarching ideas derived from the stability and the ubiquity of a user-focused space emerge. These acquired a strong boost in this period within the field of educational technology (personal learning environment, alternatives such as flipped or hybrid training environments). The innovation milestones take place within the reference training contexts, since they concern pilot experiences, although there will be an impulse towards a shift to the faculty as the unit for innovation with common, interdepartmental aims. Support guides and tutorials for students and lecturers are produced and included in the common dynamics of subjects and in work groups within the digital space. Self-regulation and formative assessment (direct and personalised feedback) and peer learning (social networks, forums and work groups) processes are specially explored.

III.2.3. Multidimensional results

•Technological dimension. Most noteworthy are the stability and profitability of the MaharaZar resource. It enables a range of purposes over long timescales, is easy to install and free. However, its stability can be upgraded for speed and storage and there could be better use of the economy of Moodle-Mahara resources. Although Mahara is highly versatile (social media, hyperlinks and hyper-texts), the potential for personalisation is reduced due to the rigid counter-intuitive interface in terms of navigation and construction of the e-portfolio structure. Features such as institutional control (centralised security and training), potential for communication (academic social network) and for hybrid academic practices (feedback on learning) prevail. Regarding the results from the experiences, some minimal achievements were reached in the production of e-portfolios (efficiency in obtaining the assessment sense) and in autonomy in their digital and organised submission (effective assessment management). Although the sense of authorship is motivating, few users take advantage of the versatility of artefact use in the showcase portfolios (20%) or in profiles and walls (5%). Pedagogical understanding is barely developed: resistance and obstacles for students that were common and previously associated with the traditional portfolio now associated with technology (scapegoat); the majority of task submissions are partial and integrated documented learning is uncommon; the exploitation of collaborative learning potential is fairly limited. These issues are revealing for the initiation of new teachers that are not trained in the use of the e-portfolio, which suggests the need for intense and ongoing initial training. With respect to sustainability, digitalisation improves assessment management by articulating evidence in terms of preselected presentations and defences: effectiveness (time and workload) and efficiency (credibility and consistency in decision-making). However, this is affected by teachers’ efforts and mediation times to maintain the pedagogic sense during implementation.

•Political dimension. The rhetoric of innovation is intensified. The pilot experiences highlight the teaching and organisational needs and define the demand for a digital solution to university governance. The cross-sectional coordination of degrees is confirmed, especially to develop the full potential of MaharaZar as a personal learning environment. Without disrupting the administrative management of study programmes, the user-centred environment reverts to a global and modular resignification of the curriculum, which confers pedagogical continuity to the administrative discontinuity of credits and subjects. This potential shows the great distance between the real and the possible but, above all, the overriding political nature of the change from a different mental model. In the era of an innovation conditioned by policies of accountability for universities, initiatives compete within a framework of conflicting interests. The adoption of innovation requires relationships of persuasion and incentivisation within the university complex that should be mostly financial and meritocratic, and whose leadership involves the distribution of power within the organisation.

•Cultural dimension. The adoption of innovation is resolved through the social and professional relationships that constitute the synergies and collaborations within the cultures of research groups, teaching teams and departments. They are obvious in areas of knowledge and research groups in the case of pilot experiences and in interdepartmental and academic relationships when faculties are considered as innovation units. This affiliation is more likely if innovation provides a solution to explicit teaching issues requested from teachers’ onset premises and not when, as is the case of the e-portfolio, pedagogic references are questioned and new needs that require profound changes in professional world-view are generated. It is clear that the acculturation of innovation needs learning and practice communities amongst groups with heightened awareness, but driven forwards by institutional teacher training and students’ initiation structures.

III.2.4. Emerging problems and learning lessons

Those arising from the first branch remain (deep learning, academic tutoring and learning communities). From the perspective of deep learning, digitalisation alters language, so new artefacts resignify the authorship and personalisation of showcase portfolios, thus discovering a new self-reflection and professional dialogical purposes. The processes of teaching mediation are enriched and redefined, thus confirming their value. However, dilemmas concerning directivity and modelling and the difficulty in confronting them are identified.[75]^,[76] These issues are the inspiration for the first experiences with the university guidance portfolio, and the potential to develop programmes for university guidance from critical incidents and transitions in the academic-professional careers of students is inferred.[77]

Two new problems are defined (see fig. 1): the personal learning environment and learning organisation. On one hand, the technological solution, open to integrate academic activities in terms of life-long learning, clearly surpasses the e-portfolio concept. On the other hand, attention must be drawn to how the university institution is unable to reach a definitive understanding to implement the learning environment, since it is not sufficiently prepared to undertake the necessary strategic commitments. Change complexity points to the need for new institutional beliefs and attitudes to be adopted by the university organisation.

III.3. Branch 3. Designing a policy for expansion and communication of the MaharaZar digital personal learning environment

III.3.1. Methodology

This is developed through four projects starting in 2013 and up to the present.

•First project (2013-2014). It concerned the transfer into other contexts of the innovation collective. It was the onset of the first phase of a policy to expand the personal learning environment. The relevance of its potential was emphasised, not only to address the needs already identified within specific groups of lecturers but also to offer alternatives to change and improve a diversified training, academic and professional reality in higher education.

•Second project (2014-2015). It worked on disseminating the institutional resource throughout the university community. The focus was a training programme for university teaching staff within their campuses and faculties. A total of 47 lecturers from a diverse range of fields and specialisms took part in this experience. The quality and impact of the dissemination process was evaluated through questionnaires, training materials (student and teacher manuals, results from workshops, good practices, etc) and statistical reports on the use of the resource.

•Third project (2015-2016). Already mentioned within the other branches, it took the faculty as the innovation unit, which meant a key milestone in the institutionalisation phase. The experience allowed for the evaluation of more relevant and effective paths to greater visibility of the resource; the validity of strategies and opportunities already explored in other areas of learning and other departments; and the reactions, perceptions, needs and attitudes of the teaching staff in their first contact with the environment. The results of this experience were presented to the responsible academics of the faculty and the university in order to refine an expansion plan for the resource.

•Fourth project (2017-2018). It proposed to assess the MaharaZar environment in the university during the first five years after its installation. The assessment design involved the use of surveys and in-depth interviews with teaching staff (61 lecturers in 13 departments) and the analysis of the documents produced during the experience (teaching folders, self-reports, and narrative accounts). There was a descriptive phase (who uses it, for what reason and how it is used) and an interpretive phase (meaning attributed to working dynamics and the processes and products of the learning achieved). These collectives recovered assessment with the aim of consolidating learning communities.

III.3.2. Contexts, objectives, and resources

This branch is concerned with the institutionalisation phase, which focuses on the teaching staff. The possibility of directing dissemination and training towards the students was considered, but its autonomous adoption was not viable without a prior promotion by the teaching staff of specific pedagogical conditions. The experience assessed the degree of use (perceived benefits), compatibility (adaptability in the recipient context) and agency (entrepreneurial and support actions). The objectives evolved from dissemination (pilot experiences in collectives and faculties), to training (learning and practice communities, teacher training programmes) and to the design of an expansion policy (project for a communication plan). With respect to the context, the innovation units experienced a micro-macro shift, from immediate teaching contexts, to the faculty as innovation unit and, finally the whole university community. Resources produced included mailing lists (98 lecturers), guides and support materials for dissemination and training (work groups and courses), institutional portal and an executive project for a 5-year communication plan.

III.3.3. Multidimensional results

•Technological dimension. Although the projects reveal new requirements for institutionalisation, in general terms, they confirm the results from related international scientific production. Despite the portfolio was considered as part of the great majority of degree assessment processes, the use of the environment in the university is low, inefficient, and inappropriate. The resource is little known and, once known, there are difficulties in achieving full understanding of its potential of use. Therefore, it can be stated that institutional interest is based on its profitability, widening the breadth of available resources without any cost to the university.

•Political dimension. Although MaharaZar is offered as part of the teacher training programme since 2017, the environment has a disruptive nature that is not taken care of nor redirected. It is apparent that the integration of the personal learning environment implies a curriculum revolution in and a high level of institutional commitment. The steps towards dissemination that have been taken through the innovation milestones are defined within a non-reformed institutional vision, achieving, at best, an exchange of experiences between innovation collectives that share certain professional sensitivities. This reality can be verified both by innovation groups and by centres and units responsible for the management of innovation incentive programmes. The agency of responsible academics is not about resolution but about compensation: either the interests of teaching and curricular innovation are not sufficiently important, or there is not sufficient vision and foresight to distinguish the relevance of these changes with respect to other pressing concerns.

•Cultural dimension. The complexity of the acculturation process with regards to these innovations is obvious. Clearly, there is an incompatibility between the multidimensional character of innovation and university culture, its centres, groups and professional identities of its teaching staff. The milestones identify the need for realistic actions up and down the spectrum. Actions at the top of the spectrum would be directed at structures (services and programmes, institutional circles and protocols), while actions further down the spectrum would consider structures as constituted from specific social practices; the former prioritise policies of accountability and the latter address whether effectiveness and continuity are compromised if learning communities continue to be promoted via self-organisation within the innovation collectives and local networks.

III.3.4. Emerging problems and learning lessons

This policy of teaching innovation is revealed as ineffective. Attempts at institutionalisation show that the usefulness of innovation is devalued when the way problems are formulated does not evolve, thus weakening its rhetoric to promote change. Although innovation has been strategically adopted in order to transform the university model, paradoxically it was not done to boost and consolidate change in the teaching paradigm.

IV. Conclusions

IV.1. Practical implications

The analysis results define criteria for an appropriate use of the e-portfolio in higher education as follows:

•Adapted sustainability. This requires a viable e-portfolio design that prioritises academic and professional skills and that has significant and authentic tasks for every academic context, which are adapted to the students’ initial level and to the requirements of developing the required transversal skills (critical reflection, communication, self-regulation and collaboration). Academic tutoring makes a substantial contribution to this sustainability, which is presented as an unavoidable area of action in current university teaching culture.

•Oriented digitalisation. This appeals to the possibilities of personalisation and communication that technological solutions can provide and requires support for gradual and increasingly complex immersions (user guides, examples, models, etc.). From a learning standpoint, these possibilities consolidate deep learning (metaphor, analogy, irony, simulation, comparison, transfer and design) and a significant connectivity with activity contexts that are as authentic as possible.

•Transversal nature and ongoing continuity. This concerns the e-portfolio as a personal learning environment during higher education and throughout life. Digital solutions should offer institutionally controlled personal spaces and ensure skill validation and accreditation throughout students’ time at university (degrees, transitions, mobility, non-formal and informal education and university orientation). It also appeals to a culture of coordination and collaboration between teaching staff and responsible academics, forming a personal environment as a common and shared scenario of educational construction and reconstruction.

•Strategic institutional change. It requires a university roadmap outlining gradually more integrated e-portfolio uses. A framework should be adopted for the various university levels: subject assessment, subject module assessment, course assessment, degree assessment, show-case portfolio and, from a transversal perspective, university orientation assessment. On an operational level, we need to consider how partial use results in every educational structure recover in more inclusive structures over time.

These practical implications would form a shared vision and become one of the critical conclusions detailed below.

IV.2. Limitations

For a better and more exhaustive exploration of e-portfolio uses and of personal learning spaces at the university, limitations arise from the current status of the institution. Lack of knowledge of the e-portfolio’s potential by most of the university community stems from hardly any or inappropriate use of it. Consequently, studies are restricted to highly aware minority collectives with extremely specific purposes. This makes it hard to compare with other teaching scenarios and to research other purposes and target audiences that facilitate lifelong learning in higher education. The literature review of the projects in the line of research proves that this situation occurs in all Spanish universities.

IV.3. Future research

Further research would cover these areas:

•Exploring proposals, target audiences and artefacts. The aim is to expand on and take advantage of the potential of the e-portfolio through case studies and new innovative teaching experiences in other scenarios and collectives to meet higher education’s many and diverse requirements.

•Impact on developing academic and professional skills and on consolidating deep learning. Conducting longitudinal studies is unavoidable to assess the quality of the education in terms of skills and achieving authentic learning in emerging personal and social situations.

•Usefulness for lifelong continuing professional education and development. Essential and theoretical research seems necessary to ensure effective transfer into university academic policy and, therefore, that e-portfolio uses can contribute to the stable development of new educational methods in a knowledge society context (blended, online, alternance and multi-agent training).

•Strategy to improve the communication plan and the expansion policy plan. The research results need to yield applied and operational models that can provide the university community with educational information on how the e-portfolio can be used. Technological solutions must also be adapted coherently, and opportunity management is necessary for gradual institutionalisation. Evaluative research to understand explicit needs perceived by the university community would lead to more effective and suitable institutional responses.

These proposals would have to be integrated in specialised institutional structures that can be transferred for professional development in all educational communities.

IV.4. Critical conclusions

Considering universities as learning organisations, the following critical issues are presented:

•Mental model. The lessons learned transcend individual experiences and point to the institutional rationality that supports and tolerates them. The resistance and obstacles detected while implementing the e-portfolio and the challenges involved in it being adopted across the university community can be interpreted as symptoms of greater resistance; that is, the resistance of universities to constitute themselves as learning organisations. The supports required to optimise sustainability and institutionalisation of teaching practices are generally scarce and limited. In the complex management of organisational change, abandoning bureaucratic and linear models of teaching innovation to adopt specific professional models seems appropriate. This conclusion, viewed as trivial in other areas of innovation, reports an institutional immaturity within the field of university teaching innovation and it affects any change or improvement in teaching systems and training modalities in the digital university.

•Shared visions. This involves implementing an activity theory in personal learning environments, in specific teaching contexts and, from an accumulative perspective, on other more major educational units. Universities need to evolve in the four practical implications highlighted above and to promote actions, such as the following: students (developing transversal skills based on initial training and throughout degrees, academic tutoring and university orientation with responses integrated into building a career plan), teaching staff (lifelong and initial training programmes on socio-constructivist principles, situated learning and authentic assessment, and professionally oriented models for university academic tutoring) and coordinators and responsible academics (strategic programmes and plans for communication and expansion phases and teaching coordination).

•Social learning. This would be standardised through learning communities and communities of professional practice and learning among teachers, technical staff and responsible academics. Consequently, difficulties and solutions would be shared, and repositories established for resources and materials, lessons learned and frequently asked questions in the format of discussion fora and didactic guides for various collectives.

•Professional development. This cannot be left to goodwill or be reduced to exceptional spaces where collectives exchange practices. It requires specific institutional structures that address the abovementioned prospective lines of research and that convey educational research and innovation results in the university. Insurmountable ethical-professional dilemmas are inferred, highlighting the precarious nature of university infrastructure for teaching and guidance. Guidance and multimedia services, centres of educational research and technological innovation are essential for universities’ political agendas so they can map out training and professional situations, design and implement integrated programmes, build specific learning communities and establish themselves as interoperative networks with other agents so they can be co-responsible for the quality of university training and lifelong learning. One important mission of these centres would be issuing guidelines to attain a paradigm shift that would enable deeper and more enduring learning.

•System thinking. The above transformations require incentives that stem from outside the university sphere, in other words regional, national and international policies promoting a university social training and orientation model. This involves consolidating change patterns such as the following: creating an integrated professional orientation system, ensuring quality lifelong continuing education, reviewing emerging academic and professional skill models, alliances among higher education, other educational levels and other social agents in the area, etc. This system thinking would be based on three aspects: users (person-focused responses), skills (qualification and transversality), educational and orientation services (integrated and interoperative) and professional services (expert knowledge, programme and service coordination).

It is clear that the e-portfolio is an analyser of disruptive, pedagogical, technological and institutional changes.

IV.5. Specific policies

Finally, the above-mentioned practical implications and critical conclusions will be implemented by adopting specific policies at several levels:

1.Institutional policies. University strategic plans will establish a system of incentives to promote continuous quality towards teaching excellence and collaboration with their educational units to evaluate research and innovation. The aim is to select teaching innovation lines and programmes that underpin and consolidate experiences with the e-portfolios and personal learning spaces.

2.Academic policies. They will boost long-lasting intensive and professional theoretical models on the use of e-portfolios and personal learning spaces, thus validating content, resources and lessons learnt from teacher training programmes and from university managers in the three prioritised focal points: development of competences, academic tutoring and mentoring, and institutional coordination and communication.

3.Teaching policies. They will promote the leadership of reference innovation groups in building professional communities by launching a transfer and normalisation plan of e-portfolios and personal learning spaces in centres and faculties through a nested logic, in other words, structured in the different intervention spheres at medium and long term (courses, module, degree and career development).

The ongoing standardisation of personal learning spaces and of the use of e-portfolios in their several forms has huge potential to align with the model of lifelong learning and lifelong orientation in our universities and to help meet the major challenges of the future.

Bibliography

Ardoino, Jacques, Patrick Boumard, and Jean-Claude Salaberry. Actualité de la théorie de l’institution: hommage à René Lourau. Paris: L’Harmattan, 2003.

Arraiz, Ana, and Fernando Sabirón. The ethnographic portfolio: a socio-constructivist device for the recognition of learning.” [In French.] Carriérologie 12, no. 3-4 (2012): 319-335.

Arraiz, Ana, Alfredo Berbegal, and Fernando Sabirón. Academic tutoring focused on assessment: analysis of needs from the perspective of students and teachers. [In Spanish.] REDU 16, no. 2 (2018): 211-229. https://doi.org/10.4995/redu.2018.5992.

Ashford-Rowe, Kevin, Janice Herrington, and Christine Brown. “Establishing the critical elements that determine authentic assessment.” Assessment & Evaluation in Higher Education 39, no. 2 (2014): 205-222. https://doi.org/10.1080/02602938.2013.819566.

Baregheh, Anahita, Jennifer Rowley, and Sally Sambrook. “Towards a multidisciplinary definition of innovation.” Management Decision 47, no. 8 (2009): 1323–1339. https://doi.org/10.1108/00251740910984578.

Barnett, Ronald. “University knowledge in an age of supercomplexity.” Higher Education 40, no. 4 (2000): 409–422. https://doi.org/10.1023/A:1004159513741.

Barrett, Helen C., and Nathan Garrett. “Online personal learning environments: Structuring electronic portfolios for lifelong and life wide learning.” On the Horizon 17 no. 2 (2009): 142–152. https://doi.org/10.1108/10748120910965511.

Bell, Vikki. “Performative Knowledge.” Theory, Culture & Society 23, no. 2-3 (2006): 214-217. https://doi.org/10.1177/026327640602300245.

Berbegal, Alfredo, Ana Arraiz, Fernando Sabirón, and Carolina Falcón. The MaharaZar Portfolio at the University of Zaragoza: Emerging Reflexions. [In Spanish.] In Good Practices in ICT-supported University Teaching: Experiences in 2014, edited by José-Luis Alejandre, 135-144. Zaragoza: PUZ, 2015.

Blaj-Ward, Lai, and Kim Winter. “Engaging students as digital citizens.” Higher Education Research & Development 38, no. 5 (2019): 879-892. https://doi.org/10.1080/07294360.2019.1607829.

Brady, Malcolm, and Naoimh O’Reilly. “Learning management systems and their impact on academic work.” Technology, Pedagogy and Education 29, no. 3 (2020): 251-268. https://doi.org/10.1080/1475939X.2020.1743746.

Bratianu, Constantin. “Organizational learning and learning organization.” In Organizational Knowledge Dynamics: Managing Knowledge Creation, Acquisition, Sharing and Transformation, edited by Constantin Bratinu, 286-312. Hershey: IGI Global, 2015.

Bruijn-Smolders, de Monique, Caroline F. Timmers, Jason C.L. Gawke, Wouter Schoonman, and Maris Ph. Born. “Effective self-regulatory processes in higher education: research findings and future directions. A systematic review.” Studies in Higher Education 41, no. 1 (2016): 139-158. https://doi.org/10.1080/03075079.2014.915302.

Bunney, Dian, Elaine Sharplin, and Christine Howitt. “Generic skills for graduate accountants: the bigger picture, a social and economic imperative in the new knowledge economy.” Higher Education Research & Development 34, no. 2 (2014): 256-269. https://doi.org/10.1080/07294360.2014.956700.

Burnes, Bernard. “Kurt Lewin and complexity theories: back to the future?” Journal of Change Management 4, no. 4 (2004): 309-325. https://doi.org/10.1080/1469701042000303811.

Cai, Yuzhuo. “From an Analytical Framework for Understanding the Innovation Process in Higher Education to an Emerging Research Field of Innovations in Higher Education.” The Review of Higher Education 40, no. 4 (2017): 585-616. https://doi.org/10.1353/rhe.2017.0023.

Carless, David. “Exploring learning-oriented assessment processes.” Higher Education 69 (2015): 963–976. https://doi.org/10.1007/s10734-014-9816-z.

Cumming, Jim. “Contextualised performance: reframing the skills debate in research education.” Studies in Higher Education 35, no. 4 (2010): 405-419. https://doi.org/10.1080/03075070903082342.

Dakka, Fadia. “Competition, innovation and diversity in higher education: dominant discourses, paradoxes and resistance.” British Journal of Sociology of Education 41, no. 1 (2020): 80-94. https://doi.org/10.1080/01425692.2019.1668747.

Del Canto, Francisco. “University As a Global Actor in the International System of the 21st Century.” Tuning Journal for Higher Education 6, no. 1 (2018): 169-98. https://doi.org/10.18543/tjhe-6(1)-2018pp169-198.

Driessen, Erik. “Do Portfolios have a future?” Advances in Health Sciences Education 22 (2017): 221–228. https://doi.org/10.1007/s10459-016-9679-4.

Engeström, Yrjö. “Activity theory and individual and social transformation.” In Perspectives on Activity Theory. Learning in Doing: Social, Cognitive and Computational Perspectives, edited by Yrjö Engeström, Reijo Miettinen, and Raija-Leena Punamäki-Gitai, 19-38. Cambridge: Cambridge University Press, 1999.

ETNOEDU. The ethnographic portfolio: a tool of assessment skills. [In Spanish.] Zaragoza: PUZ, 2007.

Falcón, Carolina, and Ana Arraiz. Efficient and sustainable career construction: the professional portfolio as a resource at university guidance. [In Spanish.] REOP 28, no. 2 (2017): 8 – 29. http://revistas.uned.es/index.php/reop/article/download/20116/16665.

Flood, Robert Louis. “The Relationship of Systems Thinking to Action Research.” Systemic Practice and Action Research 23 (2010): 269–284. https://doi.org/
10.1007/s11213-010-9169-1.

Friesen, Norm. “Media: Digital, Ecological and Epistemological.” E-Learning and Digital Media 8, no. 3 (2011): 175-180. https://doi.org/10.2304/elea.2011.8.3.175.

Inken Gast, Kim Schildkamp, and Jan T. van der Veen. “Team-Based Professional Development Interventions in Higher Education: A Systematic Review.” Review of Educational Research 87, no. 4 (2017): 736–767. https://doi.org/10.3102/
0034654317704306.

Gerritsen-van Leeuwenkamp, Karin J., Desirée Joosten-ten Brinke, and Liesbeth Kesterd. “Assessment quality in tertiary education: An integrative literature review.” Studies in Educational Evaluation 55 (2017): 94–116. https://doi.org/10.1016/j.stueduc.2017.08.001.

Gibson, David C. “ePortfolio decisions and dilemmas.” In Handbook of research in eportfolio, edited by Ali Jafari, and Catherine Kaufman, 135-145. London: Idea Group Reference, 2006.

Greeno, James G. “The Situativity of Knowing, Learning, and Research.” American Psychologist 53, no. 1 (1998): 5-26. https://doi.org/10.1037/0003-066X.53.1.5.

Gulden, Manarbek, Kondybayeva Saltanat, Doszhan Raigul, Turarov Dauren, and Abylay Assel. “Quality management of higher education: Innovation approach from perspectives of institutionalism. An exploratory literature review.” Cogent Business & Management 7, no. 1 (2020). https://doi.org/10.1080/23311975.2020.1749217.

Hallam, Gillian C., and Tracy Creagh. “ePortfolio use by university students in Australia: a review of the Australian ePortfolio Project.” Higher Education Research & Development 29, no. 2 (2010): 179-193. https://doi.org/10.1080/
07294360903510582.

Harper, David A., Félix-Fernando Muñoz, and Francisco J. Vázquez. “Innovation in online higher-education services: building complex systems.” Economics of Innovation and New Technology (2020). https://doi.org/10.1080/10438599.2020.1716508.

Hasanefendic, Sandra, Julie M. Birkholz, Hugo Horta, and Peter van der Sijde. “Individuals in action: bringing about innovation in higher education.” European Journal of Higher Education 7, no.2 (2017): 101-119, https://doi.org/10.1080/21568235.2017.1296367.

Hill, Phil. “Online educational delivery models: A descriptive view.” Educause Review (2012): 85–97. https://er.educause.edu/-/media/files/article-downloads/erm1263.pdf.

Holman, David. “A dialogical approach to skill and skilled activity.” Human Relations 53, no. 7 (2000): 957-980. https://doi.org/10.1177/0018726700537003.

Hora, Matthew T., Jana Bouwma-Gearhart, and Hyoung Joon Park. “Data driven decision-making in the era of accountability: Fostering faculty data cultures for learning.” The Review of Higher Education 40, no. 3 (2017): 391-426. https://doi.org/10.1353/rhe.2017.0013.

House, Ernest R. “Technology versus craft: a ten-year perspective on innovation.” Journal of Curriculum Studies 11, no. 1 (1979): 1-15. https://doi.org/10.1080/
0022027790110102.

JISC. Effective Practice with ePortfolios: Supporting 21st century learning. Bristol: HEFCE/JISC, 2008.

Kauko, Jaakko. “Complexity in higher education politics: bifurcations, choices and irreversibility.” Studies in Higher Education 39, no. 9 (2014): 1683-1699. https://doi.org/10.1080/03075079.2013.801435.

Lehmann, Erik E., Michele Meoli, Stefano Paleari, and Sarah A. E. Stockinger. “The role of higher education for the development of entrepreneurial ecosystems.” European Journal of Higher Education 10, no. 1 (2020): 1-9. https://doi.org/10.1080/21568235.2020.1718924.

Lowenthal, Patrick, John White, and Karen Cooley. “Remake/Remodel: Using ePortfolios and a System of Gates to Improve Student Assessment and Program Evaluation.” International Journal of ePortfolio 1, no. 1 (2011): 61-70. http://www.theijep.com/pdf/IJEP37.pdf.

MacIntosh, Robert, and Donald MacLean. “Conditioned emergence: researching change and changing research.” International Journal of Operations and Production Management 21, no. 10 (2001): 1343-57. https://doi.org/10.1108/EUM0000000005973.

Meeus, Will, Linda Van Looy, and Peter Van Petegem. “Portfolio in higher education: Time for a clarificatory framework.” International Journal of Teaching and Learning in Higher Education 17, no. 2 (2016): 127-35. https://www.isetl.org/ijtlhe/pdf/IJTLHE27.pdf.

Merino, Abel, Alfredo Berbegal, Fernando Sabirón, and Ana Arraiz. Academic tutoring at the university: a hybrid space for constructing professional identity. [In Spanish.] In Edunovatic 2019: 4th Virtual International Conference on Education, Innovation and ICT, edited by REDINE, 237-242. Madrid: REDINE, 2019.

Moore, Michele-Lee, and Frances Westley. “Surmountable chasms: networks and social innovation for resilient systems.” Ecology and Society 16, no. 1 (2011). http://www.ecologyandsociety.org/vol16/iss1/art5/.

Oner, Diler, and Emine Adadan. “Are integrated portfolio systems the answer? An evaluation of a web-based portfolio system to improve preservice teachers’ reflective thinking skills.” Journal of Computing in Higher Education 28, no. 2 (2016): 236–260. https://doi.org/10.1007/s12528-016-9108-y.

Parkes, Kelly A., Katie S. Dredger, and David Hicks. “ePortfolio as a measure of reflective practice.” International Journal of ePortolio 3, no. 2 (2013): 99–115. http://www.theijep.com/pdf/IJEP110.pdf.

Rittel, Horst W. J., and Melvin M. Webber. “Dilemmas in a general theory of planning.” Policy Sciences 4, (1973): 155–169. https://doi.org/10.1007/BF01405730.

Roberts, Pauline. “Developing reflection through an ePortfolio-based learning environment: design principles for further implementation.” Technology, Pedagogy and Education 27, no. 3 (2018): 313-326. https://doi.org/10.1080/1475939X.2018.1447989.

Sabirón, Fernando, and Ana Arraiz. Learning from assessment: a decalogue for developing professional skills through portfolio authentic assessment. [In Spanish.] Revista Iberoamericana de Evaluación Educativa 6, no. 1 (2013): 135-152. https://revistas.uam.es/riee/article/view/3846/4031.

Scully, Darina, Michael O’Leary, and Mark Brown. The Learning Portfolio in Higher Education: A Game of Snakes and Ladders. Dublin: CARPE/NIDL, 2018.

Siemens, George. “Connectivism: A learning theory for the digital age.” International Journal of Instructional Technology & Distance Learning 2, no. 1 (2005): 3–10. http://itdl.org/Journal/Jan_05/article01.htm.

Slade, Christine, and Terry Downer. “Students’ conceptual understanding and attitudes towards technology and user experience before and after use of an ePortfolio.” Journal of Computing in Higher Education 32 (2019): 529–552. https://doi.org/10.1007/s12528-019-09245-8.

Sticchi, Maria. “From 1999 to 2019: 20 Years of European Debate, Development, and Achievements.” Tuning Journal for Higher Education 6, no. 2 (2019): 51-71. https://doi.org/10.18543/tjhe-6(2)-2019pp51-71.

Stilgoe, Jack, Richard Owen, and Phil Macnaghten. “Developing a framework for responsible innovation.” Research Policy 42, no. 9 (2013): 1568–1580. https://doi.org/10.1016/j.respol.2013.05.008.

Struyven, Katrien, Yves Blieck, and Véronique De Roeck. “The electronic portfolio as a tool to develop and assess pre-service student teaching competences: Challenges for quality.” Studies in Educational Evaluation 43 (2014): 40-54. https://doi.org/10.1016/j.stueduc.2014.06.001.

Swanson, Elizabeth, Lisa V McCulley, David J Osman, Nancy Scammacca Lewis, and Michael Solis. “The effect of team-based learning on content knowledge: A meta-analysis.” Active Learning in Higher Education 20, no. 1 (2019): 39–50, https://doi.org/10.1177/1469787417731201.

Tassone, Valentina C., Catherine O’Mahony, Emma McKenna, Hansje J. Eppink, and Arjen E. J. Wals. “(Re)designing higher education curricula in times of systemic dysfunction: a responsible research and innovation perspective.” Higher Education 76 (2018): 337-352. https://doi.org/10.1007/s10734-017-0211-4.

Thibodeaux, Tilisa, Cynthia Cummings, and Dwayne Harapnuik. “Factors that Contribute to ePortfolio Persistence.” International Journal of ePortfolio 7, no. 1 (2017): 1-12. http://www.theijep.com/pdf/IJEP257.pdf.

Trautwein, Caroline, Matthias Nückles, and Marianne Merkt. “Complex dynamics in academics’ developmental processes in teaching.” Higher Education Research & Development, 34, no. 3 (2015): 641-657. https://doi.org/10.1080/07294360.2014.973376.

Waddock, Sandra, Greta M. Meszoely, Steve Waddell, and Domenico Dentoni. “The complexity of wicked problems in large scale change.” Journal of Organizational Change Management 28, no. 6 (2015): 993–1012. https://doi.org/10.1108/JOCM-08-2014-0146.

Walder, Anne M. “Obstacles to innovation: The fear of jeopardising a professorial career.” British Journal of Education 3, no. 6 (2015):1-16.

Wenger, Etienne. “Communities of Practice and Social Learning Systems: the Career of a Concept.” In Social Learning Systems and Communities of Practice, edited by Chris Blackmore, 179-198. London: Springer, 2010.

Wernick, Andrew. “University. Theory.” Culture & Society 23, no. 2-3 (2006): 557-563. https://doi.org/10.1177/0263276406062810.

Woodward, Helen, and Phil Nanlohy. “Digital portfolios: fact or fashion?” Assessment & Evaluation in Higher Education 29, no. 2 (2004): 227-238. https://doi.org/10.1080/0260293042000188492.

---