Incremental steps towards a competency-based
post-secondary education system in Ontario

Mary Catharine Lennon

Abstract: As one of Canada’s 13 distinct jurisdictions, Ontario is a national leader in developing a competency-based postsecondary education system. Hindered by challenges of a disaggregated system of policy actors in system design, quality assurance and credit transfer, sweeping change has not occurred. Instead, various bodies with operational powers over university, college, or private-provider quality assurance have slowly incorporated concepts of competency-based education into frameworks by introducing learning outcomes. This paper outlines the challenges facing Canadian and Ontario postsecondary education, discusses the roles and responsibilities of agencies involved in quality assurance, and actions made towards developing and implementing learning outcomes at the system level. The research highlights the ad-hoc and unaligned activities, but also demonstrates the commitment to move towards a competency-based education system.

Keywords: Learning outcomes; competency-based education; system design; quality assurance; credit transfer; Canada; Ontario.

I. Introduction

Around the world, a number of policy activities over the past 20 years have focused on articulating and assessing competencies of students and embedding expectations into educational programming.^[1] Canada has joined in these activities at both global and local levels in order to understand better and demonstrate the quality of education, improve teaching and learning, and to support system level coordination: ultimately aiming to provide education that supports students in achieving their goals both within the classroom and in entering the workforce.

There are numerous reasons to support a competency-based education system. Through clearly articulated skills and capacities, the value of a credential is transparent to students, employers, and the wider public;^[2] it also aligns expectations of students, faculties, programmes, institutions and governments; improves teaching and learning techniques;^[3] and supports curriculum development.^[4] Competency-based education facilitated by clear learning outcomes can also support issues of system design, quality assurance, credit transfer and articulation which improves jurisdictional, national, and international understanding.^[5]

Recognising the varied goals, purposes, and levels at which activities can be focused, this article examines activities that are incorporating learning outcomes and the ideals of competency-based education as means to improve system design and quality assurance in Canada, and specifically the province of Ontario. Education is the sole jurisdiction of the Canadian provinces and territories. The Federal Government has limited reach into matters pertaining to education policies, system design, funding, quality assurance or curriculum development. While this presents challenges for developing nation-wide understandings and practices, it also provides opportunities for independent and innovative decisions made in the jurisdictions.^[6] Hence the article, focuses specifically on the activities of the Province of Ontario, the largest postsecondary education (PSE) system in the country and the most advanced in learning outcomes activities.

Since 2006 Ontario has been striving to develop a PSE system focused on accountability, accessibility and quality,^[7] and the continued commitment of government is aimed at supporting student achievement, ensuring high quality provision, and establishing fair accountability measures of success.^[8] This work has involved a number of policy actors including government, arms-lengths agencies, college and university non-governmental associations, and organisations responsible for quality assurance.^[9] This article focuses on initiatives that are enhancing system-wide change occurring at the level of the provincial government, its agencies, and non-governmental quality assurance organisations. Specifically, it demonstrates how the province is moving towards a competency-based system through a number of activities in a slow, uncoordinated, but steady manner by establishing expectations.

II. Concepts of Competency-based learning and learning Outcomes

Competency-based education is a conceptual framework that puts student learning at its core. It sees education as a process to provide students with demonstrable knowledge, skills and abilities. It represents a shift away from understanding and regulating education as a structural framework of inputs and outputs such as length of program and course material,^[10] and instead allows students to progress as they demonstrate mastery.

The notion of competency-based education can be traced back to the time of Plato,^[11] and has been an explicit component of vocational and professional education (such as nursing and medicine) and adult education since the 1960s.^[12] Over the past 20 years, the philosophy has gone beyond independent programs and boutique institutions: it is increasingly part of all types of programming including arts and science and humanities,^[13] has been incorporated into traditional higher education institutions,^[14] and established as part of national higher education systems.^[15]

The implementation of competency-based education systems or programming can be extreme or incremental.^[16] In the most extreme application of competency-based education, entire undergraduate degrees are provided to students who demonstrate mastery of the requirements at their own pace and through a variety of means. Western Governors University in the US is an example of comprehensive competency-based higher education that is not confined by traditional structural boundaries or time, place, or space.^[17]

The European Union is a prime example of how a meta-system and 47 national systems shifted to competency-based higher education in approximately 10 years. The parallel forces of the European Commission’s 1999 Bologna Declaration (aimed to create an integrated European Higher Education Area), and the 2002 Lisbon strategy (focusing on the modernization of higher education) revamped higher education across the continent.^[18] The processes each had a number of complimentary goals and activities,^[19] but two elements are key to the present discussion:

• Establishing a common system of degree comparison through qualifications frameworks

• Establishing common understanding and expectations of curriculum and performance criteria at the discipline level

Through these two activities, national systems (formerly based on credit hours and a myriad of structural designs) moved to a system that incorporated student work-load hours and student learning outcomes, which dramatically changed the framework of how higher education delivery and administration was conceptualized.^[20]

A less reformist way of moving towards competency-based education is to simply establish, or tack on, learning expectations to existing structures and curriculum.^[21] In this case, it is assumed that the development of competencies is already present in the curriculum, but that it becomes more transparent when articulated and identified through learning outcomes. Once established, incorporating learning outcomes into internal and external quality assurance regimes, into institutions, programs and curriculum, can be a significant task. This curriculum mapping — or constructive alignment, though simple in concept, and not nearly as reformist as the ‘extreme’ model, proves to be a challenge for many institutions. There have been a number of models of best practice put forth. For example, the US DQP/Tuning and the Ontario Tuning project present a detailed road maps for implementation from top to bottom,^[22] and Lennon and Frank^[23] describe how learning outcomes can be incorporated into a specific program and course curriculum.

The move to competency-based education is functionalised through the clear articulation of expectations of student learning. Though simple in concept, this is laden with verbiage that includes competencies, competence, learning outcomes, expectations, capacities, attributes, descriptors, standards, and more that are often entwined and can be conflicting.^[24] For example, the South African Qualifications Authority states “outcomes are the qualities…that are expected at the end of a process of learning. The meaning of outcomes is similar to the concept of competence”.^[25] Sadler, on the other hand, posits that competences are the broad “envelope term” that is made up of skills and competencies.^[26]

For clarity, this article uses two types of expectations commonly employed: ‘competencies’ and ‘learning outcomes’. Competencies are broad cumulative statements “of what a learner knows, understands and is able to do on completion of a learning process”.^[27] Very often they are categories of integrated skills, knowledge and abilities, which group sets of cognitive and non-cognitive skills that underlie performance. Commonly, four to eight competencies are established for a sector of education; be it k-12, lifelong, college-level diplomas, undergraduate, masters or doctoral degrees. For example, the Ontario Qualifications Framework and the Ontario Undergraduate Degree Level Expectations include: depth and breadth of knowledge; knowledge of methodologies, research and scholarship; application of knowledge, communications skills; awareness of the limits of knowledge, autonomy; and, professional capacity.^[28]

Learning outcomes, on the other hand, are “clearly defined and measurable statements of learning that reflect the scope and depth of performance; what a learner is expected to know, understand and be able to demonstrate after completion of a process of learning”.^[29] In the most technical sense, learning outcomes are action-oriented measurable knowledge, skills, and abilities that form the basis of the competencies.

Literature often uses the terms graduate capacities or attributes interchangeably with learning outcomes, but the intent is the same. The term ‘learning objective’ is also used in place of learning outcomes or competencies.^[30] Though Teichler and Shomburg^[31] (2013, 217-228) assert that the term ‘learning objectives’ is used more commonly and for the same purpose as learning outcomes, it is also argued that outcomes are distinct from learning objectives in that the former are short term and explicit to the content, where the latter deal with long-term ‘above content’ acquisition of knowledge, skills, and abilities.^[32]

Refraining from a more lengthy debate on the semantic use of terms,^[33] the ultimate purpose is to translate a course of study in to a set of understandable, measurable elements, and Canadian audiences use ‘learning outcomes’ as the generic term.^[34]

III. Background context and rationale for activities

Unlike some federated systems (such as the US or Australia), the Canadian Federal Government does not have any jurisdiction, or federal ministry, over educational issues.^[35] Each province or territory is responsible for organising, funding and directing K-12 and post-secondary education independently.^[36] Lacking strong federal involvement necessitates a number of actors in post-secondary education policy, at both national and provincial/territorial levels, which creates discrete realms of power in the policy and quality assurance landscape.^[37]

The number of actors and discrete realms of power in Canadian policy and quality assurance landscape is a significant hindrance to sweeping changes. What has occurred, instead, have been small changes within the parameters of the existing systems that are articulating expectations within traditional courses and programmes. Ontario’s quality assurance agencies, for example, are currently working to integrate learning outcomes into traditional course based programmes. Klein-Collins^[38] suggests this is the most basic move towards competency-based education.

Similarly, Canada has not had a significant external pressure to revise the system.^[39] Unlike Europe or South America, for example, there have been no international or regional forces to support large or strategic moves towards a system overhaul or competency-based education model. Instead, seeing the potential of the model, a variety of activities have taken place and are slowly moving the system forward.

Instead, a close watch is kept on the development of the European Higher Education Area particularly in the development of the European Credit and Articulation System and the European and International Tuning Initiatives. The emergence of these activities in provided valuable insight into how Ontario can mitigate structural confines while preserving sectoral autonomy in governance and quality assurance.^[40] Similarly viewing the US activities of Degree Qualifications Profile (DQP)^[41] and the American Association of Colleges and Universities LEAP initiative,^[42] has helped Ontario has shifted from ‘fear and denial’,^[43] towards acceptance of the value of integrating learning outcomes into the system.

This incremental approach is gaining traction in the various spheres of influence and control. Ad-hoc and unaligned, but with the dedication of many major players, the interests and activities incorporating learning outcomes at the system level, in quality assurance activities, institutions and the classroom, has increased. While there hasn’t been a system overhaul, instead there have been tentative steps towards incorporating learning outcomes, and a gradual move towards a system based on student competencies rather than a system based on administrative design.

1. National activities in quality assurance and learning outcomes

At the pan-Canadian political level, the Council of Ministers of Education Canada (CMEC) brings the jurisdictional governments together to discuss policy matters of mutual importance. It supports the provinces in common areas such as K-12 and postsecondary education, lifelong learning, aboriginal education, and is the international voice of Canadian education. In these capacities CMEC has some jurisdiction in quality assurance, credit transfer and learning outcomes conversations. In 2007 it developed the Canadian Degree Qualifications Framework (CDQF)^[44] which provides guidelines for degree-level standards/expectations for Bachelors, Masters and PhD degree programmes. This is similar to the European Qualifications Framework which provides a reference point for participating nations in conducting and examining their own systems, but is not legally binding.^[45] Unfortunately, perhaps because it is non-binding, the CDQF guidelines have been described as ‘little known’ and ‘not impactful’.^[46]

In representing Canadian interests on the international stage, CMEC supports international credit recognition and also has the opportunity to engage in international projects on learning outcomes, such as the OECD’s Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study.^[47] CMEC requires the agreement of all the provinces and territories to proceed with any activity. In the case of the AHELO Feasibility Study, the jurisdictions did not come to consensus on participation and Canada, as a nation, did not participate. Instead, the province of Ontario participated independently.

There are a number of national non-governmental organisations engaged in aspects of system design, quality assurance and credit transfer.^[48] These organisations represent their constituents’ needs to the broader Canadian audience that have contributed to the conversation on learning outcomes primarily through the publication of reports and guiding principles, conferences, and network building. One national organisation that has engaged specifically in learning outcomes is the Canadian Bureau for International Education (CBIE), a non-profit organisation supporting international activities and internationalisation of all levels of education. CBIE does not generally engage in issues of PSE quality assurance or curriculum development, focusing instead on supporting international student recruitment, student services and at-home internationalisation activities.

However, CBIE saw the value of establishing Canada-wide consensus in credentialing in order to promote international relations and understanding. In 2010, in collaboration with the European Commission, they conducted a Tuning Feasibility Study to explore establishing common standards and reference points for the fields of Engineering, Nursing and History.^[49] Despite the support of the EC and the interest of participants, their lack of political power and influence at both the provincial level and within the institutions was a challenge; and the project did not progress beyond the feasibility study.

The lack of authority of these pan-Canadian agencies and organisations demonstrates how firmly PSE lies in the hands of the provinces and territories. It is extremely difficult to establish consensus and develop nation-wide activities. Under these conditions the provinces are independently working through common issues. When possible, they come together through forums and working groups, but are independent in their legislature, policies, system design, and quality assurance models.

2. Provincial activities in quality assurance and learning outcomes

Each of the 10 provinces and 3 territories have their own system and structure for higher credential provision. Universities typically provide undergraduate and graduate degrees, where colleges generally provide more technical diplomas similar to UNESCO’s 2011 ISCED description of a short cycle qualification (as being typically practically based and occupationally specific).^[50] Some provinces, such as British Columbia, are structured to allow two years of college credits to be transferred into a bachelor’s degree. Quebec, alternatively, has a system where 2 years of college is mandatory for entrance into bachelor’s degree programmes. Others, such as Ontario, operate a binary system where there are no clear pathways between college and university sectors. The variety of system structures is important to understanding the breadth of work necessary to implement common expectations both within and across the provinces.^[51]

All provincial governments have accountability agreements with the publically-assisted universities and colleges. The agreements set out block funding and targeted funding packages for strategic initiatives. The exact nature of the accountability agreements vary in the jurisdictions, but the institutions provide significant amounts of information to the government including enrolment rate, retention rate, graduation rate, etc. which are required for funding eligibility.

Governments have differing involvement in the day to day activities of the institutions, varying by province as well as by sector. Traditionally, as colleges are more aligned with labour market needs, the governments maintain a strong role in determining programme offerings, establishing programme standards and operating quality assurance frameworks. Alternatively, Canadian universities are given complete autonomy in all these aspects. While the provincial governments legislate degree-granting status to institutions, they are not responsible for quality assurance activities.

There are a number of ways which provinces organise their quality assurance activities.^[52]’^[53] The Maritime Provinces (New Brunswick, Prince Edward Island, and Nova Scotia) have collectively developed an agency responsible for university-level quality assurance across the region. Some provinces self-regulate through member-based organisations, such as the provincial Council of Ontario Universities (COU), or through voluntary agreements with government agencies (such as British Columbia). Others, such as Alberta, allow each institution to be responsible for their own quality assurance (through governing councils and internal mechanisms) which are then related to the government. It should be noted that, to date, very few of the provinces have taken on the language of learning outcomes or competency-based education in their quality assurance systems.^[54]

The very small private sectors of PSE provision in Canada are heavily regulated by the provincial governments. Outside the purview of the public college and university quality assurance sectors, governments maintain a role in approving programming and performing institutional and programme reviews. While the governments do not accredit or fund private programming, they do ‘recognise’ programmes and provide student financial assistance to some.

The challenges of these disparate systems creates issues of recognition both across the country and within provincial systems. There are significant challenges in student mobility, credit transfer and articulation. There is a good deal of movement across provincial boundaries. In a survey of 40 universities across the nation, it was found that 18% of students lost prior credits when transferring to another province.^[55] Similar research within Ontario highlights that students transferring institutions within the province, are also likely to lose prior credits particularly if they are transferring from a college to university.^[56] For decades, most provinces have operated transfer activities on ‘case-by-case’ model of course credit acceptance,^[57] and many are still working on developing course-to-course credit transfer using course hour equivalences. One explanation for the continued work in this area is that there is no common definition of the credit: in some cases it is 24 hours of classroom time and others 39.^[58]

A number of provincially funded agencies, colloquially known at the ‘CATs’ — the Councils for Articulation and Transfer — have been developed within the past 5 years (i.e. ACAT,^[59] BCCAT,^[60] ONCAT^[61], NBCAT,^[62] etc.). These agencies are responsible for ensuring smooth transitions between institutions and programmes within each province and are critical in developing common expectations for programmes. Given the nature of the organisations, and that the majority of student transfer occur intra-provincially,^[63] they work primarily within the confines of their jurisdiction to improve college-to-university, college-to-college, university-to-university and university-to-college transfers.

Recognising the value in national conversations in this matter, in 2011 the CATs created a platform to discuss issues of inter-provincial transfers through the Pan-Canadian Council for Accreditation and Transfer (PCCAT). PCCAT is not intended to develop a common framework for interprovincial transfer, but instead designed to provide a platform for discussion and an opportunity to collaborate.^[64]

IV. Learning outcomes in the Ontario system

In order to understand how competency-based education is taking root in Canada it is best to examine one province as an in depth example. The following section outlines activities in Ontario’s governance, quality assurance and credit transfer in order to demonstrate how one province is slowly shifting towards incorporating learning outcomes and developing competency-based education.

Based on historic design and redesigns, the Ontario PSE system has a complex set of parameters that shape distinct sectors of college and university governance and quality assurance. A significant element is the binary system of the college and universities, intended to serve distinct purposes. It follows that there are separate quality assurance mechanisms in Ontario for public college, public university, and private sectors. Each has its own criteria, accountability frameworks, language, expectations and documentation.

In fact, there are seven different sets of binding accountability and quality assurance mechanisms managed by five different bodies, as well as two other agencies involved in different aspects of policy and research. Figure 1, below, illustrates the primary bodies in the Ontario quality assurance system. This is followed by a short discussion of the specific purpose, sphere of influence and competency-based education activities of each organisation. Note the differences in the content and presentation of the competencies/learning outcomes from the various agencies. Aligning the different frameworks is challenging, there is little coordination and limited formal paths of communication between the agencies (though there are informal conversations). These factors confound straightforward understanding and comparison of programming.^[65]

 

Figure 1

Ontario Quality Assurance System Actors and Roles

The Ontario Ministry of Training, Colleges and Universities (MTCU) is responsible for all postsecondary education including colleges, universities, apprenticeship and training, in both public and private institutions.^[66] It funds the institutions, establishes system design, and has accountability agreements with each individual institution. It has a different relationship with the colleges and universities, but maintains overall control of the system. For the most part, the government is not involved in day to day operations, and primarily funds the institutions through block funding, with some pockets of money reserved for particular initiatives, such as online-learning, supporting under-represented groups, etc.

MTCU developed the Ontario Qualifications Framework (OQF) in 2002. The OQF set out credential-level expectations for all levels of post-secondary education,^[67] intended to “set the standard for each credential that can be used to assess the quality of particular programmes at that credential level, and […] facilitate international recognition of credentials, credit transfer and graduate mobility’.^[68] All institutions are expected to comply with the OQF in the accountability agreement reporting.

Having an OQF follows the OCED’s advice on good practice in educational policy,^[69] and it was hoped that the framework would be utilized as a quality assurance tool^[70] by a) helping institutions to study how well their programmes are meeting expectations; b) assisting them in putting into place a quality assurance system; c) adding to the credibility of the qualifications within the framework, and; d) making it easier for students to compare similar programmes at different levels.^[71] Table 1, below, provides the example of the Qualifications Standards for Depth and Breadth of Knowledge (one of six descriptors).

Table 1

OQF Descriptors for Depth and Breadth of Knowledge^[72]

Descriptor	Diploma I	Advanced Diploma	Baccalaureate/Bachelor’s Degree	Baccalaureate/Bachelor’s Degree: Honours	Master’s Degree
Depth and Breadth of Knowledge	The skill and knowledge requirements for successful performance in a complex occupational setting.	a) The skill and knowledge requirements for successful performance of a specialized range of activities, most of which would be complex or non-routine in an occupational setting; b) At least some breadth beyond the vocational field, with exposure to at least one discipline outside the main field of study within the themes of: arts in society; civic life; social and cultural understanding; personal understanding; and science and technology, to increase awareness of the society and culture in which they live and work	a) A general knowledge and understanding of many key concepts, methodologies, theoretical approaches and assumptions in a discipline; b) A broad understanding of some of the major fields in a discipline, including, where appropriate, from an interdisciplinary perspective, and how the fields may intersect with fields in related disciplines; c) An ability to gather, review, evaluate and interpret information relevant to one or more of the major fields in a discipline; d) Some detailed knowledge in an area of the discipline; e) Critical thinking and analytical skills inside and outside the discipline; f) The ability to apply learning from one or more areas.	a) A developed knowledge and critical understanding of the key concepts, methodologies, current advances, theoretical approaches and assumptions in a discipline overall, as well as in a specialized area of a discipline; b) A developed understanding of many of the major fields in a discipline, including, where appropriate, from an interdisciplinary perspective, and how the fields may intersect with fields in related disciplines; c) A developed ability to: i. gather, review, evaluate and interpret information; ii. compare the merits of alternate hypotheses or creative options, relevant to one or more of the major fields in a discipline; d) A developed, detailed knowledge of and experience in research in an area of the discipline; e) Developed critical thinking and analytical skills inside and outside the discipline; f) The ability to apply learning from one or more areas outside the discipline.	A systematic understanding of knowledge, including, where appropriate relevant knowledge outside the field and/or discipline, and a critical awareness of current problems and/or new insights, much of which is at, or informed by, the forefront of their academic discipline, field of study or area of professional practice

1. Public College System

In the 1960’s Ontario developed the Colleges of Applied Arts and Technology to provide technical and practical education, which was seen as an important aspect of labour market development. The college system is tightly operated and regulated by the government. Beyond complying with the general government accountability framework (ensuring they receive government funding), colleges are required to provide Key Performance Indicator (KPI’s) including graduate satisfaction, employer satisfaction, and employment rate^[73] (which are not required for universities). The government is responsible for overseeing publicly funded programmes through the College Programme Standards Act, which outlines explicit content, curriculum and learning outcomes.^[74] Typical of other technical education systems,^[75] Ontario’s college sector is more advanced in learning outcomes-based education.

The province established the ‘Essential Employability Skills’ (EES) which are common expectations of standards across all college programs. The EES sets out 5 skill categories (such as ‘Numeracy’) and 11 broad learning outcomes (such as ‘execute mathematical operations accurately’).^[76] See Table 2 for an example of a competency (termed ‘Skill category’) of ‘Critical thinking and problem solving’, associated ‘skills’ and ‘learning outcomes’.

Table 2

Essential Employability Skill for Critical Thinking and Problem Solving^[77]

SKILL CATEGORY	DEFINING SKILLS: Skill areas to be demonstrated by graduates:	LEARNING OUTCOMES: The levels of achievement required by graduates. The graduate has reliably demonstrated the ability to:
CRITICAL THINKING & PROBLEM SOLVING	• Analysing • Synthesising • Evaluating • Decision making • Creative and innovative thinking	• apply a systematic approach to solve problems. • use a variety of thinking skills to anticipate and solve problems.

These generic competencies are complimented by very specific programme learning outcomes,^[78] which are to be embedded on programs across the province. Table 3 below shows an example of one of 10 ‘learning outcomes’ for a 2 year diploma in public relations.

Table 3

An example of a learning outcome from the Public Relations program standard^[79]

The Vocational Learning Outcomes

1. The graduate has reliably demonstrated the ability to participate in the planning of public relations activities, including the development of clear, measurable communication objectives and project budgets and selection of strategies, tactics, tools and resources to support a range of stakeholder* relationships and organizational objectives.

Elements of the Performance • Identify key stakeholders* and relationship management needs of a range of organizations in various sectors (e.g., commercial, not-for-profit, government, etc.) and industries • Distinguish between goals, objectives, strategies and tactics • Select strategies, tactics, tools and resources appropriate for different public relations functions (e.g., media relations, community relations, government relations, internal/corporate communications, investor/donor relations, reputation/crisis/issues management, special events, fundraising, sponsorship, etc), sectors (e.g., commercial, not-for-profit, agency, government, etc.) and industries • Participate in the selection of strategies and development of tactical plans to achieve objectives and guide public relations activities • Research and compile lists for a range of public relations functions, strategies and tactics (e.g., media lists, communication distribution lists, influencer lists, event invitation lists, lists of relevant social media platforms and Websites, potential donor lists, etc.) • Assist in needs analysis of target audiences to support the development of communications objectives • Draft communication objectives that are specific, measurable, attainable, realistic, time-sensitive (SMART), and aligned with organizational objectives • Accurately record project instructions, resources and constraints to support the planning process and guide evaluation of project success • Suggest creative approaches to public relations activity planning and problem solving • Participate in the development of project budgets that balance expenses with resources within known constraints • Apply public relations theories and practices to support the planning of public relations activities • Use project management tools to support the planning of public relations activities (e.g., work plans, critical paths, Gantt charts, etc.) • Participate in a strengths, weaknesses, opportunities and threats (SWOT) or similar analysis to support the planning of communications and public10 II — Vocational Standard relations activities • Identify and respond to the specific needs of journalists and news media in the planning of communication strategies and tactics for media relations • Source suppliers and follow principles, guidelines and relevant purchasing policies to obtain estimates for required products and services • Monitor, report and suggest responses to issues and trends that may impact the public relations activity planning process • Actively participate as a member of project planning teams, fostering collaboration and completing tasks

The programmes must demonstrate compliance with the both the EES and programme standards through the Ontario College Quality Assurance Service (OCQAS). This arms-length peer-review quality assurance agency for Ontario’s colleges is mandated by the government to provide quality assurance of college programmes through Credential Validation Services (which approves new programmes), and through the Programme Quality Assurance Process Audit (PQAPA) (which conducts academic audits).

Each programme complies with the EES as well as the OQF, but are not aligned with university programming, so it is very difficult to have credits recognised for a degree, making transfer to universities nearly impossible.^[80] Also, the very specific program learning outcomes make it difficult to recognise the transferability to other college programs.

Ontario colleges are seeking external quality monitoring. In late 2013, OCQAS was granted permission from its governing board (made up of college presidents) to become an accreditation agency.^[81] Unprecedented in Canada, the accreditation agency is expected to help validate the quality of the colleges. The development of the accreditation body is currently underway, and is expected to be operational by 2015. It will be extremely interesting to watch the development of this organisation, and see if, and how, they incorporate learning outcomes into the accreditation model.

2. Public University system

Quality assurance in the university system is a self-monitoring activity, where the institutions are responsible for their internal quality assurance process and report to the membership association the Council of Ontario Universities (COU).^[82] COU has a specific quality assessment arm, the Ontario Universities Council on Quality Assurance (OUCQA) that reviews institutional quality assurance procedures on a seven year cycle.^[83] The peer-review process examines the structure, curriculum and activities compared to the stated activities and missions of the programme. This collegial activity is a means to ensure compatibility and quality in programming across the province. These activities are of the ‘enhancement model’ suggested by Stensaker et al^[84] seeking to provide improvement rather than control on the programmes being evaluated.

In 2005 COU developed a list of Undergraduate Degree Level Expectations (UDLEs) for 3-year, 4-year and 4-year Honour’s Bachelor’s degrees, and Graduate Degree Level Expectations (GDLEs) for Masters and Doctoral degrees, based loosely on the OQF. These frameworks intended to clarify the expectations of the competencies programs would impart to students with the intention for institutions and programmes to develop their own expectations based on the frameworks.^[85] Despite using a similar format the UDLEs and GDLE’s do not explicitly correspond as each have unique competency areas.^[86] See Table 4 for an example of one of the competency areas used for both UDLEs and GDLEs.

Table 4

UDLEs and GDLEs Depth and Breadth of Knowledge^[87]

	Baccalaureate/bachelor’s degree This degree is awarded to students who have demonstrated the following:	Baccalaureate/bachelor’s degree: honours This degree is awarded to students who have demonstrated the following:	Master’s degree This degree is awarded to students who have demonstrated the following:	Doctoral degree This degree extends the skills associated with the Master’s degree and is awarded to students who have demonstrated the following:
Depth and breadth of knowledge	a) General knowledge and understanding of many key concepts, methodologies, theoretical approaches and assumptions in a discipline b) Broad understanding of some of the major fields in a discipline, including, where appropriate, from an interdisciplinary perspective, and how the fields may intersect with fields in related disciplines c) Ability to gather, review, evaluate and interpret information relevant to one or more of the major fields in a discipline d) Some detailed knowledge in an area of the discipline e) Critical thinking and analytical skills inside and outside the discipline f) Ability to apply learning from one or more areas outside the discipline	a) Developed knowledge and critical understanding of the key concepts, methodologies, current advances, theoretical approaches and assumptions in a discipline overall, as well as in a specialized area of a discipline b) Developed understanding of many of the major fields in a discipline, including, where appropriate, from an interdisciplinary perspective, and how the fields may intersect with fields in related disciplines c) Developed ability to: i) gather, review, evaluate and interpret information; and ii) compare the merits of alternate hypotheses or creative options, relevant to one or more of the major fields in a discipline d) Developed, detailed knowledge of and experience in research in an area of the discipline e) Developed critical thinking and analytical skills inside and outside the discipline f) Ability to apply learning from one or more areas outside the discipline	A systematic understanding of knowledge, including, where appropriate, relevant knowledge outside the field and/or discipline, and a critical awareness of current problems and/or new insights, much of which is at, or informed by, the forefront of their academic discipline, field of study, or area of professional practice;	A thorough understanding of a substantial body of knowledge that is at the forefront of their academic discipline or area of professional practice including, where appropriate, relevant knowledge outside the field and/or discipline.

Though intended to provide flexibility, programmes were challenged in demonstrating achievement of these rather broad generic expectations when undertaking either internal or external cyclical reviews. It was a challenge to take the traditional programme outlines and missions and align them into the broad categories in a meaningful way. The reaction from Ontario universities was to ask for more help in demonstrating that they are successfully providing students with the competencies. While intended to ensure compatibility and quality in programming across the province there is no evaluation of whether programs have embedded, assessed or measured student learning outcomes, or if programmatic or student-level expectations have been achieved^[88]. While a small move towards supporting competency-based education, it must be considered that prior to this work there was little established agreement as to the expectations of student learning outcomes across the university sector or programmes.

3. Alternate providers

The Private Career Colleges Branch of the Ontario government is governed by the Private Career Colleges Act of 2005. The private colleges must register with the Government and “meet certain standards for the programmes they offer, as well as advertising, refund policies, and instructor qualifications”. The Ministry of Training Colleges and Universities is responsible for ensuring institutions comply with the Act, and non-compliance punishments usually come in the form of financial penalties. The guidelines are primarily operational, and there is no curriculum — or learning outcomes — established by the government. The government does not conduct quality audits of programming, and the sector generally operates under ‘caveat emptor’ rules.

When the government opened PSE to new providers in the early 2000’s, it simultaneously created the Postsecondary Education Quality Assessment Board (PEQAB) to monitor them. PEQAB is responsible for approving degree programmes in private and out of province providers, as well as degrees in Ontario’s public colleges. By conferring consent to programs/institutions, PEQAB acts as a de facto accreditation agency for a small number of programmes.^[89] PEQAB was intended to bridge a gap because the new programmes were outside of the purview of both the college and university quality assurance mechanisms. Hence, the programmes are directly, and heavily, monitored by the government through detailed quality standards. Though there is some reference to learning outcomes (as PEQAB applied the Ontario Qualifications Framework when considering the applicability and quality of the programme), the majority of the standards relate to processes.

4. Government arms-length agencies

The Higher Education Quality Council of Ontario (HEQCO) is an arms-length government organisation developed in 2005 tasked with providing the Ontario Government with research-based policy advice on all aspects of post-secondary education. Having conducted significant research into international activities in ‘quality initiatives’, HEQCO set out a learning outcomes research agenda in 2011.^[90]

Two activities revolved around the assessment of learning outcomes and competencies. The Collegiate Learning Assessment (a tool that measures the generic skills students gain during a programme) was piloted in 8 institutions in the province.^[91] At the request of government, HEQCO also implemented the Organisation for Economic Cooperation and Development’s (OECD) Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study. AHELO sought to determine and assess internationally common learning outcomes in generic skills, economics and engineering.^[92] Both of these activities were valuable in demonstrating the potential of system-wide tests of learning outcomes and competencies, as well as highlighting some of the challenges including resistance to large-scale assessments, student recruitment and administration issues, as well as overall fit for purpose. They also provided insight into the importance of assessing learning outcomes.

Another significant activity was a ‘Tuning project’^[93] which aimed to establish provincially common learning outcomes. Given the existing work in the province, HEQCO did not duplicate or reshape the existing OQF, EES or UDLEs and GDLEs. Instead, it intended to create explicit, measurable expectations across credentials at the sector levels of physical science, life and health science and social science. With similar intentions as the European Sectoral Qualifications Framework, the purpose was to bridge detailed program expectations with the broader competencies.^[94] Furthermore, the intention was to work within these discipline groupings as the majority of transfers occur within discipline sectors (i.e. from one health sciences program to another), suggesting that the core competencies and learning outcomes would be similar. The learning outcomes were also intended to be specific enough that they would provide guidance for quality assurance, programme design, students and employers, to support an understanding of the ‘essence of the sector’.^[95]

Another action the Tuning project took was to work across credential levels (2-year, 3-year, 4 year honour’s bachelor’s degree and master’s degree), in order to aid credit transfer and mobility conversations. Despite structural differences in the systems, it was found that the core of the sectoral knowledge and competencies remained intact across the various credential types.

The sector groups designed a set of five common competency areas,^[96] sub-competencies and learning outcomes, with one competency area that was considered to be unique to each sector (practice and methods). The learning outcomes presented are more descriptive than what is set out in the qualifications framework and other quality assurance models. See, for example, Table 5, which provides the Knowledge competency’s subcompetencies and associated learning outcomes.

Table 5

Tuning Learning Outcomes Knowledge Competency^[97]

Subcompetency	Two-year diploma	3-year diploma	Bachelor’s degree	Master’s degree
Theory and Concepts	Describe and apply the major concepts, theories and practices in the discipline	Describe and apply major theories, principles and practices in the discipline	Drawing on fundamental principles, describe, apply and integrate major theories and practices in the discipline	Drawing on fundamental principles, describe, apply and integrate the major theories, research methods and approaches to inquiry and/or schools of practice in the field of study
Subcompetency	Two-year diploma	3-year diploma	Bachelor’s degree	Master’s degree
Numeracy	Interpret quantitative information, apply quantitative reasoning and perform appropriate calculations to draw conclusions	Interpret quantitative information, apply quantitative reasoning and perform appropriate calculations to draw conclusions	Interpret quantitative information, apply quantitative reasoning and perform appropriate calculations to draw conclusions	Interpret quantitative information, apply quantitative reasoning and perform appropriate calculations to draw conclusions
Limits of Knowledge and Qualification	Describe limitations of personal knowledge and tasks for which they are qualified	Describe limitations of personal knowledge and tasks for which they are qualified	Describe the limits to their own knowledge and how uncertainty and ambiguity influence their analyses and interpretations	Delineate the current limits of theory, knowledge and/or practice in the field and how uncertainty and ambiguity influence analyses and interpretations
Multidisciplinarity	Apply prescribed principles from related disciplines to their field of study	Identify and apply principles from related disciplines to their field of study	Identify and integrate principles from related disciplines to their field of study	Identify and integrate principles of other fields of study in independent research
Breadth of Knowledge	Describe and apply basic concepts theories and practices from across the sectors	Describe and apply basic concepts theories and practices from across the sectors	Describe and apply basic concepts theories and practices from across the sectors	Describe and apply basic concepts theories and practices from across the sectors

As part of the second half of the Tuning exercise, the groups turned their attention towards how learning outcomes can be integrated into programmes, and how they can be used to show compliance with existing regulations. As noted previously, one of the comments from the university sector was the difficulty in demonstrating compliance with the extremely broad competency areas in the OQF, UDLEs and GDLEs. The Tuning project mapped how learning outcomes can be used in quality assurance processes, integrated in to program design, course planning and student assignments.^[98]

The HEQCO Tuning learning outcomes are a non-binding contribution to the learning outcome conversation in Ontario. Given the existence of formal quality assurance processes the work intended to support institutions in both demonstrating compliance and integrating learning outcomes into programming. For the most part, this will not require a significant redesign of programmes, rather should make explicit what is already occurring in the classroom, programme and across the institutions. Ultimately, the Tuning learning outcomes attempt to provide a framework for competency-based education in the province. Part of this work was demonstrating how the quality assurance frameworks of the government, university and college sector were aligned with the Tuning learning outcomes and therefore aligned with each other.

The Ontario Council for Articulation and Transfer (ONCAT) is a government agency designed to support credit transfer and mobility of Ontario students. Developed in 2011, it has a 5-year mandate to improve student mobility in Ontario’s public institutions.^[99] Complementing HEQCO’s work on the sectors, ONCAT has established discipline working groups of college and university faculty members to plot credit transfer and block transfer agreements. As of July 2013, 9 projects had gone through the process of examining course by course expectations seeking commonalities and establishing credit worth.^[100]

The organisation has not yet moved towards developing a student workload based credit transfer system. It has been argued that the challenges and boundaries of the existing system are insurmountable and full system overhaul doesn’t make it feasible at this time.^[101] Hence, working within the system rather than against it, ONCAT was working methodically through course-by course credit equivalencies using the established credit hour model.

Though potentially successful in programme transfers in the short term, it was recognised that this way of working lacked an overarching format: the groups were not guided by explicit learning outcomes or competency-expectations. In fall 2013, ONCAT established a work plan to develop a common framework for developing discipline based learning outcomes,^[102] and, at the time of writing, the expectation is that the HEQCO Tuning learning outcomes will be used as the framework to guide further discipline specific activities.

V. Summary and Conclusion

The complex system of Canadian actors and authorities responsible for accountability, quality assurance and credit transfer has created a number of haphazard policies, compliance frameworks, and quality assurance mechanisms. Most notably, the provincial systems are unable to work together in a formal way.^[103] Hence, most provinces are focused on the very practical and pressing issues of improving student mobility within the existing framework rather than thinking critically about how the current framework can be reshaped to enhance the overall system.

Despite the fractured landscape of power and control in system design and quality assurance across the country and within the province of Ontario, there is a growing recognition that competency-based education and articulated learning outcomes are a means to improve and support a strong post-secondary education system. Though it will require significant efforts, the reimagining of this system is critical to serving the purpose of the students and the national system.

Ontario is striving to develop a competency-based PSE system by incorporating learning outcomes into the existing system. Given the patchwork set of organisations involved in various aspects of system design and quality assurance it is necessarily a slow process of developing common understanding, and working towards mutual goals. Having crossed the first barrier of gaining general acceptance on the value and potential of learning outcomes, the stage of implementing them in to quality assurance processes and supporting their integration into programming has now been reached.

Lacking a significant policy shift, the expectation is that within the next several years the conversations will continue and the collaborative efforts will reap changes. There is conceptual agreement that competency-based education is sound andragogy. Ontario has passed the first hurdle of accepting the potential value of establishing learning outcomes and is currently in the early stages of working towards clear competency expectations and usable learning outcomes that are appropriate for programme development within each sector. The first step of which has been to shift to a more outcomes based system by clearly articulating the expected outcomes of programs and credentials, as well as exploring system-wide measures of student achievement.

There are two major hurdles Ontario will have to face in the coming years. The first is coming to consensus on how to coordinate the various quality assurance learning outcomes, and perhaps adopt a common framework. Until the jurisdiction has internal agreement it will remain impossible to coordinate with other provinces or internationally.

The second hurdle is one faced by countries, quality assurance agencies and institutions around the world: finding suitable assessment mechanisms to provide appropriate demonstrations of achievement by students, and realistic measures of programme quality in institutions. This move towards assessments of learning outcomes and incorporating them into quality assurance mechanisms is a critical element, and is the next step in Ontario’s path to a competency-based higher education system.

Bibliography

Association of Community Colleges and Universities. “Liberal Education and America’s Promise (LEAP).” Accessed February 26, 2014, https://www.aacu.org/leap/.

Association of Universities and Colleges of Canada. The Revitalization of Undergraduate Education in Canada. Ottawa, Ontario: Association of Universities and Colleges, Canada, 2011.

Canadian Bureau for International Education. “About IE: Tuning.” Accessed February 26, 2014, http://www.cbie-bcei.ca/about-ie/tuning/.

Clark, Ian, Greg Moran, Michael Skolnik, and David Trick. Academic Transformation: The Forces Reshaping Higher Education in Ontario. Kingston, ON: McGill-Queen’s University Press, 2009.

Colleges Ontario. “Key Performance Indicators.” Accessed January 26, 2014, http://www.collegesontario.org/outcomes/key-performance-indicators.html.

Council of Ministers of Education, Canada, Ministerial Statement on Quality Assurance of Degree Education in Canada. Toronto, Ontario: Council of Ministers of Education, Canada, 2007.

European Centre for the Development of Vocational Training. The Shift to Learning Outcomes: Conceptual, Political and Practical Developments in Europe. Luxemburg: European Centre for the Development of Vocational Training, 2008.

Haskel, Barbara. “Where there’s a Will …: Reforming Postsecondary Education in Canada’s and the European Union’s Decentralized Systems.” Canadian Public Administration 56, no. 2 (2013): 304-321.

Hatchette, Virginia. “The Value of Learning Outcomes: The Canadian Perspective.” In Measuring the Value of a Postsecondary Education, edited by Norrie, Ken and Mary Catharine Lennon, 87-101. Kingston-Montreal: McGill-Queen’s University Press, 2012.

Heath, Nick. Student Mobility in Canada across Canadian Jurisdictions. Windsor, Ontario: Pan-Canadian Consortium on Admissions and Transfer, 2012.

   . Student Mobility within the Province of Ontario: Supplement to the Report on the Pan-Canadian Survey of Student Mobility. Windsor, Ontario: Pan-Canadian Consortium on Admissions and Transfer, 2012.

Higher Education Strategy Associates. Changing Times, Changing Places: The Global Evolution of the Bachelor’s Degree and the Implications for Ontario. Toronto, Ontario: Higher Education Quality Council of Ontario, 2012.

Higher Education Quality Council of Ontario. Annual Report 2010/2011. Toronto, Ontario: Higher Education Quality Council of Ontario, 2010.

Jankowski, Natasha and David Marshall. DQP Road Map to Enhanced Student Learning: Implementing the DOP and Tuning. Urbana, IL: National Institute for Learning Outcomes Assessment (NILOA) and Institute for Evidence-Based Change (IEBC), 2014.

Jones, Glen A., ed., Higher Education in Canada: Different Systems, Different Perspectives, edited by New York: Garland Pub., 1997.

Junor, Sean and Alex Usher. Student Mobility and Credit Transfer a National and Global Survey: Educational Policy Institute, 2008.

Kennedy, Declan, Aine Hyland and Nora Ryan, “Writing and using Learning Outcomes”, In Bologna Handbook, Implementing Bologna in your Institution, C3.4-1, (2006): 1-30.

Klein-Collins, Rebecca. Competency-Based Degree Programs in the U.S.: Postsecondary Credentials for Measurable Student Learning and Performance. USA: Council for Adult and Experiential Learning, 2012.

Lennon, Mary Catharine. Piloting the CLA in Ontario. Toronto, Ontario: Higher Education Quality Council of Ontario, 2014.

Lennon, Mary Catharine and Linda Jonker. AHELO: The Ontario Experience. Toronto, Ontario: Higher Education Quality Council of Ontario, 2014.

Lennon, Mary Catharine and Brian Frank, “Learning outcomes assessments in a decentralized environment: The Canadian Case.” In Higher Education Learning Outcomes Assessments: International Perspectives, edited by Hamish Coates, 89-112. Frankfurt; Peter Lang, 2014.

Lennon, Mary Catharine, Brian Frank, Rhonda Lenton, Kirsten Madsen, Abdelwahab Omri, and Roddy Turner. Tuning: Identifying and Measuring Sector-Based Learning Outcomes in Postsecondary Education. Toronto, Ontario: Higher Education Quality Council of Ontario, 2014.

Lumina Foundation. The Degree Qualifications Profile. USA: Lumina Foundation, nd.

Miller, Margaret A. From Denial to Acceptance: The Stages of Assessment. USA: National Institute for Learning Outcomes Assessment, 2012.

Ministry of Training, Colleges and Universities. “Essential Employability Skills.” Accessed January 26, 2014, http://www.tcu.gov.on.ca/pepg/audiences/colleges/progstan/essential.html.

   . “Ontario Qualifications Framework (OQF).” Accessed February 26, 2014, 2014, http://www.tcu.gov.on.ca/pepg/programs/oqf/QsAsOQF.html.

Mitchel, Amy, Lane Trotter, Wendy Wilson, and Ryan Walmsley. Facilitating College to University Transfer in the European Higher Education Area and Beyond: Opportunities for Ontario’s College of Applied Arts and Technology. London, Ontario: ONTransfer and College University Consortium Council, 2013.

Mulder, Martin, Tanja Weigel, and Kate Collins. “The Concept of Competence in the Development of Vocational Education and Training in Selected EU Member States: A Critical Analysis.” Journal of Vocational Education & Training 59, no. 1 (2007): 67-88.

Ontario College Quality Assurance Agency. Colleges Moving to an Accreditation Process. New Model Starts 2015. Toronto, Ontario: Ontario College Quality Assurance Agency, 2013.

Ontario Council on Articulation and Transfer. Credit Transfer Saves Ontario’s Postsecondary Students Time and Money. Toronto, Ontario: Ontario Council on Articulation and Transfer, 2013.

Ontario Government, Reaching Higher: The McGuinty Government Plan for Postsecondary Education. Ontario: Ontario Government, 2010.

   . Forging New Pathways to Improve Student Mobility in the Province of Ontario. Toronto, Ontario: Ontario Council on Articulation and Transfer, 2013.

   . ONCAT Project Status Report: Diploma to Degree System Transfer Pathways. Toronto, Ontario: Ontario Council on Articulation and Transfer, 2013.

   . “Public Relations (Ontario College Diploma) Program Standard”. Accessed May 14, 2014. http://www.tcu.gov.on.ca/pepg/audiences/colleges/progstan/aa/50243.pdf

   . “Essential Employability Skills”. Accessed May 14, 2014. http://www.tcu.gov.on.ca/pepg/audiences/colleges/progstan/essential.html

Ontario Universities Council on Quality Assurance. Quality Assurance Framework. Toronto, Ontario: Ontario Universities Council on Quality Assurance, 2010.

“Quality Assurance in Ontario.” Accessed February 26, 2014, http://oucqa.ca/framework/1-2-quality-assurance-in-ontario/.

Rae, Bob. Ontario: A Leader in Learning. Report and Recommendations. Toronto, Ontario: Government of Ontario, 2005.

Shanahan, Theresa and Glen Jones. “Shifting Roles and Approaches: Government Coordination of Post-Secondary Education in Canada, 1995-2006.” Higher Education Research and Development 26, no. 1 (2007): 31-43.

Skolnik, Michael. (How) Do Quality Assurance Systems Accommodate the Differences between Academic and Applied Higher Education? (forthcoming)

Stensaker, Bjorn, Liv Langfeldt, Lee Harvey, Jeroen Huisman, and Don Westerheijden. “An in-Depth Study on the Impact of External Quality Assurance.” Assessment & Evaluation in Higher Education 36, no. 4 (2011): 465-478.

Trampusch, Christine. “Europeanization and Institutional Change in Vocational Education and Training in Austria and Germany.” Governance 22, no. 3 (2009): 369-395.

Tremblay, Karine, Diane Lalancette, and Deborah Roseveare. AHELO: Feasibility Study Report: Volume 1: Design and Implementation. Paris, France: Organisation for Economic Cooperation and Development, 2012.

Wagenaar, Robert. “Qualifications frameworks, Sectoral Profiles and Degree Programme Profiles in Higher Education.” Tuning Journal for Higher Education: New Profiles for New Societies 1, no. 1 (2013): 71-103.

Copyright

Copyright for this article is retained by the Publisher. It is an Open Access material that is free for download, distribution, and or reuse in any medium only for non-commercial purposes; provided any applicable legislation is respected, the original work is properly cited, and any changes to the original are clearly indicated.