Competences and learning outcomes:
a panacea for understanding the (new) role of Higher Education?
Abstract: The competence and learning outcomes approach, which intends to improve effective performance of academic staff and students, is becoming dominant in today’s higher education. This was quite different 15 years ago. This contribution aims to offer insight in the reforms initiated and implemented, by posing and answering the questions why the time was appropriate — by identifying and analysing the underlying conditions — and in what way the change was shaped — by focusing on terminology required and approaches developed. Central here is the role the Tuning project — launched in 2000-2001 — played in this respect. The contribution starts with contextualising the situation in the 1990s: the recession and growing unemployment in many European countries on the one hand and the development of a global society and the challenges the higher educational sector faced at the other. It offers the background for initiating the Tuning project, and the discourse on which its approach is based. In particular, attention is given to choosing the concept of competences, distinguishing subject specific and general/generic ones, as an integrating approach of knowledge, understanding, skills, abilities and attitudes. The approach should serve as a means of integrating a number of main goals as part of the learning and teaching process: strengthening employability and preparing for citizenship besides personal development of the student as a basis for the required educational reform. Tuning’s unique contribution is the alignment of this concept to learning outcomes statements as indicators of competence development and achievement and by relating both concepts to profiling of educational programmes.
Keywords: Competenc(i)es; generic competences; subject specific competences; learning outcomes; transferable skills; subject specific skills.
In the current debate about higher education the concepts of student centred and active learning, competence development and learning outcomes have obtained a central place. This was quite different when the Bologna Declaration of 1999 was published now fifteen years ago. These concepts do not appear in that document which launched the reform process of higher education in Europe to develop one European Higher Education Area to match one European economic area. At its start, the Bologna Process focused on easily readable and comparable degrees, introduction of a two cycle system, establishing a credit system and mobility, cooperation in quality assurance and the promotion of European dimensions. As one of the reasons behind this initiative it mentions explicitly “to promote European citizens employability”, although the document carefully avoids making a direct link to the European economic agenda. Instead it stipulates the importance of a Europe of Knowledge “as an irreplaceable factor for social and human growth and as an indispensable component to consolidate and enrich the European citizenship, capable of giving its citizens the necessary competences to face the challenges of the new millennium, together with an awareness of shared values and belonging to a common social and cultural space”.
Four years later at the Bologna Follow-up conference held in Berlin (2003), “the European Ministers of Education were much more explicit in confirming that the conclusions of the European Councils in Lisbon (2000) and Barcelona (2002), which aimed at making Europe “the most competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth with more and better jobs and greater social cohesion”, should be taken into account”. The objectives “easily readable and comparable degrees and introduction of a two cycle system” were rephrased as “to elaborate a framework of comparable and compatible qualifications for their higher education systems, which should seek to describe qualifications in terms of workload, level, learning outcomes, competences and profile”. The rephrasing shows the development in thinking about the role of higher education and the content of its degree programmes. In practice it implied a change of paradigm with wide implications.
The aim of this contribution is to offer insight into this change of paradigm and its implications, by posing and answering the questions of (i) why the time was appropriate, by identifying and analysing the underlying conditions, and (ii) how the change was shaped, by focusing on the terminology required and approaches developed. Central here is the role the Tuning project — launched in 2000-2001 — played in this respect. Was it a means to understand the (new) role of higher education institutions in today’s world?
In the late 20th century a number of developments coincided which triggered change with regard to the higher education agenda in Europe. Economies, already in recession, slowed down further as a result from the East-Asian financial crisis in 1997 and the Ruble crisis in 1998, preceded by the Mexican crisis of 1995. In 1996 the European Commission stated in a White Paper on Education and Training that long term unemployment continued to increase, resulting in the spread of social exclusion, particularly among young people.
At the same time mass higher education reached its peak in Europe. As Peter Scott had pointed out already in 1995, this development of mass education did not take place in isolation but was related to a much wider transformation in the nature of society and the structure of the economy(ies) in the late 20th century and it affected the intellectual culture as well as science and technology. At the same time the limits of public spending on (higher) education were reached. Cost effectiveness, efficiency and accountability of higher education became serious issues, and were related to quality assurance mechanisms and (high) dropout rates. In the background the information and communication society, based on the revolutionary development of new technology, developed at tremendous speed. Internet was commercialised in 1995 and gave rise to electronic mail and instant messaging. As an effect, new social media methods developed by the end of the 1990s. Transparency became a buzz word. The global society was further stimulated by faster and cheaper transportation, which facilitated the internationalization of higher education in terms of staff, but in particular student mobility and curriculum development.
Mechanisms for the latter were — as is well known — developed from 1985 in the framework of the Erasmus Programme. In the autumn of 1988 the European Commission launched an initiative to create one European credit system based on student workload instead of contact hours, which was to be developed on the basis of a bottom-up approach. The Commission invited institutions and their academics to setup and test such a system. This was done successfully. However, at the end of the1990s a growing concern developed among the initiators and developers of the European Credit Transfer System (ECTS) that one credit system would not be a sufficient answer to societal challenges as described above and was not the sole solution for (trans)national mobility of students and cooperation between higher education institutions in and outside Europe. It was concluded that the emphasis should be much more on the outcomes of the learning process — as well as on the accumulation of credits —, with much more attention to be given to transferable or general skills and competences. It was also thought necessary to highlight the role of higher education institutions as major contributors to the welfare of society. This fitted in the described trend in which higher education institutions were forced to show that they are accountable, responsible, and sustainable. It was thought that this should not only apply to the higher education institution but also to all individual degree programmes on offer.
III. The Tuning initiative
These concerns led to the grass roots initiative in 2000 to establish the Tuning Educational Structures in Europe Project. The initiative was based on the following assumptions formulated as challenges:
— Higher education structures and programmes and qualifications should be reformed on a large scale to be able to respond to the needs of society, to be understood in terms of preparing for employability and citizenship;
— Academics should be given a key role in this process;
— The reform process would require the development of internationally shared reference points / standards at disciplinary / subject area level;
— A language for communication should be developed which would be understood by all major stakeholders, that is academics, students, graduates, (potential) employers of graduates as well as professional organisations;
— All stakeholders, including (potential) employers and professional organisations, but in particular graduates should be (indirectly) involved in the process of curriculum design and quality enhancement;
— The focus should be on diversification of degree programmes by profiling and stimulating flexibility;
— The reform should facilitate national and international mobility and the recognition of periods of study, including qualifications for obtaining access to the next level of programmes.
As stated in the introduction the focus in this article is on the methodology and terminology developed and applied to respond to these challenges. During time these have been further developed and fine-tuned.
When developing the Tuning project the terminology to be used was not chosen lightly. By focusing on competence development and relating these to the outcomes of the learning process, the initiators were very much aware that the change in higher education had to be made from the then dominant staff centred approach to a student centred approach. Tuning was unique and first by integrating the concepts of competences and learning outcomes (1) by stating that level of competence should be expressed in terms of learning outcomes statement as well as vice versa and (2) by relating both concepts to the definition of profiles for subject areas.
By making this choice Tuning aligned with a development which had obtained credibility in the preceding decade. In 1995 Robert B. Barr and John Tagg published their groundbreaking article in Change. The Magazine for Higher Education, entitled “From Teaching to Learning. A new paradigm for undergraduate education”. According to the authors a shift was taking place in US colleges from providing instruction to producing learning. This meant, in their vocabulary, a change from the Instruction Paradigm to the Learning Paradigm. They bluntly called the first paradigm no longer effective, by quoting an article in the same Journal published in the autumn of 1994: “the primary learning environment for undergraduate students, the fairly passive lecture-discussion format where faculty talk and most students listen, is contrary to almost every principle of optimal settings for student learning.”
Peter T. Ewell, executive editor of the magazine noted in the autumn of 2002 that the Barr-Tagg article had been “arguably the most widely cited piece that Change has ever published”. He made this remark in the foreword of a book The Learning Paradigm College, John Tagg published in 2003 as a follow-up to the article. The article gained support and met severe criticism. Tagg himself was surprised by its reception which in his wording “struck a responsive chord with many in the higher education community” This is remarkable against the background that already during the mid-1980s national groundbreaking reports were published in the US which highlighted the issues at stake. The terminology regarding learning we now know so well has been developed since. What alarmed the readers of the article was probably less its content as the use of the phrase educational paradigm shift. Did the reports and the article have much impact in the shorter run? In 2002 they had not yet, as Ewell stipulates in the foreword cited above. That does not mean that the issues covered by the article of Barr and Tagg were not current. What makes the article very important is not only the readability of their argument but also and in particular the comparison of the old and new paradigms, the Instruction Paradigm and the Learning Paradigm, structured around six topics: mission and purposes; criteria for success; teaching/learning structures; learning theory; productivity/funding and nature of roles.
Some of the key words are picked out here from this comparison: produce learning (versus delivering instruction); improve the quality of learning (versus improve the quality of instruction); quantity and quality of outcomes (versus quantity and quality of resources); holistic (versus atomic); cross disciplinary (versus independent disciplines); external evaluations of learning (versus grading within class); degree equals demonstrated knowledge and skills (versus accumulated credit hours); knowledge is constructed, created (versus learning is cumulative and linear); learning is student centred and controlled (versus learning is teacher centred and controlled); learning environments and learning are cooperative, collaborative and supportive (versus the classroom and learning are competitive and individualistic); funding for learning outcomes (versus funding for hours of instruction). This all culminates in a changing role for (the) academics. According to the new paradigm they are no longer primary instructors but primary designers of learning methods and environments, which requires academics and students work in teams with each other and with other staff. The main role of academic staff is to “develop every student’s competencies and talents”.
The Tuning initiators concluded, like Barr and Tagg, that a change of paradigm was required in the way higher education was organized and implemented. This was the core of the project proposal submitted to the European Commission in the autumn of 2000. Nevertheless, there were also striking differences. First of all Tuning intended to cover both undergraduate and (post)graduate studies. It also thought it wise to involve all stakeholders, besides academics and students, that is, graduates, employers and professional organisations, in implementing the initiative. This involvement was clearly related to the aim formulated in the Bologna Declaration to promote European citizens’ employability. To avoid any misunderstandings it stipulated at the same time that this should be read in conjunction to the role of higher education institutions have for personal development of the student as well as preparing them for citizenship.
Tuning developed a two way approach: on the one hand to contribute to the development of easily readable and comparable degrees by identifying common ground and on the other by developing a bottom-up approach for modernizing existing and new degree programmes by strengthening the role of higher education institutions in today’s societies. The context was most of all a European one. Having more than ten years of experience with large scale student mobility in Europe — which showed partial failure in recognition of period of studies taken elsewhere — as well as a growing gap between the content of degree programmes and the requirements of society, urged action on the part of universities. If the universities and their academics were not able to take the lead in the required reform process, others would do it for them. The Sorbonne (1998) and the Bologna (1999) Declarations were clear indicators in this respect.
IV. ‘New terminology’
To develop its agenda Tuning needed a “language” to be able to communicate with all stakeholders involved, and for the stakeholders to communicate among each other; a language, which would be understood as being relevant for the changes to be made and the results to be obtained. It introduced — in conjunction — the concepts of competences and learning outcomes in its discourse with academics from a range of disciplines, which were named subject areas to stress the wider context. Of course, these were not new concepts, but linking the competences and learning outcomes gave a unique focus to the project. The use of the term competence symbolises and underpins the intention to relate the higher education sector to the labour market. Tuning was well aware that the language chosen originated from outside academia. The term competence has a long history going back to Ancient times, but seems to be ‘established’ already in different languages in the 16th century. In 1973, it was related to education by Harvard professor David McClelland. In the 1980s it was linked to the professional world in particular Human Resource Development and in the 1990s to the learning paradigm. In a report to the US National Institute of Education (1980), G.O. Klemp defined competence as “an underlying characteristic of a person which results in effective and/or superior performance on the job”. S.R. Parry refined this definition in 1996 to “a cluster of related knowledge, skills, and attitudes that reflects a major portion of one’s job (a role or responsibility), that correlates with performance on the job, that can be measured with well-accepted standards, and that can be improved with training and development”. The development and use of the concept is reflected in educational research, in which three main traditions developed, starting with the behaviourist approach (McClelland and the Hay Group) followed by the generic approach and the cognitive approach. All three are “performance” oriented but the scope of competences to be owned / developed is widened through time.
In 1997, Walo Hutmacher could state in the European Journal of Education that the term competency is now widely used and accepted in Europe. He added however that there was still discussion on its definition, “doubtless partly due to language differences”. However, he also stipulates that there seems agreement that “the notion of competency lies very firmly within the field of ‘knowing how’ rather than ‘knowing that’”. This is an important observation. For Tuning the same notion was the reason to state that learning outcomes are expressed in terms of competences. This is reflected in its initial definition where it says “by learning outcomes we mean the set of competences including knowledge, understanding and skills a learner is expected to know/understand/demonstrate after completion of a process of learning — short or long”. This fits the definition offered by Hutmacher: “Competency is a general capability based on knowledge, experience, values, dispositions which a person has developed through involvement with educational practices. Competencies cannot be reduced to factual knowledge or routines; to be competent is not always synonymous with being knowledgeable or cultivated”.
This did not imply that the concept of competences was not challenged. It was from the moment that the concept was linked to education. From a present day perspective criticisms in the 1990s discourse regarding the term were strongly inspired by defending the staff centred approach. As Sultana summarizes the debate: “competence approaches were considered to focus on performance at the expense of complex intellectual processes, and reflection in and on action”. The argument sounds familiar, the competence approach “understates and belittles the role of knowledge and understanding, with the focus on skill serving to separate theoretical from practical knowledge and undermining values of personhood”. It symbolized the ivory tower mentality, which did not value that notions and concepts might change their meaning over time or might be perceived differently in another language or country. Although Sultana argues the contrary, the notion of competences as a way to frame learning was not seriously influenced by these criticisms. Many quibblers seemed to have ignored the fact that the term developed during the most recent two decades into having a more holistic or encompassing meaning covering all elements of learning. In the ongoing discourse among educational scientists the term is put aside by some of them as suffering from “fuzziness and conceptual confusion”. This judgement is based on a number of arguments as spelling of the word (GB versus US vocabulary / competence versus competency), related to different meanings in different contexts: behaviourism versus outcomes. In this argument behaviourism is defined as the competency of an employee related to high level of performance, while competence is measured against defined standards. Most users of the terms will not be aware of this difference and use them interchangeable nowadays. The difference seems to be relevant for purists only. Another criticism is (or was) that the concept of competence is used as equivalent to knowledge, skill, or ability, which adds to the confusion. Is this indeed conceptual inflation or is it an outdated opinion?
In 2000 it was concluded in an OECD expert paper produced in the framework of the OECD’s Definition and Selection of Competencies (DeSeCo) Project (1997-2003) that there is not a single theoretical concept of competence and the paper recommended a pragmatic approach towards the term. According to these experts competencies should be “conceptualized as the necessary prerequisites for meeting complex demands”. 
However, at their meeting in April 2001 the OECD Education Ministers stated in a communiqué that “Sustainable development and social cohesion depend critically on the competencies of all of our population — with competencies understood to cover knowledge, skills, attitudes and values.” In a summary report of the DeSeCo Project, published in 2005, it is stated that “a competency is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilizing psychosocial resources (including skills and attitudes in a particular context”. It also states that “despite the fact that competencies comprise more than just taught knowledge, the DeSeCo Project suggests that a competency can itself be learned within a favourable learning environment”.
As stated above, Tuning choose as its language of communication with stakeholders the concepts of competences and learning outcomes. By doing so it made the choice for a holistic interpretation of competences. It became firm ground in the years to come. This was confirmed in a CEDEFOP report finished in 2005. This report which relates in particular to the VET sector, signaled that the dominant theoretical educational frameworks were converging. “Functional and cognitive competences are increasingly being augmented by social or behavioural competences and there appears to be a general movement towards the more holistic approaches that have been associated with Germany and France, where knowledge, skills and social attitudes and behaviours are viewed as related dimensions of competence.”
This not only counted for the VET sector, it was also obviously true for Higher Education. In particular in continental Europe the concept of competences to structure teaching and learning gained influence during the last years of the last century and the first decade of the 21th century. A good example in this respect is the introduction of domain competences for clusters of educational programmes developed by the Universities of Applied Sciences in the Netherlands, which were published from 2004 on, and which were prepared in the previous years. This is a relevant example for Tuning because the role of this type of university is to educate graduates for defined employability sectors. Due to their mission they are much closer to the world of employment than most academic disciplines.
V. Role of Tuning
The intention of Tuning was most of all to focus on research based universities and traditional subject areas, not — at first instance — the more applied studies or multi- and interdisciplinary ones. They would follow later. The argument was that when these traditional disciplines could be convinced to adopt the notions of competences and learning outcomes, others would follow. At the launch meeting of the Tuning project in May 2001 many academics were sceptical. In particular because the initiators stressed that the project planned to give special attention to generic competences or transferable skills besides subject specific ones.
Although the initiators were not aware of the “paradigm shift” discussions taking place in the USA, they were well informed about the European discourses regarding competences and teaching and learning. For them competence development and a shift of paradigm were closely related. The ECTS experience showed them that full recognition of periods of studies or even degrees would never take place if the focus was kept on the content of knowledge. Ten years of intensive experience with ECTS as a transfer system proved that trust and confidence could actually be developed between academics but that course to course comparison continued to be very strong, because equivalent knowledge was sought. The Tuning initiators were also aware that universities were very much focused on themselves (and each other) instead of following what was happening in society. Studies were input and staff-centred based instead of output and student oriented.
When preparing Tuning, no serious discussion took place about (dominant) educational frameworks or typologies to follow. If fitting in any, it would be the holistic multi-dimensional or integrative approach towards competence-based models, allowing for an analytical concept of competences and by giving it its own flavour. This would encourage new practices in teaching, learning and assessment, including problem-based learning, project-based and team learning as well as active and autonomous learning. What was also clear, was that the approach would be student-centred, by which is meant an approach or system that supports the design of learning programmes which focus on the learners’ achievements, accommodates different learners’ priorities and is taking into consideration student workload (i.e workload that is feasible within the duration of the learning programme). Thus promoting greater student involvement in the choice of content, mode, pace and place of learning.
The emphasis in Tuning would be on the methodology to apply; to identify common ground among academics in an international context. Teamwork and consultation were identified as the key features of this approach and its sense for success. What had been noticed that the theoretical discourse among educational scientists about educational frameworks and typologies and new approaches regarding learning, teaching and assessment had (very) limited impact in day-to-day life in higher education institutions. University teachers might have heard of or even been acquainted with the typologies such as those of Benjamin Bloom and theories such as of John Biggs regarding learning, but these had limited to no influence in the teaching and learning process itself. In other words, advocating educational theory and methodologies developed by experts would not lead to change in how educational programmes were actually designed and implemented.
The strategy, instead, was to come-up with simple approaches and solutions for complex issues. This on the one hand meant to develop a feasible and convincing methodology to modernize higher degree programmes and on the other hand to develop guidance and acceptable models on which the reform of individual degree programmes could be built. This would require full alignment and serious commitment of large numbers of academics who could act as peers for their subject area.
To establish the basis for change, it was thought necessary to identify common points of reference. These reference points should be dynamic and allow for diversity, autonomy and flexibility. At all costs Tuning wanted to avoid harmonization of degree definitions, because it would not only hinder personal development, but most of all possibilities for employment and mobility of students and graduates. Although they might be used in setting standard for a given discipline, the term standard as such was avoided to make clear that the points of reference to develop were not set in stone. The idea was and is to update them regularly to keep reflecting present day developments and the state of the art of the discipline involved.
This philosophy thus implied the choice of the competence approach as the backbone for the Tuning initiative. This choice would allow for an open discussion for what made the core of a discipline but also its relation to and relevance for society both in terms of research outputs and type of students to graduate. As stated above, in the project outline emphasis was given to the growing role of generic competences or transferable skills (subdivided in instrumental, interpersonal and systemic ones) besides subject area competences for the dynamic society of the 21st century. Although the distinction between generic and subject specific competences was made, there was full awareness from the start that they are closely related. This implied agreement that generic competences could not seriously be developed and applied without a domain of knowledge and that — although named transferable skills — these were influenced by the subject area involved. In some cases generic competences would even be seen as subject specific ones, if they were at the core of the subject area. To illustrate this point: the competence abstract thinking, analysing and synthesizing in history is based on a different theoretical and methodological framework than for example the one used in physics or mathematics. This also applies — in general — for oral and written skills, leadership, teamwork, entrepreneurial spirit etc. because each discipline has its own academic culture and paradigm.
Those who are acquainted with the Tuning approach know its methodology. In the framework of this volume it is nevertheless useful to describe it once more, also because it developed and was enhanced over time. The basic idea is to develop — on the basis of agreed and shared reference points — so-called (inter)national competences frameworks for subject areas. For each of these competences frameworks a group of 12 to 15 international experts is established. Its members are selected from a wide range of countries and are representing their country model and higher education institution in the field involved.
The process started with a mapping process to describe the field involved and to identify the employability field(s) for which graduates are prepared (in wide terms if required) followed by a consultation of stakeholders. To prepare for the first Tuning consultation process ever a common list of generic competences was drawn up by the different groups together. This led to a list of 30 generic competences for Europe. That list was revised over time, which implied that competences were rephrased and replaced. Tuning projects in other part of the world draw up their own agreed list of generic competences.
Furthermore, each disciplinary group — originally in Europe, later in other world regions — developed its own list of so-called key subject area competence statements which should ‘frame the subject area’. This list contained on average of 25 statements. That list was established on the basis of a collection of ideas and expectations regarding degree programmes in the subject area resulting from open reflection and group discussion. On the basis of this long list consensus was sought on a short list (key competences) to be used for the consultation process. Because the focus was on competences to be covered in the subject area as a whole, the first cycle / bachelor and the second cycle / master were not distinguished. To allow for the consultation process, each university drew up a list of relevant employers for its field, a list of graduates which graduated within the last 3 to 5 years and a list of academics to be consulted. In the first European consultation round (2001-2002) no students were consulted. The Tuning America Latina project decided in 2004 to involve also students in the latter years of their degree programme. This has been standard procedure since. After finishing the consultation process, its results were analysed by the subject area group involved. This led to a redefining of the original subject specific competences lists. The next step in the process was to design academic and professional frameworks for each of the cycles, preferably formulated as descriptors and to build consensus on the most relevant competences for each of them, combining both what is common for academic recognition and what is different (the specific features).
As part of the Tuning America Latina 3 (2011-2014) project ideas about the development of competences frameworks were further refined and deepened, by introducing the concept of meta-profiles. Both reference points and meta-profiles have the aim to identify and describe the core (elements) of a discipline / subject area. However, the meta-profile approach is slightly different. While in the original approach the focus is on identifying the core or key competences, both generic and subject specific, in the second approach the focus is on the clustering of generic and subject specific competences to derive to so-called meta-competences. The interlinked groups of meta-competences then serve as the basis for defining a meta-profile (competences framework) which captures the essence of the discipline in more general terms. When the meta-profile is decided, it can be used as a basis for constructing individual degree programmes. This approach has successfully been used as the recent Tuning publication Meta-perfiles y perfiles. Una nueva aproximación para las titulaciones an América Latina shows us.
This new method which was and is being applied later in Tuning projects in other regions of the world such as Russia, Africa and Central Asia, offers us a more sophisticated way forward, because the existing template of collecting a long list of competences and then boiling it down to the more essential ones, lacked sufficient structure.
Two main approaches have been developed for the grouping of competences. The first is to cluster the most related competences in a feasible number of groups, a minimum of five to a maximum of eight. After having done so, a label for each group is decided which reflects best its content and purpose. However, it is also possible to work the other way around, defining labels for the group’s first and then using these as a basis for clustering the competences. Each group or meta-competence will contain a mix of generic and subject specific competences. This is fully in line with the Tuning philosophy, which requires that these are developed together.
VI. Definitions and Confusion of Terms
Having explored the position of the concept and the way Tuning has absorbed and applied these, it seems appropriate to offer insight into the definitions developed and/or used by Tuning over time and to position these in the current debate.
Against the background that the term and concept of competence / competences / competency / competencies were challenged, it has been important to develop clear definitions. Tuning did not have and does not have the illusion it would and will be standard setting in this respect, but at least it wanted to provide clarity for the users of its approach.
Day to day practice shows that many academics (still) have difficulties in distinguishing the concepts of competenc(i)es and learning outcomes. This also applies to learning outcomes and learning objectives. The difference between competences and skills also seems to be problematic. Although there are different appreciations of the terms in different languages and cultures, we focus here on the use of English.
The basic idea in Tuning was and is that the role of education is primarily to make the student / learner more competent as a result of a learning process. This is wider than knowledgeable and skilled, it also involves acting and ‘how to be’. This is relevant for personal development, preparing for citizenship as well as positively affecting the learner’s employability.
As was stated previously, Tuning uses an encompassing definition of competence. Competences should be understood as a dynamic representation of demonstrated knowledge, understanding/insight/comprehension, (subject specific and generic) intellectual, practical and interpersonal skills and (ethical) values. They cover the whole spectrum of capabilities from pure theoretical and methodological knowledge to vocational knowledge/insight and from research abilities to practical abilities. Fostering these competences is the object of all educational programmes. Competences are formed during the process of learning by the student in succeeding course units or modules and are assessed at different stages. Competences are therefore owned by the student / learner.
A learning outcome is understood as a statement of what a learner is expected to know, understand and be able to demonstrate after completion of a process of learning. Learning outcomes indicate the level of competence that is desired and should be achieved. They are in other words the specifications of the results and outcomes of a learning process. The learning process again is based on an identified set of competences. Learning outcomes are distinguished in degree programme learning outcomes and module and/or unit learning outcomes. Learning outcomes are defined by academic staff preferably involving student representatives.
Although, these definitions seem to be clear, Tuning has nevertheless been criticized for its use of terminology. The criticisms seem to originate from a misinterpretation of the concepts Tuning developed. Because Tuning is using the concept of competence(s) it is thought that it is embracing the traditional concept of Competence-Based Approach to Education and Training (CBET) which was developed in the 1990s. This is not the case. It is also a misunderstanding — expressed by educational scientists — that the Bologna Process is built on this approach. Tuning has drawn attention to the development of generic competences besides the subject specific ones as part of the learning process, because they are of crucial relevance for functioning in today’s society. These generic competences are intellectual as well as practical. To name a few: “identify, pose and resolve problems”; “to be critical and self-critical”; “abstract and analytical thinking and synthesis of ideas”; “generate new ideas”; “to take the initiative and to foster the spirit of entrepreneurship and intellectual curiosity”; besides “applying knowledge in practice”; “work in a team”; “evaluate and maintain the quality of work produced”. In the view of Tuning these types of competences should be developed / learned in close alignment with a body of knowledge, that is the subject area. Tuning has showed these can be developed at different stages during the learning process by using level descriptors / levels of mastery and indicators. This approach can be named Competence-based learning but this is not the same as CBET. As stated before, Tuning uses identified sets of competences for identifying the reference points of a particular subject area now named meta-profile or conceptual framework.
Therefore, it does not recognise itself in such criticisms as expressed by Hyland and Hager for example. Hager, author of the integrated competence approach states: “despite its laudable aims, a project that features such flaws in its foundations, is argued to be fatally deficient …because it fails to recognize a number of clear conceptual distinctions between, e.g. ‘performance’ and its ‘outcomes’”. What is meant here in the wording of Hager is that “...performance outcomes can be specified precisely, and that the Tuning learning outcomes are a species of performance outcomes. However, by contrast, competences cannot be specified precisely in this way. So the Tuning Project, by mistakenly equating learning outcomes and competences, gives the latter a false objectivity.” As has been shown above, Tuning is very much aware of the difference. However, it must also be recognized there is still room for improvement in formulating competences as well as learning outcomes statements. More time will be needed to build up experience.
A learning objective outlines the material the teaching staff intends to cover or the questions related to the discipline that the class will address. This approach means in practice that the focus is on the teaching process (not the learning process) and on knowledge transfer of the academic staff member to the students. However, in particular in the United States but also in other countries learning objectives may be read and understood as learning outcomes. The fact that sometimes different wording is used for the same concept or notion does not automatically mean that the terminology used is blurred.
More complicated is the use of the terms skills and competences. Tuning has not always been consistent in its use. According to the definition outlined, skills are an intrinsic part of competences. In reality the project sometimes used the terms alongside each other in its publications and presentations. This happened in particular during the first years of Tuning, when the project was still developing. However since the launch and endorsement of the European Qualifications Framework for LLL the landscape of terminology has changed. In the EQF, which is mostly VET inspired but also covers Higher Education, three main categories are distinguished to order the outcomes based descriptors: knowledge, skills and competences. According to its official definition ‘competence’ “means the proven ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development. In the context of the European Qualifications Framework, competence is described in terms of responsibility and autonomy.” Tuning would have preferred that the term ’wider competences’ would have been used as one of the descriptors to avoid misunderstandings. Although there is agreement regarding the definition of the term, at the same time it is clear we have now to deal with both a wider and a tailored meaning of it. Tuning has shown already that it is able to handle the term when developing competence based Tuning Sectoral Frameworks for the Social Sciences, the Humanities and the Performing and Creative Disciplines. It has also offered instruments to write degree programme profiles on the basis of its approach, to define good learning outcomes statements as well as an approach to calculate workload on the basis of its philosophy.
VII. Final remarks
There is no doubt that the concepts of competences and learning outcomes underpinning the student-centred approach are phenomena in today’s Higher Education. They have become tools for degree programme design, development and enhancement, as well as for quality assurance and accreditation. Although there might still be some confusion about the exact definitions of terms introduced and used, this does not imply that the multi-dimensional or integrative approach towards competence-based models has not taken the lead. It has, although there is still a long way to go to get it implemented at all levels of higher education. As might be expected many academics have difficulties coping with the changes proposed, in particular because they are so fundamental. Working according to a new paradigm requires time and effort. Although Tuning and the European Commission, in particular CEDEFOP, have developed tools to assist this change in paradigm, it is still a challenge to follow. Identifying an appropriate set of competences to define a degree programme profile is already an effort, let alone the definition of degree programme learning outcomes and programme unit learning outcomes. To make these learning outcomes measurable requires skills and experience which in many Higher Education institutions and departments still have to develop. However, it cannot be stressed enough that using the competence based approach is rewarding, in particular for the group of students educated today and tomorrow, but also for the educational staff and society at large.
Here we refer not only to employability, but explicitly also to personal development and citizenship. There is no doubt that it is rewarding for an institution, a department and the individual teachers when their graduates find a place in society that suits them well. However, this should not be limited to finding the right job and making a career according to the level of education, although that is a mission in itself.
Education has always been associated with personal growth. The opportunities of learning, of understanding how nature and society work holds a significant attraction to the human mind no less than the curiosity of managing natural forces, one’s own behaviour or the dynamics of society. The competence-based approach, promoting flexibility and individual pathways offers students of today more opportunities to follow their interest and answer their curiosity.
One of today’s challenges for those involved in planning, designing and offering degree programmes is to define precisely what the degree tries to establish and hence which competences it will need to develop in students. An important role of education is to educate students in such a way that they develop greater consciousness of their commitment to society. Who will doubt that knowledge, reflection, capacity, learning, education leading to degrees should not make people more understanding of what is happening in society at various levels, more critical of the outcomes, the limits and the impacts of issues at stake and more creative about various solutions for and possibilities to handle problems? Democratic societies require citizens who think, debate, search, and look for alternatives. The role of education in this case is undeniable.
The concept of competences and learning outcomes in the framework of the introduction of the paradigm of student-centred learning as has been developed and applied by Tuning during the last 14 years, is relevant for the higher education sector today. This article has intended to show that the choices made were not self-evident. Have these choices paid out and proven to be a panacea to reforming degree programmes, making them more relevant for the needs of today? Only time will tell.
Arguelles, A., and A. Gonczi, A. Competency Based Education and Training: a world perspective. Mexico: Editorial Limusa S.A. de CV and Grupo Noriega Editores, 2000.
Association of American Colleges. “Integrity in the college curriculum: a report to the academic community: the findings and recommendations of the Project on Redefining the Meaning and Purpose of Baccalaureate Degrees.” Washington, DC., 1985.
Barr, Robert B, and John Tagg. “From Teaching to Learning. A new paradigm for undergraduate education.” Change. The Magazine for Higher Education 27, no. 6 (November/December 1995): 13-25. http://www.athens.edu/visitors/QEP/Barr_and_Tagg_article.pdf
Beneitone, Pablo, Julia González, and Robert Wagenaar, eds. Meta-perfiles y perfiles. Una nueva aproximación para las titulaciones en América Latina. Bilbao: University of Deusto Press, 2014.
Biggs, J. Teaching for Quality Learning at University: What the Student Does. Buckingham: The Society for Research into Higher Education and Open University Press, 1999.
. Teaching for Quality Learning in Universities. Maidenhead,1999.
. “What the Student Does: teaching for enhanced learning” Higher Education Research & Development 18, no. 1 (1999): 57-75.
., and K. Collis. Evaluating the Quality of Learning: the SOLO taxonomy. New York: Academic Press, 1982.
Bloom, B.S., M.D. Engelhart, E.J. Furst, W.H. Hill, and D.R. Krathwohl. Taxonomy of Educational Objectives: the classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company, 1956.
Eraut, M. Developing Professional Knowledge and Competence. London: Routledge, 1994.
European Commission. “European Qualifications Framework (EQF).” http://ec.europa.eu/education/lifelong-learning-policy/eqf_en.htm.
. “Teaching and Learning: Towards the Learning Society”. White Paper on Education and Training. Luxembourg, European Commission, 1996. http://europa.eu/documents/comm/white_papers/pdf/com95_590_en.pdf
European Ministers for Higher Education. “Joint declaration of the European Ministers of Education.” Bologna 19 June 1999. http://www.bologna-berlin2003.de/pdf/bologna_declaration.pdf
. “ ‘Realising the European Higher Education Area’: Communiqué of the Conference of Ministers Responsible for Higher Education in Berlin on 19 September 2003 [Berlin Communiqué].” Berlin: Bologna-Berlin2003 Project Team, 2003. http://www.bologna-berlin2003.de/pdf/Communique1.pdf
Guskin, Alan E. “Reducing Student Costs and Enhancing Student Learning. The University Challenge of the 90’s — Part II: Restructuring the Role of Faculty.” Change, The Magazine for Higher Education 26, no. 5 (September/October 1994): 16-25.
Hager, P. “Some Conceptual Questions about the Tuning Project.” Prospero 12, no. 1 (2006).
Hyland, Terry .“Swimming against the tide: reductionist behaviourism in the harmonisation of European higher education systems.” Prospero 12, no. 1 (2006).
Hodkinson, P., and Issitt, M.. The challenge of competence. London: Cassell, 1995.
Hutmacher, Walo. “Key Competencies in Europe.” European Journal for Higher Education, 32, no. 1 (1997).
Klemp, G.O. “The assessment of Occupational Competence. Report to the National Institute of Education.” Washington, 1980.
Kothik, Rudi. “Process Oriented Teacher Education as an Alternative to the Competence-Based Approach to Education and Training”. In Curating the European University. Exposition and Debate, edited by M. Simons, a.o., 73-80. Leuven: University Press, 2011.
McClelland, D.C. “Testing for Competence Rather Than for Intelligence.” American Psychologist 28, no. 1 (January 1973): 423-447. http://www.therapiebreve.be/documents/mcclelland-1973.pdf.
“Meeting of the OECD Education Ministers, Paris 3-4 April 2001; Investing in Competencies for all (Communiqué).” http://oecd.org/dataoecd/48.24/1870589.pdf.
Mulder, M., T. Weigel, and K. Collins, “The concept of competence concept in the development of vocational education and training in selected EU member states. A critical analysis.” Journal of Vocational Education and Training 59, no.1 (2006): 65-85. http://www.mmulder.nl/wp-content/uploads/2011/11/2007-01-19-Mulder-Weigel-Collins-JVET.pdf
National Institute of Education (ED). “Involvement in Learning: Realizing the Potential of American Higher Education. Final Report of the Study Group on the Conditions of Excellence in American Higher Education.” Washington, DC., October 1984. http://eric.ed.gov/?id=ED246833.
Norris, N. “The trouble with competence.” Cambridge Journal of Education 21, nº 3 (1991): 331-341.
OECD. “Definition and Selection of Competencies: Theoretical and Conceptual Foundations (DeSeCo). Bakground paper.” Revised December 2001.
. “The Definition and Selection of Key Competencies.” Executive Summary. http://www.oecd.org/pisa/35070367.pdf.
Parry, S.R. “The Quest for Competence.” Training Magazine (July 1996): 48-56.
Radelet, Steven, and Jeffrey Sachs. “The Onset of the East Asian Financial Crisis.” Harvard Institute for International Development, 30 March 1998. http://scholar.google.nl/scholar_url?hl=nl&q=http://academiccommons.columbia.edu/download/fedora_content/download/ac:124144/CONTENT/paper27.pdf
Rychen, Dominique S., and Laura H. Salganik, eds. Defining and Selecting Key Competencies. Göttingen, 2001.
Scott, Peter. The Meanings of Mass Higher Education, Buckingham: Open University, 1995.
Schlusmans, K., R. Slotman, C. Nagtegaal and G. Kinkhorst, G. “Competentiegerichte leeromgevingen.” Utrecht: Lemma B.V., 1999.
Serrano, Rosario Mérida, Julia Angulo Romero, Manuel Jurado Bolo and José Diz Pérez. “Student Training in Transversal Competences at the University of Cordoba.” European Educational Research Journal 10, nº 1 (2011): 34-52.
Spencer, L.M., and S.M Spencer. Competence at Work: Models for Superior Performance. New York: John Wiley and Sons, 1994.
Sultana, Ronald G. “Competence and competence frameworks in career guidance: complex and contested concepts.” International Journal for Educational and Vocational Guidance 9, no. 1 (2009): 15-30.
Tagg, John. The Learning Paradigm College. Bolton, MA, USA: Anker Publishing Company, 2003.
Tuning Educational Structures in Europe, Final Report. Pilot Project — Phase 1. Bilbao and Groningen: University of Deusto Press, 2003)4.
. (Tuning Europe). http://www.unideusto.org/tuningeu/.
. Tuning SQFs for the Humanities and the Arts. Final Report 2010-2011. Bilbao, 2012: http://www.unideusto.org/tuningeu/sqf-humanities-and-arts.html.
. “Tuning SQF for the Social Sciences.” Final Report 2008 — 2009. Bilbao: University of Deusto, 2012: http://www.unideusto.org/tuningeu/tuning-sqf-social-sciences.html.
Van den Bosch, H., and R. Gerritsen. “Het verwerven van conceptuele competentie als doelstelling van wetenschappelijk onderwijs.” Tijdschrift Voor Het Hoger Onderwijs 15 (1997): 365-389.
Van de Wiel, Iris. “The Russian Crisis 1998. Economic Report.” Economic Report, Rabobank, Economic Research Department, September 16, 2013. https://economics.rabobank.com/publications/2013/september/the-russian-crisis-1998/
Villa Sánchez, Aurelio, and Manuel Poblete Ruiz, eds. Competence-based learning. A proposal for the assessment of generic competences. Bilbao: University of Deusto Press, 2008.
Westera, W. “Competences in education: A confusion of tongues.” Journal of Curriculum Studies 33, nº 1 (2001): 75—88.
Winterton, Jonathan, Françoise Delamare - Le Deist, Emma Stringfellow. Typology of knowledge, skills and competences: clarification of the concept and prototype. Cedefop Reference series; 64. Luxembourg: Office for Official Publications of the European Communities, 2006).
World Bank. “Country Note F. Lessons and Controversies From Financial Crises in the 1990s.” In Economic Growth in the 1990s: Learning from a Decade of Reform, World Bank, 242-251. Washington, D.C.: World Bank Group, 2005. http://www1.worldbank.org/prem/lessons1990s/
 “Bologna Declaration,” http://www.bologna-berlin2003.de/pdf/bologna_declaration.pdf.
 “Berlin Communiqué,” http://www.ehea.info/Uploads/Declarations/Berlin_Communique1.pdf
 Steven Radelet and Jeffrey Sachs, “The Onset of the East Asian Financial Crisis,” Harvard Institute for International Development, 30 March 1998, http://scholar.google.nl/scholar_url?hl=nl&q=http://academiccommons.columbia.edu/download/fedora_content/download/ac:124144/CONTENT/paper27.pdf; Iris van de Wiel, “The Russian Crisis 1998” (Economic Report, Rabobank, Economic Research Department, September 16, 2013), https://economics.rabobank.com/publications/2013/september/the-russian-crisis-1998/; World Bank, “Country Note F. Lessons and Controversies From Financial Crises in the 1990s”, in Economic Growth in the 1990s: Learning from a Decade of Reform, World Bank (Washington, D.C.: World Bank Group, 2005), 242-251; http://www1.worldbank.org/prem/lessons1990s/
 European Commission, “Teaching and Learning: Towards the Learning Society” (White Paper on Education and Training, Luxembourg, European Commission, 1996), 1, http://europa.eu/documents/comm/white_papers/pdf/com95_590_en.pdf
 Peter Scott, The Meanings of Mass Higher Education (Buckingham: Open University, 1995).
 Tuning Educational Structures in Europe, Final Report. Pilot Project — Phase 1 (Bilbao and Groningen: University of Deusto Press, 2003), 24.
 Robert B. Barr and John Tagg, “From Teaching to Learning. A new paradigm for undergraduate education,” Change. The Magazine for Higher Education 27, no. 6 (1995): 13-25. http://www.athens.edu/visitors/QEP/Barr_and_Tagg_article.pdf
 Alan E. Guskin, “Reducing Student Costs and Enhancing Student Learning. The University Challenge of the 90’s — Part II: Restructuring the Role of Faculty,” Change 26, no. 5 (1994): 16-25.
 John Tagg, The Learning Paradigm College, ix-xii.
 National Institute of Education (ED), “Involvement in Learning: Realizing the Potential of American Higher Education. Final Report of the Study Group on the Conditions of Excellence in American Higher Education” (Washington, DC., 1984), 127, http://eric.ed.gov/?id=ED246833; Association of American Colleges, “Integrity in the college curriculum: a report to the academic community: the findings and recommendations of the Project on Redefining the Meaning and Purpose of Baccalaureate Degrees” (Washington, DC., 1985), 47.
 Robert B. Barr and John Tagg, “From Teaching to Learning”.
 M.T. Mulder, Weigel and K. Collins, “The concept of competence in the development of vocational education and training in selected EU member states. A critical analysis,” Journal of Vocational Education and Training 59, no.1 (2006). The Oxford English Dictionary suggests the first use was by Shakespeare in Henry IV (1597) in the sense of ‘a sufficiency of’ and E. Burke, French Review (1790, 291), in the sense of ‘capacity to deal with a subject’.
 David McClelland, “Testing for Competence Rather Than for Intelligence,” American Psychologist January (1973): 1-14, http://www.therapiebreve.be/documents/mcclelland-1973.pdf.
 P. Hodkinson and Issitt, M., The challenge of competence (London: Cassell, 1995).
See also: M. Eraut, Developing Professional Knowledge and Competence (London: Routledge, 1994); L.M. Spencer and S.M Spencer, Competence at Work: Models for Superior Performance (New York: John Wiley and Sons, 1994); H. van den Bosch and R. Gerritsen, “Het verwerven van conceptuele competentie als doelstelling van wetenschappelijk onderwijs,” Tijdschrift Voor Het Hoger Onderwijs 15 (1997): 365-389; K. Schlusmans, R. Slotman, C. Nagtegaal. and G. Kinkhorst, G. Competentiegerichte leeromgevingen (Utrecht, Lemma B.V., 1999). See also the References as included in the chapter “Learning Outcomes: Competences”, Tuning Educational Structures in Europe, Final Report. Pilot Phase 1. Deusto and Groningen, 2003, 94-98.
 G.O. Klemp, The assessment of Occupational Competence (Report to the National Institute of Education. Washington, 1980), 21.
 S.R. Parry, “The Quest for Competence,” Training Magazine (July 1996): 50.
 Walo Hutmacher, “Key Competencies in Europe,” European Journal for Higher Education 32, no. 1 (1997): 45.
 Tuning Educational Structures in Europe, Final Report. Pilot Project — Phase 1 (Bilbao and Groningen: University of Deusto Press, 2003), 24.
 Walo Hutmacher, “Key Competencies in Europe,” 45.
 Ronald G. Sultana, “Competence and competence frameworks in career guidance: complex and contested concepts,” International Journal for Educational and Vocational Guidance 9, nº 1 (2009): 18. See also W. Westera, “Competences in education: A confusion of tongues,” Journal of Curriculum Studies 33, nº 1 (2001): 75—88.
 Sultana, “Competence and competence frameworks in career guidance,” 19-20. See also W. Westera, “Competences in education: A confusion of tongues,” Journal of Curriculum Studies 33, nº 1 (2001): 75—88.
 OECD, “Definition and Selection of Competencies: Theoretical and Conceptual Foundations (DeSeCo). Bakground paper” (revised December 2001), 6. See also: Dominique S. Rychen, and Laura H. Salganik, eds., Defining and Selecting Key Competencies. Göttingen, 2001.
 Jonathan Winterton, Françoise Delamare - Le Deist, and Emma Stringfellow, “Typology of knowledge, skills and competences: clarification of the concept and prototype” (Cedefop Reference series; 64. Luxembourg, Office for Official Publications of the European Communities, 2006 ).
 Annemarie Knottnerus en Chiel Renique, “Domeincompetenties in het HBO als sleutel voor herkenbaarheid en flexibiliteit,” Thema 5, nº 4 (2004), http://www.vno-ncw.nl/SiteCollectionDocuments/Cmsdocs/domeincompetenties.pdf
 J. Biggs, Teaching for Quality Learning at University: What the Student Does (Buckingham: The Society for Research into Higher Education and Open University Pres. Maidenhead, 1999); J. Biggs, “What the Student Does: teaching for enhanced learning,” Higher Education Research & Development 18, nº 1 (1999): 57-75; J.B. Biggs and K. Collis, Evaluating the Quality of Learning: the SOLO taxonomy (New York: Academic Press, 1982); B.S. Bloom, ed., Taxonomy of Educational Objectives, the classification of educational goals — Handbook 1: Cognitive Domain (New York: McKay, 1956).
 Julia Gonzalez and Maria Yarosh, “Building degree profiles: the Tuning approach,” Tuning Journal for Higher Education 1, no. 1 (2013): 37-69.
 Terry Hyland, “Swimming against the tide: reductionist behaviourism in the harmonisation of European higher education systems,” Prospero 12, no. 1 (2006); Rudi Kothik, “Process Oriented Teacher Education as an Alternative to the Competence-Based Approach to Education and Training,” in Curating the European University. Exposition and Debate, eds. M. Simons et al. (Leuven: University Press, 2011), 74.
 Aurelio Villa Sánchez and Manuel Poblete Ruiz, eds., Competence-based learning. A proposal for the assessment of generic competences (Bilbao: University of Deusto Press, 2008).
 Terry Hyland, “Swimming against the tide”.
 P. Hager, “Some Conceptual Questions about the Tuning Project,” Prospero 12, no. 1 (2006).
 Definition used by the University of Toronto, http://www.teaching.utoronto.ca/topics/coursedesign/learning-outcomes/outcomes-objectives.htm
 The following example is taken from the University of Texas to illustrate this: “A competency is the capability to apply or use a set of related knowledge, skills, and abilities required to successfully perform “critical work functions” or tasks in a defined work setting. Competencies often serve as the basis for skill standards that specify the level of knowledge, skills, and abilities required for success in the workplace as well as potential measurement criteria for assessing competency attainment. (…). Competencies define the applied skills and knowledge that enable people to successfully perform their work while learning objectives are specific to a course of instruction. Competencies are relevant to an individual’s job responsibilities, roles and capabilities. They are a way to verify that a learner has in fact learned what was intended in the learning objectives. Learning objectives describe what the learner should be able to achieve at the end of a learning period. Learning objectives should be specific, measurable statements and written in behavioral terms. In short, objectives say what we want the learners to know and competencies say how we can be certain they know it.” University of Texas School of Health, https://sph.uth.edu/content/uploads/2012/01/Competencies-and-Learning-Objectives.pdf
 University of Texas School of Health, https://sph.uth.edu/content/uploads/2012/01/Competencies-and-Learning-Objectives.pdf. Another interesting example of a learning objective definition is the following. The example is taken from the University of New Mexico School of Medicine: A learning objective is an outcome statement that captures specifically what knowledge, skills, attitudes learners should be able to exhibit following instruction. A common misapplication of objectives is for the teacher/presenter to state what he/she is going to do (e.g., “My plan this morning is to talk about...”), rather than what the student is expected to be able to do (e.g., “After this session, you should be able to...”). http://ccoe.rbhs.rutgers.edu/forms/EffectiveUseofLearningObjectives.pdf
 “Tuning Sectoral Qualifications Framework for the Social Sciences,” http://www.unideusto.org/tuningeu/tuning-sqf-social-sciences.html; “Tuning Sectoral Qualifications Frameworks for the Humanities and the Arts,” http://www.unideusto.org/tuningeu/sqf-humanities-and-arts.html
 Rosario M. Serrano et al., “Student Training in Transversal Competences at the University of Cordoba,” European Educational Research Journal 10, no. 1 (2011): 47.
Copyright for this article is retained by the Publisher. It is an Open Access material that is free for download, distribution, and or reuse in any medium only for non-commercial purposes; provided any applicable legislation is respected, the original work is properly cited, and any changes to the original are clearly indicated.