Tuning Educational Structures in Europe: Line 1: Learning Outcomes: Competences

Executive Summary
Why focus on competences?
The questionnaire
References
Annexes

The work on Line 1 of the Tuning project reflects the importance of focusing on competences alongside knowledge in joint reflection at the level of European universities. These are seen in the context of an adequate response to the paradigm of a primarily student-centred education, and as contributing towards a clearer definition of academic and professional profiles. Providing sets of dynamic descriptors, they add transparency to the debate on learning outcomes and hence facilitate mobility.

Mutual trust and confidence have been distinctive features of European cooperation. They are intimately linked with transparency. So is quality, which could be related with transparency of purpose, of processes and of outcomes. In each of these three the reflection and the identification of academic and professional competences may add an aspect of consistency. They are also relevant for the enhancement of employability, active citizenship and personal development in lifelong learning.

Also, in the creation of the European Higher Education Area, the joint study of competences together with knowledge by European universities will contribute to the development of easily readable and comparable degrees, and a system essentially based on two main cycles. Furthermore, the joint debate on the nucleus of competences and the articulation of levels and programmes by European networks can clearly enrich the European dimension of Higher Education. It also builds on the consistency of systems of accreditation by increasing information on learning outcomes, and contributes to the development of common frameworks of qualifications, hence promoting understanding, clarity and the attractiveness of the European Higher Education Area. Besides, an increase in transparency of learning outcomes and processes will definitely be a further asset for the encouragement and enhancement of mobility.

The Tuning project consulted with graduates, employers and academics in 7 subject areas (Business, Education Sciences, Geology, History, Mathematics, Physics and Chemistry), from 101 university departments in 16 European countries, by means of questionnaires, to which a total of 7,125 people responded (comprising 5,183 graduates, 944 employers and 998 academics). This is not to mention the informal teamwork, reflection and debate provoked at the level of departments, disciplines and countries. The objectives of this consultation were to initiate joint debate and reflection at institutional, subject area, and European levels, starting from a base of updated information reflecting the reality of the current situation. The consultation dealt with both generic and subject-specific skills and competences.

Thirty generic competences were selected from three categories: instrumental, interpersonal and systemic. Respondents were asked to rate both the importance and the level of achievement by educational programmes in each competence, and also to rank the five most important competences. The questionnaires were translated into 11 official languages and sent by each participating institution to 150 graduates and 30 employers of graduates in their subject area. The questionnaire for academics was prepared based on 17 competences judged most important by graduates and employers. For each of the competences, the respondents were asked to indicate: the importance of the skill or competence for work in their profession and the level of achievement of the skill/competence that they estimate they have reached as a result of taking their degree programme.

Consultation with stakeholders and joint reflection on up-to-date information and at European level is regarded as necessary for elaboration and reformulation of degree programmes.

Based on a preliminary analysis of the results, some initial conclusions can be drawn.

One of the most striking conclusions is the remarkable correlation between the ratings given by employers and those of graduates.

In relation to importance, these two groups consider that the most important competences to be developed are: capacity for analysis and synthesis, capacity to learn, problem solving, capacity for applying knowledge in practice, capacity to adapt to new situations concern for quality, information management skills, ability to work autonomously and teamwork.

Looking at the other end of the scale, there appear: understanding of cultures and customs of other countries, appreciation of diversity and multiculturality, ability to work in an international context, leadership, research skills, project design and management, and knowledge of a second language. One striking aspect is the concentration of the "international" competences in the lower part of the scale with respect to importance. This opens a number of questions which would need further analysis.

In relation to achievement, the items which appear highest in the scale, in the opinion of the graduates are: capacity to learn, basic general knowledge, ability to work autonomously, capacity for analysis and synthesis, information management skills, research skills, problem solving, concern for quality and will to succeed. Six of these items coincide with those that graduates and employers considered important and ranked highest in the scale. The remaining reflect the tasks which the universities have traditionally been performing for centuries.

Looking at the bottom of the scale, the competences are: leadership, understanding of cultures and customs of other countries, knowledge of a second language, ability to communicate with experts in other fields, ability to work in an international context, and ability to work in an interdisciplinary team. It is remarkable that these competences, with the exception of knowledge of a second language, all appear near the bottom of the table for importance. A number of open questions remain, which call for further reflection.

But it is the level of subject-related competences where Tuning makes perhaps its greatest contribution, since that subject-related competences are crucial for identification of degrees, for comparability and for the definition of first and second degree cycles. Each of the groups has identified a list of competences related to their subject and consulted with other academics to reflect on the relative importance of these competences and their best location at the level of first and second cycle. Because of the close relationship between this reflection and knowledge, this analysis appears in line

2- Why focus on competences?

The Tuning project considers that the development of competences in educational programmes can significantly contribute to opening an important area of joint reflection and work at university level in Europe about:

1. the new educational paradigm;
2. the need for quality and the enhancement of employability and citizenship;

the creation of the European Higher Education Area.

1. Competences in the development of the new educational paradigm

   The world is nowadays characterised by rapid change. A series of general factors such as globalisation, the impact of information and communication technologies, the management of knowledge and the need to foster and managed diversity, among others, make for a significantly different environment for education. Any reflections on the future developments of education must be placed in this context. The challenges of this change and the nature of these forces, as well as the speed with which they take place, have been widely documented in the literature and referred to by European Fora, International Organisations, and papers of the European Commission.

   A change is taking place in the teaching/learning paradigm, where approaches centred on the learner are increasingly important.

   The "society of knowledge" is also a "society of learning". This idea is intimately linked with the understanding of all education in a wider context: the continuum of lifelong learning, where the individual needs to be able to handle knowledge, to update it, to select what is appropriate for a particular context, to learn permanently, to understand what is learned in such a way that it can be adapted to new and rapidly changing situations.

   The need to recognize and value learning could also be seen as having an impact on qualifications and on the building of educational programmes leading to degree qualifications. In this context, the consideration of competences side by side with the consideration of knowledge offers a number of advantages which are in harmony with the demands emerging from the new paradigm.

   Change and variety of contexts both require a constant check on social demands for professional and academic profiles. This underlines the need for consultation, and constant revision of information on adequacy. The language of competences, since it comes from outside higher education, could be considered more adequate for consultation and dialogue with groups not directly involved in academic life, and can contribute to the necessary reflection for the development of new degrees and for permanent systems of updating existing ones.

   In the reflection on academic and professional profiles, competences emerge as an important element which can guide the selection of knowledge which is appropriate to particular ends. It presents an integrative capacity to choose what is appropriate from a wealth of possibilities.

   Trends are complex, often discontinuous processes whose effects on actors vary. However, the trend towards a "learning society" has been widely accepted and consolidated for some time. This involves a move from teaching-centred to learning-centred education. Reflecting on the different aspects which characterise this trend, the relevance of focusing on competences becomes apparent. The previous paradigm involved an emphasis on the acquisition and transmission of knowledge. Elements in the changing of this paradigm include: education centred on the student, the changing role of the teacher, further definition of objectives, change in the approach to educational activities, shift in from input to output, and a change in the organization of learning. Each of these elements will be discussed in turn.

   The interest in the development of competences in educational programmes is in accordance with an approach to education as primarily centred on the student and his/her capacity to learn, demanding more protagonism and higher quotas of involvement since it is the student who ought to develop the capacity to handle original information and access and evaluate information in a more varied form (library, teacher, internet, etc.)

   This relates implicitly with the changing role of the teacher, from being the structurer of knowledge, the key player in the teaching and articulation of key concepts, as well as the supervisor and director of work of the student, whose knowledge he/she assesses. A student-centred vision gives the teacher more of an accompanying role, so that the learner attains certain competences. While the role continues to be critical, it shifts more and more towards containing higher levels of advice, counselling and motivation in relation to the importance and place of areas of knowledge, understanding and capacity to apply that knowledge, in relation to the profile which needs to be attained, personal interests, gaps and capacities, critical selection of materials and sources, organization of learning situations, etc.

   The emphasis on students getting a particular competence or set of competences may also affect the transparency in the definition of objectives set up for a particular educational programme, adding indicators with higher possibilities for being measured accountably, while making these objectives more dynamic in taking into consideration the needs of society and employment.

   This shift normally relates to a change in the approach to educational activities, teaching material and a great variety of educational situations, since it fosters the systematic involvement of the learner with individual and group preparation of relevant issues, presentations, organized feedback, etc.

   Besides, the shift in emphasis from input to output is reflected in student evaluation, moving from knowledge as the dominant (even the single) reference to include assessment centred on competences, capacities and processes closely related to work and activities as related to student development and in relation to academic and professional profiles already defined, also showing a greater wealth of assessment strategies (portfolio, tutorial work, course work...) as well as taking into consideration situational learning.

   Finally, different ways of participating in education (full time, part time...) changing contexts and diversity also affect the pace or rhythm at which individuals and groups can take part in the educational process. This also has an impact not only on the form and structure of programme delivery but in the whole approach to the organization of learning, including more focused programmes, more short courses, more flexible course structures, and more flexible delivery of teaching, with the provision of more guidance and support.

2. Competences, the search for quality and the enhancement of employability and citizenship.

   In the Salamanca Convention quality was considered as a fundamental foundation, the basic underlying condition for trust, relevance, mobility, compatibility and attractiveness in the European Higher Education Area.

   While compatibility, mobility and attractiveness will be dealt with in relation to the creation of the European Higher Education Area, it is important to look briefly into the role of education by competences, relevance of degree programmes as indicators of quality.

   Mutual trust and confidence have been distinctive features of European cooperation. These are intimately linked with transparency. So is quality, which could be related with transparency of purpose, of processes and of outcomes. In each of the three, the reflection and the identification of academic and professional competences may add an aspect of quality and consistency.

   Relevance in the context of the Salamanca Convention relates particularly to employability, which needs to be reflected in different ways in the curricula "depending on whether the competences acquired are for employment after the first or the second degree." Employability, in the perspective of lifelong learning, is considered as best served through a diversity of approaches and course profiles, the flexibility of programmes with multiple exit and entrance points and the development of generic competences.

   In fact, the relationship between reflection and work on competences and employment is a longstanding one. The search to find a better way to predict successful performance in the work place, moving beyond measurements of intelligence personality and knowledge is often regarded as the initial point. This emphasis on work performance continues to be of vital importance.

   From the perspective of the Tuning Project, learning outcomes go beyond employment to contain also the demands and standards that the academic community has set in relation to particular qualifications. But employment is an important element. In this context competences and skills can relate better and may help to prepare graduates for crucial problems at certain levels of employment, in a permanently changing economy. This needs to be one of the points of analysis in the creation of programmes and units through constant reflection and evolution.

   The consideration of education for employment needs to run parallel with education for citizenship, the need to develop personally and to be able to take social responsibilities and, according to the Council’s follow-up report to the Lisbon Convention, facilitating the access of all to education.

3. Competences and the creation of the European Higher Education Area
The focus on competences in the Tuning Project is closely linked with the creation of the European Higher Education Area, and very explicitly with the Bologna process and the Prague Communiqué.

In relation to a system of easily readable and comparable degrees aimed at facilitating academic and professional recognition so that citizens can use their qualifications through the European HE Area, the introduction of Line 1 in Tuning sought to provide comparability and readability in reference to the competences (generic or subject-related) that the graduates from a particular degree aimed at attaining. In fact, the capacity to define which competences a programme seeks to develop, or what its graduates should be able to know, understand and do, cannot but add a further dimension to the degree transparency. They can also contribute to the development of both better-defined degrees, and systems of recognition that are "simple, efficient and fair", "reflecting the underlying diversity of qualifications" since competences add angles and levels, selecting knowledge appropriate to a particular profile. This works in favour of diversity.

As regards the adoption of a system essentially based on two main cycles:

The identification and initial discussion by a European body of academics of a set of subject-related competences for first and second cycle could be considered one of the major contributions of the project. In connection with knowledge, this is crucial for the development of European points of reference which could be considered common, diverse and dynamic in relation to specific degrees and the creation of frameworks of reference for clarification and further understanding of the relationship and nature of the qualifications.

Following on from this, joint reflection and work on competences and skills is an extremely important element in the work towards common standards and profiles for recognized joint degrees. Furthermore, the joint debate on the nucleus of competences and the articulation of levels and programmes by European networks can clearly enrich the European dimension of HE. It also builds on the consistency of systems of accreditation by increasing information on learning outcomes, and contributes to the development of common frameworks of qualifications, hence promoting understanding, clarity and the attractiveness of the European Higher Education Area.

An increase in transparency of learning outcomes and processes will definitely be a further asset for the encouragement and enhancement of mobility. Information which takes into consideration objectives expressed in the language of competences will present a more holistic perspective on the programme, but hopefully also will develop a systematic approach to each of the units in terms of the capacities which they will hope to gain. However, the specific contribution that Tuning in general and Line 1 in particular seek to offer relates particularly to the mobility of professionals and degree holders all over Europe, and has often been referred to as vertical mobility: the movement of graduates to take the second cycle of their studies in another country. In this respect the contribution of Tuning to the Diploma supplement is of great relevance.

3- The questionnaire

In the Tuning Project the debate on each of the lines follows one of the many different approaches possible. For the debate on skills and competences a questionnaire was proposed.

1. The objectives

The objectives of the questionnaire included:

• The wish to initiate the joint discussion on this field of competences and skills at the European level, based on consultation with groups from outside academia (graduates and employers) as well as from a broader base in relation to academics (beyond Tuning representatives from each of the subject areas involved).

• The attempt to gather updated information for reflection on possible trends and the degree of variety and change all over Europe.
• The desire to start from the experience and the reality in order to reach levels of diversity or commonality between the different countries, starting the debate from specific questions with concrete language.
• The importance of initiating the reflection and debate at three different levels: the institutional level (the basis and the first one to take place), the subject area level (a reference point for the HE institutions) and the aggregate level (a second reference point in relation to the situation at European level).

1. The content of the questionnaire

Several terms: capacity, attribute, ability, skill, competence,... are used with an often interchangeable, and to some degree overlapping meaning. They all relate to the person and to what he/she is able of achieving. But they also have more specific meanings. Ability, from the Latin "habilis" meaning "able to hold, carry or handle easily", led to the word "habilitas" which can be translated as "aptitude, ability, fitness or skill."

The term skill is probably the most frequently used, with the meaning of being able, capable or skilful. It is often used in the plural, "skills," and sometimes with a more restricted meaning than that of competences. This explains the choice of the term competences in the Tuning Project. In the questionnaire to the graduates and employers, however, the two terms "skills" and "competences" appear together for a more encompassing meaning.

Competences tend to convey meaning in reference to what a person is capable or competent of, the degree of preparation, sufficiency and/or responsibility for certain tasks.

In the Tuning Project, the concept of competences tries to follow an integrated approach, looking at capacities via a dynamic combination of attributes that together permit a competent performance or as a part of a final product of an educational process. This also links with the work done in HE. In Line One, competences and skills are understood as including knowing and understanding (theoretical knowledge of an academic field, the capacity to know and understand), knowing how to act (practical and operational application of knowledge to certain situations), knowing how to be (values as an integral element of the way of perceiving and living with others and in a social context). Competences represent a combination of attributes (with respect to knowledge and its application, attitudes, skills and responsibilities) that describe the level or degree to which a person is capable of performing them.

In this context, a competence or a set of competences mean that a person puts into play a certain capacity or skill and performs a task, where he/she is able to demonstrate that he/she can do so in a way that allows evaluation of the level of achievement. Competences can be carried out and assessed. It also means that a person does not either possess or lack a competence, but commands it to a varying degree, so that competences can be placed on a continuum.

In the Tuning Project two different sets of competences were focused on: Firstly, those competences which are subject-area related. These are crucial for any degree and they are intimately related to specific knowledge of a field of study. They are referred to as academic-subject-related skills and competences. These give identity and consistency to the particular degree programme.

Secondly, Tuning tried to identify shared attributes which could be general to any degree, and which are considered important by particular social groups (in this case former graduates and employers). There are certain attributes like the capacity to learn, the capacity for analysis and synthesis, etc, which are common to all or most of the degrees. In a changing society where demands tend to be in constant reformulation, these general skills or competences also become very important. The first questionnaire tried to identify these so-called generic skills and competences and how they were valued, first by graduates and employers and then in the second questionnaire (first part), by academics.

This paper deals with the generic skills and competences, since subject-related competences have been analysed with an approach which was deemed adequate to the subject by the relevant groups of expert.

In the design and redesign of educational programmes, it is crucial that the University takes into consideration the changing needs of society as well as present and future employment possibilities. While these are not the unique consideration for the development of study programmes and degrees, they are of vital importance.

Obviously the list of competences and skills identified and able to be reflected upon is vast. The choice of a number of items to be included in a questionnaire is always partial and debatable. In order to prepare the questionnaire for graduates and employers a review of over twenty studies in the field of generic skills and competences was carried out. A list of 85 different skills and competences was identified. They were regarded as relevant by institutions of Higher Education or companies. These items were categorised as instrumental, interpersonal and systemic. There is obviously a number of different possible classifications and it is recognized that no classification is perfect. Therefore some competences could be seen as bridging the categories. This is a possible working classification:

• Instrumental Competences: Those having an instrumental function. They include:

• Cognitive abilities, capacity to understand and manipulate ideas and thoughts.
• Methodological capacities to manipulate the environment: organising time and strategies of learning, making decisions or solving problems.
• Technological skills related to use of technological devices, computing and information management skills.
• Linguistic skills such as oral and written communication or knowledge of a second language.

• Interpersonal Competences: Individual abilities relating to the capacity to express one’s own feelings, critical and self-critical abilities. Social skills relating to interpersonal skills or team-work or the expression of social or ethical commitment. These tend to favour processes of social interaction and of co-operation

• Systemic competences: those skills and abilities concerning whole systems. They suppose a combination of understanding, sensibility and knowledge that allows one to see how the parts of a whole relate and come together. These capacities include the ability to plan changes so as to make improvements in whole systems and to design new systems. Systemic competences require as a base the prior acquisition of instrumental and interpersonal competences.

The distribution of the competences mentioned in the sources consulted (without considering the frequency of repetitions of the same competence), based on the aforementioned typology, was as follows:

• Instrumental Competences (38%)
• Interpersonal Competences (41%)
• Systemic Competences (21%)

Looking at the frequency and trying to amalgamate related concepts the percentage changed, as follows:

• Instrumental Competences (46%)
• Interpersonal Competences (22%)
• Systemic Competences (32%)

It was interesting to note that interpersonal competences represented the greatest percentage in terms of the number of different competences (41%). However, since they appeared excessively varied and were not well-determined, when analysed by frequency, this percentage went down to 22%. It seemed that instrumental competences were well delimited and coincide across many different approaches; for instance, technological competence (understood as use of a personal computer) or linguistic competence (oral and written communication).

On the other hand, interpersonal competences are very dispersed. They refer to personal aspects (self-concept, self-confidence, locus of control, etc.) or interpersonal aspects as varied as assertiveness, interpersonal communication, face-to-face style, social commitment, etc.

In April, 2001 a draft of the first questionnaire for graduates and employers was prepared. Time constraints limited the participation of members in the initial stage of the questionnaire’s design, although this would be desirable on future occasions. This initial draft tried to propose a balanced representation of competences from all three groups: instrumental, interpersonal and systemic. The provisional questionnaire was discussed at the first Tuning meeting and some items were changed by the Tuning members. Some groups also added competences more directly related to their subject area. (Mathematics, History and Education Science.)

In May 2001, these suggestions were incorporated and the definitive questionnaire was prepared. Also incorporated, in both graduate and employer questionnaires, was a series of variables for identification considered important to the study.

The definitive questionnaires comprised the following 30 competences:

• Instrumental competences

• Capacity for analysis and synthesis
• Capacity for organisation and planning
• Basic general knowledge
• Grounding in basic knowledge of the profession
• Oral and written communication in your native language
• Knowledge of a second language
• Elementary computing skills
• Information management skills (ability to retrieve and analyse information from different sources)
• Problem solving
• Decision-making

• Interpersonal competences

• Critical and self-critical abilities
• Teamwork
• Interpersonal skills
• Ability to work in an interdisciplinary team
• Ability to communicate with experts in other fields
• Appreciation of diversity and multiculturality
• Ability to work in an international context
• Ethical commitment

• Systemic competences

• Capacity for applying knowledge in practice
• Research skills
• Capacity to learn
• Capacity to adapt to new situations
• Capacity for generating new ideas (creativity)
• Leadership
• Understanding of cultures and customs of other countries
• Ability to work autonomously
• Project design and management
• Initiative and entrepreneurial spirit
• Concern for quality
• Will to succeed

Other interesting competences could have been included, for example "teaching ability". This would perhaps have provided a relevant perspective in relation to a significant sector of employment. The responses of employers might also have been affected by the use of the word "advanced" rather than "basic" in relation to knowledge or grounding in the profession. The former might have been given a higher rank.

The questionnaires were translated into the 11 official languages of the EU by Tuning members. Each of the Universities sent and received back the questionnaires from their graduates and employers and sent them on to University of Deusto where the questionnaires were processed.

Each of the Universities got back its own data file by e-mail and the graphs for the total and the different subject areas. By agreement and for confidentiality reasons, no graph or analysis was made at central level in relation to individual universities. Each university was expected to do the institutional analysis, and reflection at local level and bring this to the area group. They could, also, compare their own data with total and area results.

1. Procedure

The procedure requested of the coordinators at the participating universities with respect to the selection of the different samples was as follows:

• Every university participating in the study had to sample a total of 150 graduates.
• The graduates selected were to have graduated within the last 3 to 5 years.
• This criterion depended on the number of graduates that had graduated in this period, as well as the professional destinations of the graduates.
• If there were few graduates each year, the sample would include those graduating within the last 5 years. If there were a large number, then the sample would be limited to those graduating in the last 3 years. In those few cases where there were not enough graduates from the participating institution, graduates from other similar institutions in the same country were included.
• In relation to the professional destinations of graduates, given that the study was most interested in graduates who already were working, where graduates entered the world of work rapidly after graduation, the sample could be chosen among those who had graduated in the last 3 years. Otherwise, when graduates took longer to join the world of work, it was recommended to select the sample from those who had graduated in the last 5 years.
• The criterion of selection of the 150 graduates was at random. It was recommended that if there existed an association of graduates with an updated database of addresses, the selection was made by the above mentioned association. In this way we would avoid having questionnaires returned because they were sent to an out-of-date address.
• The corresponding university sends the questionnaires to its graduates with a letter in which, as well as presenting the questionnaire, it asks them to send it by return to the university within the space of 10 days.
• The questionnaire and the letter of introduction are sent along with a stamped addressed envelope for the return of the questionnaire.

• Every university participating in the study has to gather information from 30 employers.
• The criterion of selection was that they should be organisations which the university knew to employ its graduates, and/or organisations which in spite of not having proof that they had employed graduates of the university, seemed likely to be interesting places of work for these graduates. Within these guidelines, universities were free to select whatever employers they through appropriate. It has been suggested that a tighter control on the balance of different types of employers might have been exercised so as to obtain more representative results. However, this remains an open question.
• The corresponding university sent the questionnaires to the employers with a letter which, beside presenting the questionnaire, asked them to return it within 10 days.
• The questionnaire and the letter of introduction were sent along with a stamped addressed envelope for the return of the questionnaire.

• Every participating university gather information from, at least, 15 academics in the area in which the subject university was participating.
• Each university sent the academics a questionnaire in electronic form that they were asked to return within seven days.

1. Type of Response Requested

The questionnaires required two types of response:

1. Importance / Level of Achievement
2. Ranking the five competences considered most important

For each of thirty competences, the respondents were asked to indicate:

• The importance of the skill or competence, in his/her opinion, for work in their profession and
• the level of achievement of the skill/competence that they estimate they have reached as a result of taking their degree programme.

To indicate this respondents were asked to use a scale of 1 = none to 4 = strong.

Asking about both aspects (importance and level of achievement) responds to the interest in finding where their institution stands in terms of thirty competences arranged into four categories, represented in the diagram below:

Diagram 1. AIR (Martilla and James, 1997)

• Concentration: that is to say, competences that are considered very important but in which there is little achievement.
• Low priority: competences which are not considered very important and in which achievement is low.
• Excess effort: competences that are not considered very important but in which achievement is high.
• Maintain: competences that are considered important and in which achievement is high.

The importance of the chart is that it may help reflection and discussion at institutional level finding out the weak and strong points which could help to build policy (a matter of choice for the institution); to strengthen the weaker parts or even to get stronger at the strong points. What was really crucial was to place the development of system of consultation in context with the environment, and also to have the capacity to create systems which can help to develop joint strategies at the European level.

Ranking: As well as indicating the importance and level of achievement of each of the 30 competences, both groups (graduates and employers) were asked to indicate, in order, the five competences that they considered to be most important.

Commonly when people are asked to value the importance of different aspects of life, this valuation tends to be high. In general, the tendency is to value things as important, which can reasonably be considered as such, but without discriminating excessively between them. As we were conscious that this could happen in the case of competences, it seemed suitable to request that respondents would choose the five most important competences and rank them in order of importance. These two pieces of information, importance and ranking, seemed important for the work.

The questionnaire sent to academics, was divided into two parts:

The first part related to generic competences. The objective was to obtain a third perspective on generic skills and competences to compare with those of graduates and employers.

The content was based on the results obtained in the study of graduates and employers. Depending on this information, it was observed that there was a high level of agreement between graduates and employers on the 11 competences considered as most important by both groups. These 11 competences were included in the questionnaire sent to academics, as well as 6 others also considered as very important by graduates and employers. Academics were asked to rank these 17 competences in order of importance, in their opinion.

The second part of the questionnaire dealt with specific, subject-related competences.

The objective of this part was to find the first response, from a broader base of academics from the relevant areas, to the work done by each of the groups of Tuning experts trying to identify subject-related competences and to relate them to either first or second cycle of studies in their particular field.

The difficulty of this task is clearly understood by the Tuning members. Equally clear is the understanding that what is at stake is the development of reference points which, understood only as such and in a dynamic framework, could be of vital importance in the development of the European HE Area.

It may be considered that competences are always linked with knowledge but in the case of subject-related competences, this connection is even closer. The joint reflection at European level on what is common, diverse and dynamic, together with the identification of levels, is a crucial step towards the understanding and consequently the building of degrees, which can be taken and used throughout Europe.

The content of the second part of the academics’ questionnaire was prepared by the Tuning working groups in the different areas. Despite of the fact that the questionnaire for each area was different, the way of responding was common. Respondents were asked, for each of the competences, to gauge the level of importance that it had, in their opinion, in both the first and second cycle.

The aim of both questionnaires was, as explained above, that of initiating joint reflection, so its main achievement needs to be considered as provoking reflection and debate. It is also important to note that the processes were conceived as having, as the bottom line of the joint discussion, the reflection that each of the Tuning participants brought to the group from his or her own institution, where the questionnaire results had the best context for interpretation. This objective affected the type and form of data collected.

1. Participants in the questionnaire

A total of 101 out of a total of 105 university departments participating in the Tuning Project took part in the consultation. The choice of universities in the Tuning Project was a very complex process where the interest, the size of the country and the criteria of the local conference of Rectors had a place.

The data was first meant to be analysed at the level of the institution, to provide the maximum degree of meaning. Also the two indicators seemed different in this context. While the opinion on achievement seems very important at institutional level, particularly in relation to the graduates, it can be regarded more as a perception as it relates to aggregate data or in relation to the employers. However looking at importance it is questionable the degree to which the graduates, or even more employers, related to a particular institution or whether instead they responded to the degree of importance they attached to the particular item in terms of its relation to work or development. Hence, importance is at the base of the reflection.

Specifically, seven subject areas took part in the research: Business, Education Sciences, Geology, History, Mathematics, Physics, and Chemistry, in relation to graduates, employers and academics.

In each of these areas the following number of universities were invited to participate:

• Business: 15 universities, of which 14 participated
• Geology: 14 universities
• History: 17 universities and an international network for the study of university teachers (CLIOHNet)
• Mathematics: 15 universities, of which 13 participated
• Physics: 14 universities
• Education: 15 universities, of which 14 participated
• Chemistry: 15 universities, of which 14 participated

The data relating to the sample participating in the study are presented below.

Total

Although the intention of the consultation was to initiate a joint dialogue with social groups and the debates followed at institutional and subject area level could be considered the best results, the valuable work of 101 universities and the volume of data collected (5,183 questionnaires from graduates, 944 from employers and 998 from academics) deserve an attempt at some treatment for further reflection.

1. Methodology

The sample design was clustered, as respondents are clustered within Universities. Therefore assumptions of simple random sampling may not be valid as respondents are not strictly independent from each other. At the same time, Universities may show some cluster effect at country level.

Clustered design is widely used in research and does not represent by itself a source of bias. Cluster sampling affects the survey sampling error of any estimate produced. The sampling error is increased depending on differences in measured items among clusters.

Based on data, this design effect due to cluster sampling may be estimated by intracluster correlation: high intracluster correlation indicates that differences among clusters are high, and therefore increases the survey sampling error. It should be noted that low intracluster correlation in any item, near to zero, indicates that a simple random sample would have produced similar results.

In relation to the results of the Tuning Questionnaire on generic skills and competences simple random sampling estimates and procedures were avoided in either univariate or multivariate analysis. All estimates and conclusions take into account the clustered nature of data at both University and country level through multilevel modelling.

It was regarded as the most appropriate approach since multilevel models take into account the clustered structure of data (i.e. does not assume that observations are independent as in simple random sampling). These models have been widely used on educational data as their clustered structure, students within educational institutions, is always present.

At the same time multilevel modelling allows simultaneous modelling of individual and cluster level differences providing adequate estimates of standard errors and making appropriate any inference at both individual and cluster level.

In this context clusters are not regarded as a fixed number of categories of a explanatory variable (i.e. the list of selected universities as a fixed number of categories) but it considers that the selected cluster belong to a population of clusters. At the same time yields better estimates at individual level for groups with few observations.

Three different types of variables are analysed:

• Importance items: 30 competences rated on importance by respondents (Graduates and Employers)
• Achievement items: 30 competences rated based on achievement (Graduates and Employers)

Ranking: based on the ranking of the five most important competences provided by graduates and employers, a new variable was created for each competence. For each respondent the corresponding competence was assigned five points if it was the first selected competence, four if it was the second one, etc... and finally one point if it was selected in the fifth place. If the competence was not chosen by the respondent, zero points were assigned. For the academics, who had to rank a longer list of seventeen competences out of the previous thirty rated by graduates and employers, this ranking was created using a similar transformation applied to a seventeen points scale: seventeen was assigned if the competence was chosen first, sixteen to the second competence, etc...

1. Results

   Graduates

   Intracluster correlations (Table 1, Table 2) indicate to what extent universities are different from each other and the effect of clustered observations on sampling errors. The highest intracluster correlation is for Knowledge of a second language both as importance (0,2979) and achievement (0,2817). The next highest two are Elementary computing skills-Achievement (0,2413) and Ethical commitment-Importance (0,1853). From the list of items regarding importance, 21 out of 30 show intracluster correlations lower than 0.1 and from the list of items regarding achievement the proportion goes to 10 out from 30. Results seem consistent: when graduates rate universities, they seem to be more in terms of achievement than importance.

   Means for all items were calculated taking into account the intracluster correlation using multilevel models for each item with no explanatory variables and allowing a random intercept for each level. At this stage three levels were considered: country, university and final respondent. Therefore the intercept in the model yielded the mean for each item with adequate estimates of the sampling error for each estimate.

   The results are shown in Table 3, Table 4 and Table 5. These results were displayed as confidence intervals (1-±=95%) in Figure 1, Figure 2 and Figure 3.

   Employers

   For the data collected from employers a similar analysis was performed. Multilevel modelling showed that the country effect – employers belonging to same country- seems stronger than the university effect -employers belonging to same university in the data collection process- compared to graduates as it would be expected. Means for all items were again calculated using multilevel models as it was done before.

   The results are shown in Table 6, Table 7 and Table 8. These results were displayed as confidence intervals (1- ± =95%) in Figure 4, Figure 5 and Figure 6.

   Comparing Graduates with Employers

   Importance ratings for Graduates and Employers were compared using again multilevel modelling adding a parameter to the model accounting for the difference between both groups. Thirteen items showed a significant difference (a <0,05). The highest difference corresponds to Ethical commitment with Employers rating this item higher than students. It is interesting to note that employers rate Ability to work in an interdisciplinary team significantly higher than graduates while in the case of Ability to work autonomously the case is just the opposite graduates rating this item higher than employers. These results are shown in Table 9.

   Table 9. Significant differences in importance items. Employers vs. Graduates

Label	Description		Difference Employers vs. Graduates	a
imp28	Ethical commitment	Employers higher than Graduates	0,3372	0,00%
imp20	Ability to work in an interdisciplinary team		0,1463	0,00%
imp27	Initiative and entrepreneurial spirit		0,0979	0,07%
imp17	Teamwork		0,0957	0,04%
imp29	Concern for quality		0,0838	0,11%
imp25	Ability to work autonomously	Graduates higher than Employers	-0,1591	0,00%
imp8	Elementary computing skills		-0,1559	0,00%
imp9	Research skills		-0,1104	0,09%
imp3	Capacity for organisation and planning		-0,0900	0,04%
imp5	Grounding in basic knowledge of the profession		-0,0822	0,62%
imp11	Information management skills		-0,0739	0,35%
imp15	Problem solving		-0,0554	1,80%
imp16	Decision-making		-0,0552	3,51%

 

If the rankings of importance items obtained from each group are compared some interesting patterns are observed. This comparison is obtained joining Tables 3 and 6 as shown in Table 10.

Table 10. Importance items ranking. Employers vs. Graduates

The correlation between both rankings is quite strong (Spearman correlation = 0.899) and shows some common groups of items at both extremes of the ranking. In order to create a combined ranking, groups of items were created for both graduates and employers so that any pair of items in the same group showed non significant difference in the importance rating mean. In this manner ten groups were created in the graduates ranking and seven in the employers ranking. Each item received the mean rank of the group in which it was included and finally the mean was calculated for each item using the mean rank of the graduates list and the mean rank of the employers list. This procedure created a ranking of eighteen levels where some of the items were tied (Table 11) which perhaps seems like a more adequate manner to present final results when such groups are to be compared.

Table 11. Combined ranking. Graduates & Employers

Academics

The academics were asked to rank seventeen items selected from the thirty item list given to graduates and employers. It is true that some respondents reported that it was somewhat difficult to give a specific ranking to certain items as they seemed equally important. The adequacy of ranking versus weighting in this context is debatable and the difficulty has been well understood. This is often the case when a long list of items has to be ranked but it is clear that given that all academics faced this same difficulty – and therefore some of the positions in the ranking were given somehow at random within a specific range- aggregate results should show this same close positions in the final ranking (and no significant differences between the ranking of these items as it will be shown in results).

A numerical variable was created for each item assigning seventeen points if the item was ranked in the first place, sixteen if it was ranked in the second place and so on. The mean of this variable for each item was estimated again by multilevel modelling as it is shown in Table 12 and Figure 7. Table 12 displays the items in descending order and therefore creating again a ranking of items. Given that the order is given just by the estimation, the mean differences between items were analysed in order to find if differences were significant. In this manner eight different groups of items were created so that any possible pair of means in the group showed no significant difference. Within each group the ranking of items could be considered interchangeable at some extent.

Table 12. Academics

In order to compare the academics ranking to the previous ones, the thirteen items not present in the academics list were deleted from the graduates, employers and combined graduates-employers rankings and these rankings were reconstructed using seventeen ordered positions. The result is shown in Table 13.

The most striking difference is that academics rank Basic general knowledge in the first position of the list (although it should be remembered that shows no significant difference compared to the second ranked Capacity for analysis and synthesis) while both graduates and employers tend to rank this same item in the twelfth position. Spearman correlations are shown in Table 14 showing that employers and graduates rankings tend to be more similar among them than the academics ranking. Compared to graduates, most relevant differences are Elementary computing skills (fourth position for graduates and sixteenth for academics) and Interpersonal skills (sixth for graduates and fourteenth for academics). Compared to employers, most relevant difference is again Interpersonal skills (fifth for employers and fourteenth for academics).

Table 14. Spearman correlations

 

Country Effects

Multilevel modelling allows the estimation of what could be considered a country effect, this is, a measure of the effect of the country as a whole on respondents. This effect was measured on the thirty importance items rated by graduates. The country effect was classified in three groups: strong effect (there are strong differences between countries), mild effect (the differences are weaker) and no effect (all countries seem to be equal). This classification is shown the following table.

A graphic display for the items with a strong country effect are shown in Figures 8 to 14.

Figures 15 to 17 display the same graphic for items where the country effect was non significant so the reader is able to compare the different graphic patterns between significant and non significant country effects.

 

1. Initial Conclusions and Open Questions

The importance of the Tuning Project is to promote debate and reflection on competences at the European level, from a university perspective and from a subject area approach, offering a way forward. The level of reflection and development of competences and skills in the definition and development of university degrees in Europe is varied according to traditions and educational systems.

Another element in Tuning is that competences and skills are always linked with knowledge since it is understood that they can not be developed without learning in some field or discipline.

In this context and from the work and the debate done by the Tuning members, a number of initial conclusions can be drawn, while significant questions remain open to be dealt with in future work.

1. With regard to the importance of competences:

• The development of competences and skills fits in well with the paradigm of primarily student-centred education. It emphasises that the student, the learner is the focus, and thus brings into discussion the changing role of the teacher. This is regarded as moving towards more of an accompanying role, guiding learning towards the attainment of particular well-defined objectives. It consequently affects the approach to educational activities and the organisation of learning, which shifts to being guided by what the learner needs to achieve. It also affects assessment in terms of shifting from input to output and to the processes and the contexts of the learner. However, how the competences are to be worked, realized and assessed and the impact of this change, both at individual level and at the level of European university structures, needs further reflection and debate

• The definition of academic and professional profiles in degrees is intimately linked with the identification and development of competences and skills towards their attainment throughout the curricula. To reach this aim, the work of isolated academics is not sufficient, it needs to be approached in a transversal way through the curricula of a particular degree programme.

• Transparency and quality in academic and professional profiles are major assets in relation to both employability and citizenship, and the enhancement of quality and consistency as a joint effort should be a priority for the European Institutions. The definition of academic and professional profiles and the development of the fields of required competences, add quality in terms of focus and transparency, purpose, processes and outcomes. In this context, the use of the language of competences at the level of the Diploma Supplement would be a quality step along both fronts.

• The use of competences and skills (together with knowledge) and the emphasis on outputs adds another important dimension to balance the weight given to the length of study programmes. This is particularly relevant for lifelong learning.

• In relation to the creation of the European Higher Education Area, the joint reflection, debate and attempts to define subject area competences as dynamic reference points could be of crucial importance for the development of easily readable and comparable degrees, for the adoption of a system essentially based on two main cycles and for the enhancement of mobility, not only of students, but particularly of graduates and professionals.

1. In relation to the practice of consultation with social groups before elaboration or reformulation of degree programmes, the Tuning members have observed a variation among the European Universities in the levels at which this practice is carried out. Also they observe a significant variety in the methods used for this consultation. In this respect, the Tuning members agree that the practice of consulting relevant social and professional groups is crucial and should be encouraged using the most appropriate form and manner in each case.

• In the case of Tuning, the groups consulted were the most relevant ones: graduates, employers, and academics. Obviously, other groups could have been consulted as well. The relevance and possibility of other types of contributions remains an open question.

• The Tuning members also agree that joint reflection from the Universities based on updated data is important in the development of adequate degrees. Echoing the Salamanca convention they recognise that students need and demand qualifications which they can use effectively for the purpose of their studies and careers all over Europe. This demands not only a reflection on what local social and professional groups value and demand from their programmes but also the perspective of broader trends taking place at the European level.

1. It is important to remember that subject-related competences are crucial for identification of degrees, for comparability and for the definition of first and second degree cycles. These competences have been analysed individually by the subject area groups. The identification and initial discussion of a set of subject-related competences for the first and second cycle could be considered one of the major contributions of the project towards the development of European points of reference.
2. With regard to generic competences in a changing society where professional profiles need to be well defined while keeping a dimension of openness to change and adaptation, some messages from graduates and employers to European Universities can be identified:

• In relation to the importance given to different competences, the messages from graduates and employers are of crucial relevance. In fact, one of the most striking results of the questionnaire is the very high degree of correlation between the opinion of graduates and employers in relation to the importance and rank given to the different competences enumerated.

• These two groups consider that the most important competences to be developed are: capacity for analysis and synthesis, capacity to learn, problem solving, capacity for applying knowledge in practice, capacity to adapt to new situations concern for quality, information management skills, ability to work autonomously and teamwork.
• Looking at the other end of the scale, there appear: understanding of cultures and customs of other countries, appreciation of diversity and multiculturality, ability to work in an international context, leadership, research skills, project design and management, and knowledge of a second language. One striking aspect is the concentration of the "international" competences in the lower part of the scale with respect to importance. This opens a number of questions which would need further analysis.

• In relation to achievement in terms of the competences that the universities are considered to develop at the highest level, again there is a high level of correlation between the employers and the graduates. However, in this respect reference is only made to the graduates since it is considered that these would have the most accurate perspective.

• The items which appear highest in the scale, in the opinion of the graduates are: capacity to learn, basic general knowledge, ability to work autonomously, capacity for analysis and synthesis, information management skills, research skills, problem solving, concern for quality and will to succeed. Six of these items coincide with those that graduates and employers considered important and ranked highest in the scale. The remaining reflect the tasks which the universities have traditionally been performing for centuries.

• Looking at the bottom of the scale, the competences are: leadership, understanding of cultures and customs of other countries, knowledge of a second language, ability to communicate with experts in other fields, ability to work in an international context, and ability to work in an interdisciplinary team. It is remarkable that these competences, with the exception of knowledge of a second language, all appear near the bottom of the table for importance.
• A wider reflection on these results is necessary. There are several questions: Whether the items reflected in the questionnaire are the right ones. What is the rate of change developing in the five years gap since the first and the last graduates would have finished their degree programmes. Whether there are competences which relate to emerging needs... etc. The importance of looking at the future and trying to anticipate developments.

• The scale of appreciation of the graduates and employers also has a high degree of coincidence with the ranking by the academics with a few exceptions

• The first exception is the rank given to basic general knowledge, which for the graduates and employers shows a level of 12 out of 18 while for the academics it appears in first place. One point to note is that responses to questions involving the word basic may depend on the interpretation given to this word, which could change depending on the inclusion of questions referring to advanced knowledge.
• The second item of difference is elementary computing skills. This varies between groups, being considered more important by graduates, less by employers and least by academics.
• The third is interpersonal skills with much higher importance attached by graduates and employers (level 6) than by academics where it appears in a considerably lower position. In general, all the interpersonal skills tend to rank lower for academics than for graduates and employers. The majority of the competences which appear at the top of the scale both in terms of importance and achievement are instrumental and systemic.

• However, in relation to the issue of generic skills, several questions remain open. They include: is there a core of generic skills which may be identified and jointly developed? How many could be developed in a degree programme? Should the choice of competences be based on the different degrees or should they be characterised by institutional choices and institutional strengths? Who should be responsible for them? Which are the most adequate methods for developing them through the curricula?

• Finally, as regards the variation of ranking and the impact by country, there are 13 items were there is no variation at all. Among them there are three of the competences which appeared at the top of scale and also two of those at the bottom. Seven items showed a significant country effect and demand further thinking.

These are only initial conclusions of a joint reflection at European level on the potential that competences have in the creation of the European Higher Education Area and in the enhancement of Higher Education as a whole.

There are a number of open questions for further study and reflection: Questions related to employment potential for graduates, the gaps between importance and achievement in a more detailed way and starting from closer to the institutional level, the emerging needs of society, and future demands, and the changing nature of learning as it needs to take place in a variety of contexts.

Adam, S. (2001), Transnational Education report, Brussels: CEURC.

Agudín Y ( 2000). La Educación Superior para le siglo XXI. Didac nº 36 16-25

Argüelles, A.(1996): Competencias laborales y educación basadas en normas de competencia. México: Limusa.

Argüelles, A.(1997): Formación basada en competencias laborales. México: Limusa.

Bloy, S. and Williams, J. (2000). "Using the national key skills framework within a higher education context". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Boletín Educaweb (2001). Formar las competencias profesionales. Boletin Educaweb. 12 de Marzo de 2001, número 71. [www document]: URL: http://www.educaweb.com/esp/servicios/boletin/but010312/editorial.asp

Boyatzis, R. E. (1982) The Competent Manager, John Wiley & Sons.

Bryk, A.S. and Raudenbusch, S.W. (1992). Hierarchical Linear Models: Applications and Data Analysis Methods. Sage Publications.

Confederation of European Union Rectors' Conferences, 2001. Transnational Education Project Report and Recomendations.

Council of the European Union, 2001. The concrete future objectives of education and training systems

Crump, S. et al. (1996). Evaluation report: NSW key competences pilot project. Sidney: University of Technology Sidney.

Dalton, M. (1998): ¿Sirven de algo los modelos de competencias?,en Training & Development Digest, 24-26.

Davis, D. et al. (1998). Mid-program assessment of team-based engineering design: concepts, methods and materials. Washington: Washington State University.

De la Harpe, B. and Radloff, A. (2000). "Helping academic staff to integrate professional skills". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Draper, D. (1995) Inference and hierarchical modelling in the social sciences. Journal of Education and Behavioral Statistics 20, 115-147.

EUA European University Association, 2001. Salamanca Convention 2001

European Commission (1995). White Paper: Teaching and learning, towards the learning society, DG XXII Education Training and Youth, Brussels, 101 p

European Commission (1997a). Towards a Europe of knowledge, Communication from the Commission, COM(97) 563 final, 10 p

European Commission (2000). Commission staff working paper. A memorandum on lifelong learning

Fallows, S. and Steven, C. (2000). "The skills agenda". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Fallows, S. and Steven, C. (2000). "Embedding a skills programme for all students". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Fraser, S. and Deane E. (1998). Doers and thinkers: an investigation of the use of open-learning strategies to develop life-long learning competences in undergraduate science students. Canberra: Dept. of Employment, Education, Training and Youth Affairs.

Fundación Universidad-Empresa (1999). Guía de las empresas que ofrecen empleo.

Goldstein, H. (1992). Statistical information and the measurement of education outcomes (editorial). Journal of the Royal Statistical Society, A, 155: 313-15.

Goldstein, H. (1995). Multilevel Statistical Models. London, Edward Arnold: New York, Halstead Press.

Goldstein, H. And Spiegelhalter, D. (1996). League tables and their limitations: Statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society, Series A 159, 385-443.

Goldstein, H., Rasbash, J., Yang, M., Woodhouse, G., Pan, H., Nuttall, D., and Thomas, S. (1993). A multilevel analysis of school examination results. Oxford Review of Education, 19: 425-33.

Goleman, D. (1995). Emotional Intelligence. New York: Bantam.

González, L., de la Torre, A y de Elena, J.(1995): Psicología del trabajo y de las organizaciones. Gestión de RR.HH. y nuevas tecnologías. Eudema, Salamanca.

Gonzi, A., Hager, P. And Athanascu, J. (1993). The development of competence-based assessment strategies for the professions. Camberra: Australian Government Publishing Service.

Hakel, M.D. and McCreery, E.A.W. (2000). "Springboard: student-centred assessment for development". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Harling, J. (2000). "Creating incurable learners: building learner autonomy through key skills". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Haug, G & Tauch, Chr. (2001), Trends in learning structures in higher education (II). Follow-up report prepared for the Salamanca and Prague conferences fo March/May 2001, Helsinki: National Board of Education

HayGroup (1996). Las Competencias: Clave para una gestión integrada de los recursos Humanos. Bilbao: Ediciones Deusto.

Heywood, L. et. al. (1993). Guide to development of competence-based standards for professions. Camberra: Australian Government Publishing Service.

INCANOP(1997): La formació al segle XXI. Les competencies clau. Institut Catalá de Noves Professions, Barcelona.

Jenkins, A. (2000). "It´s a long hard road!". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Laybourn, P., Falchikov, N., Goldfinch, J. and Westwood, J. (2000). "Evolution of skills development initiatives". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Leeds Metropolitan University. Skills for Learning. [www document]. URL: http://www.shu.ac.uk/keytokey/lmucontents.htm

Levy-leboyer, C.(1997): Gestión de las competencias. Gestión 2000, Barcelona.

Mallea, J (1998), International trade in professional and educational service: implications for the professions and higher education (Paris, OECD-CERI) http://www.oecd.org/els/papers/papers.htm]

Marrelli, A.F.(1998): Introducción al análisis y desarrollo de modelos de competencias, en Performance Improvement, Mayo-Junio, 36-43.

Martilla, J. and James, J. (1977). "Importance-performance analysis". Journal of Marketing, 41, 77-79.

McClelland, D. C. (1973). Testing for competence rather than intelligence. American Psychologist, 28 (1), 1-14

McLagan, P.(1998): La nueva generación de competencias, en Training & Development Digest. 13-20.

Melton R. (1997). Objectives, competences and learning outcomes: developing instructional materials in open and distance learning. London: Kogan Page in association with the Institute of Educational Technology, Open University.

Mercer Co.(1995): Competencies, Performance and Pay. William Mercer Companies, N.Y.

Mertens, L.(1996): Competencia laboral: sistemas, surgimiento y modelos. Montevideo: Cinterfor.

Miklos, T.(1999): Educación y capacitación basada en competencias. México: Limusa.

Milne, C. (2000). "Tertiary literacies: integrating generic skills into the curriculum". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

Moloney, K.(1998): ¿Es suficiente con las competencias?, en Training & Development Digest, 55-61.

Monereo, C. y Pozo, J.I. (2001). "¿En qué siglo vive la escuela?; El reto de la nueva cultura educativa". Cuadernos de Pedagogía, 298, 50-55.

Murray, M.D., Cuebiro, J.C. y Fernández, G.(1996): Las competencias: Clave para una gestión integrada de los RR.HH. Deusto, Bilbao.

Nunan, T., George, R. and McCausland, H. (2000). "Implementing graduate skills at an Australian university". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

O`Brien, K. (2000). "Ability-based education". In S. Fallows and C. Steven. Integrating key skills in higher education; employability, transferable skills and learning for life. London: Kogan Page.

OCDE (2000). Measuring student Knowledge and Skills: The PISA 2000 assessment of Reading, Mathetaical and Scientificic Literacy.

Ordóñez, M.(1995): La nueva gestión de los RR.HH. Gestión 2000, Barcelona.

Ordóñez, M.(1997): Modelos y experiencias innovadoras en la gestión de los RR.HH. Gestión 2000, Barcelona.

Pereda, S. y Berrocal, F.(1999): Gestión de los RR.HH. por competencias. Centro de Estudios Ramón Areces, Madrid.

Prahalad, C.K. y Hamel, G.(1990): The Core Competence of the Corporation, en Harvard Business Review, Mayo-Junio, 79-92.

Prieto José M, 2002 Prólogo. Levy-Leboyer Claude: Gestión de las Competencias, Gestión 2000, SA, Barcelona

The Quality Assurance Agency for Higher Education (2000). The national qualifications framework for higher education qualifications in England, Wales and Northern Ireland: Education studies. [WWW document] http://www.qaa.ac.uk/crntwork/benchmark/education.pdf

The Quality Assurance Agency for Higher Education, 2001. The national qualifications framework for higher education qualificaitons in England, Wales and Nothern Ireland: a position paper

The Scottish Credit and Qualifications Framework, 2001. An Introduction to The Scottish Credit and Qualifications Framework

Sheffield Hallam University. Key Skills On-line. [www document]. URL: http://www.shu.ac.uk/keytokey/shucontents.htm

Smith H, Armstrong M, Brown S. (1999). Benchmarking and threshold standards in Higher Education. London: Kogan page.

Snijders, T.A.B. And Bosker, R.J. (1999). Multilevel Analysis. An introduction to basic and advanced multilevel modeling. Sage Publications.

Spencer, L.M. y Spencer, S.M.(1993): Competence at work. Modeles for superior performance. Wiley, N.

Spencer, L.M., McClelland, D.C. y Spencer, S.M.(1994): Competency Assesment Methods. History and State of the Art. Hay-McBer Research Press, Boston.

Thomas Edward, (2000). Increading lifelong learning in European Higher Education: the challenges and the prospects. In F2000 European Higher Education Expert Forum, Brussels, 24-25 January 2000

University of Deusto (2000), internal document about competences.

Van Damme Dirk,(1999). Internationalization and quality assurance: towards worldwide accreditation? Paper commissioned for the IAUP XIIth Triennial Conference, Brussels.

Van Damme Dirk,(2001). Higher Education in the age of Globalisation: The need for a new regulatory framework for recognition, quality assurance and accreditation. Introductory Paper for the UNESCO Expert Meeting Paris

Van den Berghe, W.(1997): La calidad de la enseñanza y formación profesional en Europa: cuestiones y tendencias. CEDEFOP, Salónica.

Vargas, F.; Casanova, F. Y Montanaro, L.(2001): El enfoque de competencia laboral: manual de formación. Montevideo: Cinterfor.

Villa Aurelio, 2001. Marco pedagógico de la Universidad de Deusto

Williams Peter, 2002. QAA. Council of Universties. Transparency for European Higher Education. Madrid

Wright P. (1995) Draft paper "Identifying the Notion of Graduateness" HEQC Quality Enhancement Group, London.

Annexes

1- Employability Questionnaries

2- Questionnaire for academics (word document)

3- Tables (word document)

4- Graphics (word document)