Government reform of Higher EducationWritten evidence submitted by the Economics Network, Bristol University
Mechanisms to Promote Teaching Quality in HE
This paper examines incentives to improve teaching quality in higher education.
It first examines the current situation and identifies some levers which are or have been used with mixed success.
It then goes on to identify three categories of lever: regulation; competition and market mechanisms; developing communities of practice with the culture of sharing innovation and good practice.
The paper argues that there is considerable scope within each of these three categories for enhancing teaching quality.
1. The current situation
1.1 Teaching is given a lower priority in most universities than research. Although there has been some improvement in teaching quality there is massive scope for improvement. Some improvements have already come about as a result of:
1.2 Despite these developments, teaching is still given a relatively low priority in many universities and the current incentive mechanisms drive this situation. Research output is relatively easy to measure with well-established metrics, such as journal rankings and citations, and with peer review at the heart of the research assessment/ evaluation process. It is difficult, but not impossible, to provide a parallel process to drive improvements in teaching quality, particularly if, like research, teaching quality is peer-assessed in a disciplinary context.
2. Options for improving teaching quality: regulation
2.1 Regulation: inspection. Teaching quality is less easy to measure than research quality and thus regulation which hinges on indicators is likely to be flawed and open to manipulation. This was the central problem with the QAA’s subject review process. University departments played a game of targeting the indicators in order to receive a maximum of 4 out of 4 for each of the six indicators. In many/most cases, it was the score they were after rather than genuine quality improvements. The asymmetry of information between internal staff and the visiting team allowed the team to be hoodwinked on many occasions:
2.3 Regulation: training. University staff could be required to undertake both initial training and CPD activities that lead to some form of qualification. The former has been taking place and most universities have established courses for new lecturers. The biggest weakness here, as argued above, is that they are generic and are dismissed by many lecturers as largely irrelevant to their needs as teachers of a specific discipline or specialism within that discipline. There is considerable scope here for building on the work of the subject centres to provide that discipline specificity and relevance:
3. Options for improving teaching quality: competition and market mechanisms
3.1 There is considerable competition between universities in terms of research quality and there is a thriving market in research academics. There are two key reasons for this: first, there is transparency in terms of publications and well-established indicators, allowing league tables to be easily constructed (although, it must be noted, the new impact measure in the REF will be much more problematic); second, research output is prioritised over teaching quality, despite teaching income exceeding research income in the vast majority of cases.
3.2 How can competition be created in teaching? An important “market” failure here is the asymmetry of information. People outside a university have little knowledge of what goes on behind closed classroom doors, other than what they hear from student friends or sons/daughters at university. Even within a university, provided that there are no student complaints, colleagues, including heads of department, often have little knowledge of the quality of teaching of their fellow lecturers. What knowledge they do have is often anecdotal or based on crude student satisfaction surveys. Current league tables reflect data that poorly reflect the quality of learning and teaching.
3.3 Another market failure is perversity of incentives. The few incentives related to teaching quality are often related to a poor proxy for quality, such as student satisfaction surveys or pass rates. Students often like being spoon-fed with teaching directed to a highly predictable exam. Lecturers behaving in this way may be giving their students a poor quality learning experience and yet might get high satisfaction ratings. Teaching which challenges students with difficult learning tasks may be unpopular with students at the time but may well produce better (and more grateful) graduates with skill sets that make them more employable. As far as pass rates are concerned, easy and predictable assessment is likely to give higher pass rates, but again a poor quality learning experience based on rote learning. The incentive to produce easy and popular courses is stronger when the modules are optional ones. Lecturers clearly want students to choose their options – after all, they can use the same materials as last year and will not have to prepare for alternative modules. If they make these options easy to pass, this information will quickly spread to the next cohort of students.
3.4 Mechanism 1: increased information in the public domain. One way of increasing information would be to require universities to publish various teaching-related statistics by department as well as an aggregate for the university. These could include: funding received for teaching and funding received for research; staff time allocated to teaching and research (this information does normally exist as “hours” is the normal metric for allocating staff duties); student contact time by class type; class sizes; student/staff ratios; the proportion of classes taken by graduate teaching assistants; the proportion of classes taken by visiting lecturers (i.e. part-time hourly-paid lecturers); the nature of the learning opportunities and experiences available to students; assessment regimes; progression rates; entry qualifications; retention rates; proportions of students receiving each degree classification; CPD activities undertaken by staff. Although these indicators are often too crude to be used for regulation, they provide useful information for prospective students, parents and employers and could put universities in significant competition with each other.
3.5 Mechanism 2: teaching awards. There has been some development here in terms of (a) national teaching fellows (NTFs), (b) subject-centre awards to lecturers for excellent teaching and (c) institutional awards and teaching fellowships:
3.6 Mechanism 3: Open Educational Resources. Most lecturers now use the virtual learning environment adopted by their university. The main ones are Blackboard, WebCT and the freeware product Moodle. This means that students can now access course materials – including syllabuses, previous assessments, lecture slides, notes, etc. – through a password-protected website. This, however, has reduced competition between universities, as materials that might previously have been on an open website can no longer be seen other than by the students on a particular module and their lecturer. They are even hidden from other students and lecturers within the university:
3.7 Mechanism 4: clearer and better structured career routes through teaching excellence. Just as universities compete for top researchers, so it is desirable to create an equivalent market for top teachers. One part of the solution here is for clear progression routes for lecturers within universities linked to teaching excellence. This has a number of requirements. First, there must be senior appointments for teaching-only or teaching-predominantly staff. Some universities have already instituted such routes, with teaching fellows, senior teaching fellows and associate professors as a progression ladder. Second, there must be transparent criteria used for promotion, such as student evaluations, student performance, peer review, staff appraisal, scholarly output related to teaching, teaching innovations, work with subject communities outside the university, winning of funding from outside bodies such as subject centres or professional associations. Third, there must be educational leadership in universities that values teaching.
3.8 Mechanism 5: stronger feedback loops between student evaluation of staff and changed staff practice. The market element here is an internal one within a university or department. Too often, feedback from surveys of students at module level is not acted on and lecturers are not held accountable for their performance. Whilst student surveys in themselves may give limited information on the quality of teaching and learning, as alluded to above, they can be backed up by consideration at staff/student committee level, where lecturers will be held accountable, not only to students, but to colleagues too. Evaluation can then form the basis of quality enhancement activities and the redesign of teaching for subsequent years. Student focus groups can be used to establish good practices as well as practices that do not work so well. Involvement of students in course planning is empowering and is likely to increase student pressure for genuine enhancement of teaching quality.
3.9 Mechanism 6: alumni surveys. Students who at the time like easy and predictable courses with spoon-feeding, may look back after they have graduated and be very critical of them. They may have much more insight into the quality of their learning after graduation than when they were a student. Thus alumni surveys of modules and programmes, as well as focus groups with alumni, would be an important means of addressing information problems.
3.10 Mechanism 7: monetary carrots and sticks. Short-term pay enhancement could be given for developing teaching innovations, for reorganising problem courses or for taking on a consultative role with students to assess the teaching quality within the department and recommend improvements. Staff could be given a budget to attend teaching/learning workshops and conferences, just as many now have a budget for attending research conferences.
3.11 Mechanism 8: external degrees. One of the problems with the existing system is that lecturers set their own assessment, thereby removing an important potential area of competition between universities. If they all have different assessments, how are meaningful comparisons to be made between universities. The external examiner system is supposed to regulate for this. However, the system is not fit for purpose because too little resource is allocated to the system and there is a high degree of regulatory capture. Externals in some cases are useful advisors, but rarely ensure genuine comparability of standards, given that they do not set the syllabus or assessments. What is more, most university examination regulations preclude externals altering marks, except for a whole cohort, and this would involve a much bigger marking load for the external.
4. Options for improving teaching quality: developing communities of practice
4.1 Students choose to study particular subjects at university. Academics see their primary allegiance as being to their subject. Employers may prefer to recruit graduates from certain subjects. As estates are developed, there is much interest in providing facilities which encourage subject-based learning communities, of students and academics, within universities. Academic subject “tribes” are an intrinsic feature of higher education and effective learning and teaching are best cultivated, recognised and celebrated within these.
4.2 Student communities. Students learn better in an environment where there is a sense of community, where they have joint responsibilities for improving their own learning and that of their fellow students. Some universities actively encourage the development of student subject societies, the production of student journals/magazines, the development of discussion groups outside formal classes and an active alumni association with a sense of responsibility towards the current generation of students. The careful use of group work and various classroom and assessment practices can also help to develop student communities and a sense of communal responsibility. There are plenty of examples of successful communities and information needs to be gathered and disseminated to help their development more broadly.
4.3 Communities of academics as teachers in their disciplines. The subject centres have developed networks of practitioners over the past 11 years (see separate paper “The planned abolition of subject centres”). These share ideas and innovations through various dissemination channels facilitated by the subject centres, such as case studies, workshops, discussion groups, special interest groups, the sharing of material (see open education resources above), conferences, departmental contact meetings (for dissemination by the contacts within departments), networks of associates with particular expertise. Lecturers engage willingly in such communities because they are from within the same academic “tribe”. Academics are often unwilling to engage with the teaching/learning enhancement agenda at a generic level as the language and generality is alien. However, they are often very willing to share practical solutions to issues, such as improving student performance, dealing with large numbers, tackling specific problem subjects or concepts, or relating research to teaching, with colleagues from other institutions but from the same discipline who speak the same language and experience similar problems. Developing and deepening such networks at the subject/discipline level is a way of engaging staff with the teaching/learning agenda and bringing significant quality enhancement and innovation.
5. Overcoming institutional barriers to change
5.1 The above represents a range of possibilities and they can be used in different combinations. A key factor affecting their feasibility is the attitudes of bodies, such as UUK and groups of universities, such as the Russell Group and the Alliance Group. Perhaps the key to unlocking resistance to change is the changed environment in which universities find themselves. Higher student fees, a strengthened NSS and greater transparency in terms of university performance and use of funding are all likely to drive universities towards giving better teaching a higher priority. In these circumstances, universities might be more responsive to mechanisms and regimes that allow them achieve better teaching and learning.
5.2 Appropriate incentive mechanisms are the key. These need to apply not only to individual lecturers, but also to universities.
9 March 2011