Public sector organisations such as the Teeland universities receive all their funding from central government and do not have the generation of profit as an objective. Furthermore, their objectives such as ‘improving the standard of education of the citizens of Teeland’ cannot be measured in financial terms. The value for money of the universities can be assessed using the ‘3Es’ framework of economy, efficiency and effectiveness.

Economy

This involves obtaining the inputs to the service at the lowest possible cost, while still maintaining the quality of the inputs. In practice, this may be difficult to do, and a reduction in cost may lead to a reduction in quality.

From the performance data given, the average annual payroll cost per member of academic staff is highest in Northcity University at $62,286 ($109m/1,750). This is 51% higher than Eastcity University, which has the lowest cost at $41,237 of the four universities given. The high costs in Northcity University may reflect the fact that staff there may be more highly qualified or that highly skilled, trained staff are attracted to work there because of the high standards of teaching and academic research.

A higher payroll cost per member of academic staff may not necessarily mean a particular university is not giving good value for money. The relatively high salary costs in Southcity University of $62,083 may simply reflect the higher costs of living in the capital city and so comparison between the regions may be inappropriate.

As such, there may be a conflict between this performance measure and the ability of the universities to achieve their objectives, such as to improve educational standards in Teeland. The politician’s proposal to reduce salary levels for new recruits may reduce the number of appropriately qualified and skilled staff who wish to work there. This may reduce the standard of teaching and academic research, and as a result, the universities’ performance against their stated objectives. This means that focusing solely on economy would not ensure the universities achieve value for money.

Efficiency

Efficiency measures the amount of outputs relative to the amount of inputs. The number of academic staff per student may be a suitable measurement but there may be differences between universities which would make it hard to compare results.

For example, there are 10·1 (17,600/1,750) students per member of academic staff in Northcity University, which is ranked number one in the Teeland government’s provisional league table. There are 17·9 in Westcity University which is ranked much lower at 21. It would seem therefore, that increasing the number of students relative to the number of academic staff as per the politician’s proposal may reduce the performance of the universities against their stated objectives and as such will not increase value for money.

The politician’s comparison with the number of students per academic staff in neighbouring Veeland may not be appropriate. Whilst the politician’s assertion that educational standards are higher in Veeland may be correct, the undertaking of high quality academic research may not be a key objective of the universities there, or the quality of research may be lower than in Teeland.

Effectiveness

Effectiveness measures whether the objectives of the organisation are being met. The stated objectives of the Teeland universities are to improve the overall standard of education of citizens in Teeland, to engage in high quality academic research and to provide well-qualified university graduates to meet the needs of the graduate jobs market in Teeland.

Objective to improve the overall standard of education in Teeland

Currently, there is no direct measure of the performance of the universities’ stated objectives to increase the level of education of the citizens of Teeland. The number of graduates entering graduate jobs and the results of the TSOR survey may indirectly measure the effectiveness of the universities in achieving this aim. More direct measures such as the number of students completing their studies, or obtaining good results in university exams may be more appropriate.

The effectiveness of the universities in improving the standard of education should really be related to the entry requirements of each university. Westcity University is the lowest ranking of the four universities in the provisional league table and has the lowest proportion of graduates entering graduate jobs at 50% (1,750/(11,200/3·2)). It also receives the lowest amount of research funding per student at around $1,250 ($14m/11,200), but it has the lowest entry requirements, which have been relaxed to encourage students from a more diverse range of backgrounds to study there. Therefore, the improvement in educational standards relative to students’ attainment on entry may be higher than the position in the league table may suggest.

Objective to engage in high quality academic research

The stated aim of the universities to engage in high quality academic research is also not currently measured. The amount of research funding received from government and other organisations by each university may indirectly reflect the quality of the research there as providers of funds for research would probably look to fund high quality research.

This measure may, however, equally reflect that some types of research are more expensive than others. Southcity University, which is successful in science and technology subjects, receives $15,592 per student compared to $5,000 in Eastcity, which specialises in arts and humanities subjects. The funding received may be more indicative of the past quality of academic research rather than future quality, or may not even reflect the quality of the research and may be high due to wastage or inefficiency. The definition of ‘quality’ of research is unclear.

Objective to meet the needs of the graduate jobs market

The number of graduates from each university obtaining graduate jobs each year is an indirect measure how well each university is achieving this objective. However, the number of graduate jobs filled may simply reflect the number of students at each university and demand for graduate jobs in the economy.

Many graduates, including the most talented of them, may take graduate jobs overseas. In which case, the measure of graduate jobs filled may not be a clear measure of the objective to meet the needs of the graduate jobs market in Teeland. A more precise measure of graduates entering graduate jobs in Teeland may be more appropriate. Even this may be misleading, if graduates later return from overseas to enter graduate jobs in Teeland.

The definition of what is a ‘graduate’ job is subjective and likely to change over time, for example, as a result of changes in the economy. This again limits the usefulness of this measure in determining whether universities have met this objective.

The TSOR survey is a measure of effectiveness reflecting a basket of measures, one of which is students’ own perceptions of their future job prospects. The measures in the TSOR survey are, however, highly subjective according to individuals’ personal perceptions, for example, about students’ job prospects and the quality of teaching at the university. Furthermore, the survey also covers a wider range of factors such as their overall satisfaction with university life. As such, this may not reflect the stated aims of the universities in Teeland. The TSOR score for Westcity University is considerably higher than for all the other universities, despite it having the lowest overall ranking.

Effect of league tables on the quality of the Teeland universities

League tables encourage competition between universities

The publishing of league tables of the performance of the universities in Teeland should stimulate competition for high ranking between them. This should encourage them to find better ways to improve the quality of the service, achieve their performance objectives, and deliver greater value for money.

Sharing of resources may be discouraged

However, this may discourage knowledge or even resource sharing between the universities if they see themselves in competition for the best students and limited research funding. This could particularly be a problem if the university leaders were given performance targets and rewards based on their university’s position in the league table. They may be encouraged to undertake ‘gaming’ and focus solely on their position within the league table to the detriment of other aspects of performance. This may mean the objectives for the universities overall in Teeland may not be achieved.

University leaders may focus on a narrow range of objectives

Students’ overall experience of university life may be influenced by access to social, pastoral and sporting facilities. Management time and resources may be diverted away from these important areas by focusing on a narrow range of performance objectives such as improving educational standards and high quality research in order to boost the position in the league table.

Use of league table to ensure accountability of the universities

If the performance league tables are made publicly available, for example, on the internet as they are in Veeland, this should ensure the performance of the universities is transparent. This will mean that they are held accountable by the public for the quality of service and value for money which they provide.

League tables give choices to students and staff

To be really effective, league tables should provide users of the performance data, such as the public, with the ability to make choices based on the data given. Prospective students and staff will be able to make choices about which university to study or work at according to their position in the league tables. In addition, providers of funding such as the government will be able to identify where corrective action or additional funding may be required.

This could also be a disadvantage of using league tables, as the best students are likely to be attracted to the best universities, which are in turn likely to attract the most funding for academic research. This may cause a reduction in the overall performance of lower ranking universities, and therefore a failure of the universities as a whole to achieve the stated aim of improving educational standards in Teeland.

League tables may not reflect variation of standards at each university

There may also be considerable variation of the standards within one university, which a ranking in a league table will not address. For example, Southcity University offers courses in a variety of subjects and is particularly successful at science and technology subjects. This may mean that it achieves a high overall ranking in a league table, whereas its performance in other subjects may be weaker. For all of these reasons, the provision of league tables or performance data at a lower level, such as by subject area, may therefore be a better way to make the universities accountable to the public.

Regions may not be directly comparable

There may be inherent differences between universities which make comparison using league tables misleading or of limited value. For example, Eastcity University specialises in teaching arts and humanities. It may be inappropriate to compare it to universities offering a wide range of courses or specialising in science and technology subjects. Similarly, it may be misleading to compare universities of different sizes or in locations in different parts of Teeland. Universities which also attract students from outside Teeland, for example, those in capital or major cities, are likely to have more access to the most able students.

Due to its capital city location, Southcity University may incur additional costs of providing facilities and have to pay staff more to reflect the higher cost of living there. It may be more appropriate to produce league tables at a lower level, like within individual cities, to make the comparisons more meaningful and eliminate the effect of regional variations such as this.

There may be differences in the way different universities collect and report performance data. These problems can be reduced by the use of consistent and enforced policies for compiling performance data.

Resources required to produce league tables

Other problems relate to the targets chosen, external influences on results and cost of collating and measuring performance for the league table.

In Veeland, there is a sizeable government department dedicated to measuring performance. The cost of collecting performance data and compiling the league tables may therefore outweigh the benefits to be gained from doing so. It will use up resources which could be used, for example, in providing additional funding to the universities themselves.

Performance measures used to create league tables may be confusing and conflicting

By having a large number of targets such as in Veeland, this may cause confusion to managers, who may not know which to focus on. Similarly, it may be difficult to decide which performance measures are the most important.

Many performance measures chosen are likely to conflict. Reducing the salaries paid to academic staff may make it harder to recruit and retain experienced and well-qualified staff, which may lead to failing to meet the universities’ key objectives. Similarly, increasing the number of students per member of academic staff is unlikely to help achieve any of the universities’ objectives as staff will have less time available for teaching and for engaging in high quality research.

The use of league tables may demotivate staff and students

The Teeland politician’s proposed target of a 5% annual increase in the number of students entering graduate jobs sounds very challenging and may be unrealistic. This will be especially so if accompanied by a reduction in the number of academic staff and a reduction of the salaries of new recruits. If the targets are felt to be unachievable by the university leaders, they may become demotivated, and give up on trying to achieve them altogether. This will especially be the case if this is an ‘all or nothing’ target. A more realistic target may at least ensure that some progress is made.

Students may also become demotivated if their university ranks poorly in league tables, even if the students themselves are satisfied with the standard of education they are receiving.

Similarly some performance measures, such as the number of students entering graduate jobs, which may be highly dependent on the economic conditions in Teeland, are not entirely under the control of the university leaders. The economic conditions in Teeland will be a key factor in determining the number of graduate jobs available there. University leaders may become demotivated by this measure as well and stop trying to improve.

The relative importance of different performance measures is subjective

The weighting of the different performance measures used by the Teeland government in arriving at the provisional ranking of the universities is subjective. It is difficult to determine which measures are the most important, and hence how each should be weighted. The opinions of different stakeholders on which are the most important measures will also vary. Academic staff and students may view different measures as important compared to the providers of finance (the Teeland government) and organisations who recruit university graduates.

Determining the relative importance of different performance measures in order to publish a league table is therefore difficult. Furthermore, the measures chosen, and their relative weightings, could differ from those in other countries. This would make it difficult to benchmark performance against universities in other countries, such as Veeland.