Australian Research Management Society Conference
On 18 September Professor Chubb addressed the Australian Research Management Society Conference. You can read it below or download the speech as a PDF here
Thank you for inviting me back to speak to you. It’s always a pleasure to be here and able to speak at this important conference.
I want to begin today by exposing a couple of what some might come to call –obsessions’ of mine.
Basically, I think that over many years in Australia we have tended to duck the truly hard questions – at least as far as what we are all here to talk about is concerned.
Mind you, I think that I could extend that comment beyond our topic into other areas of activity; like the question: what do we want the country to be? Debt free? Sure. But what does it mean to accommodate the fancies of a few economists – and make cuts as if every single element cut was sitting there unconnected to anything else? How does that make Australia better? Do we actually care that the end result is really just the sum of the actions of individual agencies acting independently? An outcome that applies particularly but not only to what we are here to discuss: Australian research and development; and their management.
But because we don’t ask the hard questions, like where should we focus our R&D effort since we can’t do everything (at least not everything well), we live in the thin fog of complacency generated by the –she’ll be right’ approach, or the –no worries’ motto or the –we punch above our weight’ clichÃ©.
None of them is useful. And all of them in some way suggest that we can muddle through – because we have so far. But while we muddle, the world moves. And I find that a tad alarming.
So I am going to ask some questions: like how good are we at research; what should we do to get better; how should we focus our effort and our funding; how should we establish and sustain better links with business; how should we combine our national interest with our role as a global citizen; how well does our education system prepare us for the future?
The list goes on – but I won’t.
I am not even going to try to answer them all today. But I do want you to know that I am not complacent. I do believe that we have real and present challenges and it is time to stop pretending that –she’ll be right.’ It won’t be anywhere near right unless we work at it – and stretch to do better than we have ever done before.
The same old, same old will not work for us – we need imagination, creative thinking and learning from what the rest of the world is doing. We can’t be timid or lazy. We need to evaluate, manage and take some risk. And we need to do it all on a scale that we have never bothered with before. We don’t have to reinvent wheels, just Australianise them – but we need them.
Having got that out of the way, let me begin.
I am reminded that the nineteenth century physicist and mathematician Lord Kelvin once observed “when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.”
What he’s essentially describing is what we still regard as the crux of science today: that quantitative analysis is science’s prime tool and the means by which we make meaning of, and compare observations about, the world.
So when those in universities, government offices and boardrooms are faced with the decisions on what research and researchers should receive more support and funding than others, they turn this prime tool of science on itself. University administrators are increasingly using bibliometrics of research performance, such as citation rates, as an indication of their accomplishments, capacity and potential.
You might not like this. You may think that universities, faculties and individuals shouldn’t be measured by their publication record. Perhaps the final call should be a personal judgment made by individuals with experience and expertise.
My response would be twofold.
Science research is so broad, and the substance of individual studies so complex, that specialised personal knowledge is not a sufficient tool alone to make comprehensive, across the board decisions about the relative merits of one study, or field of study, over another.
My second point would be to remind you of Lord Kelvin’s observation. Although a relatively young field, if we can quantify the quality of scientific research and continue to improve the metrics we use to do this, we can turn the benchmarking and funding of science into a science.
We can expand this bibliometric approach beyond weighing up the relative quality of researchers and universities – we can use it to measure how Australia as a whole performs compared to the rest of the world.
There are a lot of data out there on this – and there are a lot of individual studies that use different data sources, using different metrics and looking at different aspects of Australia’s performance. Some of these have come out of my office in the last few years. There is, however, no holistic report that encompasses all publicly available data so that we can get a broad picture of where we are as a nation in terms of our science – or our STEM.
That’s why, for some time now, my office has been working on a comprehensive report that will provide some sensible, thought-provoking but broad indications of Australia’s performance in STEM research, across a suite of bibliometrics and comparisons. This report will be released in the coming weeks.
I hope it also provokes some sensible constructive discussion – although I can almost taste the letters to the editor column of the Canberra Times – –if only the Chief Scientist understood’ it will say – and I will be lucky if I get the honour of an upper case C and an upper case S.
The first question to ask when considering benchmarking our performance is –to whom should we compare ourselves?’ The instinctive response is to say –everyone’ – then we can get a picture of where we stand in the whole world. But this approach is fatally flawed, in my view.
90% of all citations are attributed to papers originating from just 23 countries – that is only around 11% of all the countries included in the data.
This is one of the reasons why I am not a fan of the clichÃ© –world-class’. It’s a meaningless or at least misleading phrase. It’s a waste of time to pat ourselves on the back for being much better than the world average when the numerator is dominated by so few countries and the denominator by so many which contribute so little to the numerator.
I can give you a simple example. I used to think I was a great footballer – destined to play first ruck for South Melbourne (now the Sydney Swans for those who don’t know about these things). From not long after I could walk, I could kick a ball. And I used to win every match I played – in the back yard, against my dog.
Were my dog able to kick a ball or even comprehend the rules, this might have been a useful way to assess my football prowess. But he could do neither; so it was a pretty low bar to set.
Of course, we could do the same in our report; set a low bar of comparative countries whose average performance is low enough for us to be above it. Or find another denominator to keep us happy.
Or we could compare what we do with countries we could be like, and in whose company we feel we belong – countries which, like us, are free market economies with a serious science engagement.
So throughout the report, we compare ourselves to 11 Western European nations, the USA & Canada and countries in our region.
The report examines seven areas:
- Our research performance
- Our patent intensity
- Trends in national research funding
- International research collaboration
- Our STEM research workforce
- The pipeline of STEM students and graduates from higher education
- And the performance of our schools.
The report will make clear that we do have some research output that is amongst the very best in the relevant field globally. We have a noticeable share of the top 1% of field-weighted cited publications in Natural Sciences and Engineering. And Australian output is represented in every field of highly cited research globally.
But when we turn to the average, and compare ourselves to a selection of countries in Western Europe and North America – countries that we like to think we could be like – our average field weighted citation rate is below them all.
Medical research and Engineering are strengths and we also publish 7.5% of the world’s Environmental Science papers – our highest percentage contribution in any field.
But there is room for improvement: as I said, our average is below them all.
So how should we go about building on and improving the quality of Australian research? For us to be able to stand confidently and proudly shoulder to shoulder with the nations that are among the best in the world?
The answer to me is obvious: get the answers to some hard questions – and act on them. One of the hard questions we could imagine would be: why should we expect the Australian public pay to support every Australian academic to do whatever research they want to do – whenever they want to do it? Now I know they don’t do that literally – but when you have a definition of a university that includes the notion that all should eventually conduct research in every area they teach – that is the implication.
Realistically, that will not happen. Research funding will continue to be rationed. And if we were able to learn, we would know by now – after decades – that it doesn’t work to ask for more money on the grounds that we don’t have enough; not enough to spend on whatever turns up that exceeds threshold quality standards. It hasn’t worked before and I can’t see it working now.
We need to be a lot smarter than that. We need to change tactics.
My contribution has been to call for a strategy.
It’s been just over two weeks now since I released a document called STEM: Australia’s Future. It contains a suite of recommendations to the government for a strategic approach to Science, Technology, Engineering and Maths. And I have to say that it struck a chord. There have been about 460 articles about it in the print media alone since it was released.
I’ve been stressing the need for a national STEM strategy for over a year now because, as I never get tired of pointing out, we are the only OECD country without one and we’re suffering because of it.
Australian research is very broad in its scope. As I said, it’s not realistically achievable to lift up every one of these fields to be among the best in the world – at least, not all at once.
So we need to find areas where we have a critical need or comparative advantage; where we have capability and a capacity to make a difference. Then we need to support and fund those areas as best as possible.
This is not about following fads or current trends – researching areas just because they’re famous or popular. It’s about having a framework that allows one to make the decisions that will have to be made in an informed and intelligent way – whether you’re a research funder investing money or a researcher investing your time and energy.
It certainly isn’t about shutting out basic and curiosity driven research. If we exclusively fund marketable applied research, we’ll soon find ourselves running out of fundamental science to apply.
In my strategy proposal, one of my recommendations is that we ensure capability in the core sciences and maintain key components of basic research. This is, after all, where the continuous flow of new knowledge comes from.
When Minister Ian MacFarlane spoke at the launch of STEM: Australia’s Future, he addressed this very point. He said “we need people across government to understand that some money will be spent to improve knowledge but not return a cent in terms of income to the country – there is appreciation of that within Cabinet.”
I would go a little beyond this, and say that some money should be spent to improve knowledge that might not immediately, directly or obviously return a cent to the country.
In the late 30s, a small group of physicists began thinking about how atomic nuclei interact with magnetic fields. They thought about it and investigated it purely because it was interesting and it would be knew fundamental knowledge about the world. Even after three of them were awarded Nobel Prizes , they had no idea that someone would take the fundamental work that they had done and use it to create the world’s first MRI machine.
I like this example because it also demonstrates another point. MRI has had a profound impact on medicine and yet it wasn’t born out of medical research. Its success has been the product of physicists, engineers along with medical researchers.
A lot of people outside the research community tend to labour under a lot of misconceptions about how science works. Sticking with medical research as an example, there’s a romantic notion that at the end of the day a doctor takes off the stethoscope, picks up the pipette and starts looking for the cure for cancer – waiting until they suddenly and dramatically see the answer in the petri dish and shout –eureka!’
As you know, this is not how it works. While important matters like peer review, clinical trials and experimental techniques are not fully appreciated by the public, the notion of cross-discipline collaboration is not always appreciated either. It’s not often considered that solutions for problems in one field might come from another; that engineers and statisticians solve problems for geneticists and computer scientists for chemists, for example.
So while we portion funding and support to areas where we have a critical need or strategic advantage, we still need to maintain support across the board of disciplines. Otherwise some of our critical areas might fail due to the demise of supporting fields.
There are many other necessary aspects of a national STEM strategy that I haven’t touched on – a secure education pipeline, strong international engagement and the means to reinforce Australian competitiveness.
That’s all included in more detail in STEM: Australia’s Future.
To put it all simply, let us understand our solid foundation, work out what to do, and let’s get better.
 Lecture on “Electrical Units of Measurement” (3 May 1883), published inPopular LecturesVol. I, p. 73
 InCites. Cumulative measure over 2002-2012