Assessing Research Quality and Impact

We're starting on the final plenary sessions at SPIN now. The first speaker here is Toss Gascoigne from CHASS, the Council for the Humanities, Arts, and Social Sciences. Toss notes the fact that the HASS sector is wildly underrepresented within the government sector - while there are organisations representing the sciences and other areas, and an Australian Chief Scientist, there is no such representation for the HASS field. Toss and his organisation are working on changing this, and some of their work was presented yesterday already (see e Commercialisation in the Humanities). Unfortunately, of course, the HASS sector is also measured in terms of its validity using science-derived rules (publications output etc.), and CHASS has been asked by the federal government to provide alternatives to such measures. Toss also points out the continuing need for organisations in the HASS sector to become members of CHASS and take part in its decision-making and advisory processes.

Toss points out that the project between CHASS and the Australian federal government Department of Education, Science and Training to develop alternative quality and impact assessment systems for publicly funded research in the HASS sector is a once-in-a-decade opportunity. On Tuesday, DEST released its new issues paper "Research Quality Framework: Assessing the Quality and Impact of Research in Australia" which sets the theme for this work. Toss now hands over to Jonathan Powles to describe this project in some more detail. Jon notes that the move here is away from purely quantitative measuring systems (number of papers published, etc.) to more qualitative approaches which evaluate excellence - and so it is important to be clear about terms and definitions in this context. There are three key terms here, with working definitions:

• quality - academic or artistic excellence as viewed form within the academy or the community of artistic practice
• impact - value or benefit to society-at-large of research or artistic practice (this is different from traditional scientific definitions of the term, which focus on impact within the academic community and is usually measured through tools such as citation indices)
• capability - capacity and preparedness to contribute to enhancing the research or artistic capabilities of a group of practitioners, or the nation as a whole (this aspect, then, deals with people rather than works or other outputs - with skills rather than the content generated)

Jon goes on to pose four questions to the audience now:

1. When are artistic work and creative practice not research? Rod Wissler now suggests that perhaps there is an area of routine artistic work which is not research, and Allison Richards suggests that the research intention of the activity needs to be articulated; another audience member disagrees and suggests that there needs to be insight delivered by the work, whatever the intent. Now Luke Jaaniste points out that this question requires a stronger definition of research in the first place.
2. What are the boundaries of creative practice and research? Does it depend on the participants, the funding? An audience suggestion is that this depends on the processes involved, and again the discussion returns to the question of what we define as research in the first place.
3. In what special ways to the creative arts and industries have 'value' or 'benefit' to society at large?
4. When is 'education' in the creative arts the same as 'research'? (To learn in the arts field it is usually necessary to research as well.) In response, Stuart Cunningham points out that while very broad definitions of research are certainly intellectually possible, from a pragmatic point of view in the context of this exercise they are not useful, as DEST is unlikely to accept them. He suggests a definition of research as new knowledge as ratified by an independent peer assessment process. Jon also points to the fact that more inclusive definitions also enable CHASS to claim more impact for research in the sector, however...