Discussion paper: Making and assessing claims of scholarship and scholarly activity

Purpose

As part of its regulatory roles, TEQSA is required to assess evidence of ‘scholarship’ that is put forward by providers, whether evidence of scholarship by individuals, such as teaching staff within the provider, or for the provider as a whole.

TEQSA is seeking to review whether its current approach to assessing claims of scholarship and scholarly activity (as described in the Guidance Note on Scholarship) is adequate, or if the approach needs to be reconceptualised. The purpose of this discussion paper is to set out, for consideration by the sector and other stakeholders, draft principles that are proposed to guide providers in making claims related to scholarship, and to inform TEQSA’s assessments of such claims.

This paper does not seek to replace established frameworks or long standing approaches to scholarship, but rather, seeks to elicit feedback on how TEQSA can improve its approach to assessing claims of scholarship and scholarly activity.

The sector’s responses to the questions posed within this paper will also inform TEQSA’s revisions to the Guidance Note on Scholarship which is currently under review.¹

Consultation process and timeline

Feedback on this paper is invited. It would be helpful if your responses address the discussion questions posed in the paper, together with any other related matters that you may wish to raise.

Comments should be provided in Microsoft Word or PDF files. Submissions may be made public unless accompanied by a request providing reasons they should not be made public.

The closing date for feedback is 14 December 2020.

Submissions should be directed to standards@teqsa.gov.au.

Background

Part A of the Higher Education Standards Framework (Threshold Standards) 2015 (HES Framework) specifies a number of requirements involving inputs arising from ‘scholarship’, either directly or indirectly. These include:

1. Scholarly underpinning of specified learning outcomes (1.4.2 a-d)
2. Research training outcomes (1.4.7)
3. Course design (3.1.2 a-c)
4. Staffing (3.2.3 a-b)
5. Research training environment (4.2.2).

The corporate and academic governance functions of a provider are also expected to have oversight of the broader academic activities of the provider, including the impact of scholarship on the provider’s academic activities (e.g. 6.2 f, 6.3).

Part B of the HES Framework also identifies scholarship as an essential characteristic of a higher education provider (Part B1.1.4 for a HEP, and for other types of provider in Parts B 1.2 – 1.4) and of a provider with self-accrediting authority for its courses (Part B3.d).

TEQSA’s Guidance Note describes ‘scholarship’ as ‘activities concerned with gaining new or improved understanding, appreciation and insights into a field of knowledge, and engaging with and keeping up to date with advances in the field. This includes advances in ways of teaching and learning in the field and advances in professional practice, as well as advances in disciplinary knowledge through original research.’

This discussion paper proposes a set of principles that would guide providers when offering evidence of their capacity to meet the requirements of the HES Framework in relation to scholarship. Submissions to this discussion paper will guide future revisions to the Guidance Note on Scholarship.

The importance of scholarship across different types of providers

The HES Framework requires scholarship as an essential characteristic of higher education, both for individuals and for the provider as a whole (referred to as scholarship at the institutional level in this paper). As a consequence of these requirements, providers will need to be able to demonstrate that individual academic staff are engaged in scholarship as required by the HES Framework and that the institution has evidence that this is occurring. The provider will also have to demonstrate that it has created an institutional climate that requires and enables scholarship in accordance with the HES Framework. TEQSA expects that students on all campuses of an institution should benefit from an institution’s scholarly activities.

While the requirements of the HES Framework apply to all providers, TEQSA acknowledges that those requirements may manifest in a variety of different practices according to context. For example, TEQSA recognises that how individuals go about maintaining scholarship and how the institution supports scholarship will be quite different in scale and nature between a large research-intensive provider offering higher degrees by research and a smaller, more focussed teaching-only provider. Similarly, the circumstances for individual staff will vary across a spectrum from a full-time research orientated setting through to a part-time teaching only setting. With recognition that student and staff profiles may vary between an institution’s campuses, TEQSA acknowledges that the nature of scholarly activity and evidence of its outputs may vary between campuses. Notwithstanding, TEQSA considers it reasonable to expect, for interests of all students regardless of the campus at which they study, evidence of scholarly activity and output at all campuses of an institution.

Classification of an activity as scholarship for regulatory purposes

Principle 1: To provide evidence of scholarship for regulatory purposes, the proposed scholarly activity must be consistent with an established typology of ‘scholarship’.

In assessing whether an activity constitutes ‘scholarship’, TEQSA proposes to draw on established frameworks for the definition and classification of types of scholarship, the best known of which is perhaps that of Boyer (1990). In so doing, TEQSA acknowledges that there have been subsequent published variations of the Boyer Framework and that providers frequently translate the Boyer model into other forms that are tailored to their particular context or discipline, while nonetheless preserving a credible typology consistent with the broad thrust of the Boyer model. TEQSA will welcome such tailored frameworks. Ideally, they would be recognised models or frameworks of scholarship and derive academic credibility.²

TEQSA proposes the above principle to guide its determination of whether proposed activities constitute scholarship for regulatory purposes.

In applying this principle, TEQSA must initially decide whether an activity presented as evidence of ‘scholarship’ does, in fact, represent scholarship for regulatory purposes within the context of the HES Framework.

In so doing, TEQSA proposes that evidence of scholarly activity would need to be consistent with one of the following types of activity:

• an activity consistent with at least one of the forms of scholarship described in the framework developed by Boyer (1990); that is the scholarship of (i) discovery, (ii) integration, (iii) application or (iv) teaching, or
• an activity of a similar kind that is consistent with subsequent peer-reviewed adaptations of the Boyer framework, or
• an activity of a similar kind that is evidently consistent with some other credible typology of scholarship that embraces similar principles and is acceptable to TEQSA.

Scholarly activities also need to be linked to the academic’s disciplinary field, as disciplinary scholarship is an essential component of scholarship that yields benefits for teaching and learning. TEQSA acknowledges that higher education providers do not engage in scholarship in the same manner. Depending on the providers’ missions and business models, some providers may focus more heavily in the scholarship of discovery (such as research), while others may focus more on the scholarship of teaching. Nonetheless, at the whole-of-institution level, academic staff need to engage with disciplinary scholarship to achieve the intended outputs and outcomes.

Demonstration of outputs or outcomes arising from scholarship

Principle 2: Evidence of scholarship must include demonstrable links to intended outputs or outcomes of that scholarship and be accompanied by mechanisms to monitor and evaluate those outputs or outcomes.

TEQSA acknowledges that individual intellectual pursuits are central to scholarship. However, in accordance with the thrust of the Boyer model and similar models, TEQSA takes the view that, for the purposes of its assessments, an activity classified as ‘scholarship’ should lead to an identifiable output or outcome that is relevant to the requirements of the HES Framework. TEQSA will also look for mechanisms that are in place to monitor and evaluate such outputs or outcomes.

Demonstrable outcomes or outputs from scholarly activities could include changes to course content, course design or improved learning outcomes. For example, research leads to the creation of new knowledge (discovery) as an output, which may in turn, inform teaching and learning or affect the collective appreciation of the field through influencing the understanding of peers over time (outcome).

If TEQSA is satisfied that claimed activities constitute an identified form of ‘scholarship’ in accordance with Principle 1, TEQSA proposes that it will also look to see that the activities have already led to, or are highly likely to lead to, intended improvements. In so doing, TEQSA will look to improvements of the following kind:

1. Advances in expected learning outcomes for students and/or graduates (e.g. arising from improvements in course content, design, methods of participation/delivery, assessment or re-definitions of learning outcomes and the like).
2. Advances in the concepts and understanding of a discipline for staff and others (e.g. through seminal influential papers, scholarly reviews etc.) and/or influences on how the discipline is taught and learned (such as a new published pedagogy).
3. Advances in a creative field of endeavour (such as new insights, challenges to orthodoxy or innovative approaches to creative outputs).
4. Advances in professional practice (e.g. new ways of practice that provide better solutions to existing challenges) including innovations (e.g. implementation of new technology).

In submitting evidence of scholarly activities, a provider would be expected to outline the links between the activities and consequent improvements in specific curricula or other teaching-related outputs or activities, and to learning outcomes or other outcomes such as influence on practice. For some types of scholarly activity, the result may be legitimately confined to demonstrable improvement of a particular course of study and its associated learning outcomes e.g. an improved course design based on changing appreciations of a field of study.

For many types of scholarship, however, dissemination of the results to wider scholarly or professional communities would ideally be evident. Such dissemination would extend beyond the originating activity, to at least to other programs within the provider and/or, ideally, to a relevant scholarly/creative/professional community more broadly. For example, a new pedagogy arising from an individual’s scholarship may be adopted throughout a school, faculty or even a provider as a whole. These institutional improvements may also be advocated more widely through a community of scholars, such as national or international conferences/publications.

In considering claimed improvements, TEQSA will take account the practical reality that improvements arising from scholarship may occur and/or have their effects in stages. For example, as a result of scholarship in teaching (such as a literature review of emerging concepts in teaching in a field of education), the initial output may be a revised course design (an output of scholarship). In time, it will be possible to demonstrate whether the intended longer-term outcomes arising from this new design, such as improved student learning outcomes and/or facilitation of improved life-long learning and professional development of graduates have been achieved. Similarly, other types of scholarship may be reflected in evidence of improvements in creative/professional practice.
In acknowledging such staged achievements, TEQSA will nonetheless expect to see that scholarly activities are accompanied by identification of the intended outcomes of that scholarship, and that some mechanisms will be in place to monitor and evaluate such outcomes. TEQSA would not regard a ‘scholarly’ activity (an input) as an end in itself; evidence of intended outcomes accompanied by a means to monitor and evaluate those outcomes are seen for regulatory purposes as no less necessary to scholarship than the activity itself.

Above all, academic staff need to acquire and maintain knowledge of contemporary developments in their discipline or field by continuing scholarship or research or advances in practice (HESF Part A, 3.2.3). This means that providers need to ensure that academic teaching staff are engaged in disciplinary scholarship as this contributes to the vitality of the intellectual climate and academic environment, and leads to improvements in learning outcomes. This may include activities such as literature review, article publications or participation in professional development. As part of their submission of evidence of scholarship and their outputs or outcomes in a discipline, providers should demonstrate that academic staff are active in their disciplinary fields.

Activities that are not necessarily seen as scholarship

Principle 3: Activities such as professional and community engagement, professional development and routine professional/artistic practice will not be regarded as scholarship unless they meet the requirements of Principles 1 and 2.

TEQSA understands and acknowledges that professional development of individuals and engagement with a professional/creative community and/or the community more widely are valued aspects of higher education in Australia. However, TEQSA is of the view that not all professional development/engagement necessarily constitutes scholarship. For example, a staff member of a provider may be a member of the board of their professional association, such as a member of the association’s audit committee. While this may be of considerable value to the association and to the individual, unless the individual teaches or researches in financial governance, such activity may not meet any accepted definitions of scholarship nor realise any scholarly outcomes and thus not be regarded as scholarship by TEQSA. On the other hand, if the individual was instead part of a professional group that was charged with re-defining the scope and nature of a profession, which, in turn, would influence course design, learning outcomes and future professional practice, this could be seen as scholarship. Evidence of such outcomes would be demonstrable in graduate outcomes over time.

Engagement in routine professional practice to maintain routine professional skills (e.g. an academic accountant running a small tax return practice) or engaging in recurrent artistic performances (e.g. performances of an established musical composition) may well be helpful to hone one’s skills. However, these are not likely to be seen as ‘scholarship’ by TEQSA without evidence that these activities would constitute or lead to advances that others could learn from. In the case of artistic endeavours, this could be demonstrated through associated scholarly commentary or a catalogue raisonné³ of an exhibition, for example. Likewise, activities concerned with development of technical skills, for example a professional development program in data management that leads to administrative efficiencies for a provider, would not be seen to constitute scholarship unless they also evidently flow through to improvements in teaching and learning or research programs in data management. Many examples of community engagement, although they may be demonstrably important to a provider’s overall mission, are also not likely to be seen by TEQSA as scholarship if they do not meet the tests of ‘scholarship’ in either their nature or their effect.

TEQSA also acknowledges and expects that a variety of individual and institutional activities may be undertaken with a view to enhancing administrative efficiency and effectiveness and/or to provide better services to students. Important as such activities are in the life of a provider, they would not be accepted by TEQSA as scholarship unless they lead directly to scholarly outcomes of the types discussed in this paper.

Claims of scholarship will be assessed according to evidence of scholarly outputs or outcomes as outlined under Principles 1 and 2.

Provider involvement in scholarship

Principle 4: Providers will be able to present a plan to create an environment of scholarship, which is monitored and reviewed, together with an aggregate representation of their involvement in scholarship within the context of the requirements of the HES Framework.

The HES Framework requires individual academic staff members in a provider to be engaged in scholarship (3.2.3). It also requires a provider to create and foster a scholarly environment for research training (4.2.2). Monitoring, review and improvement mechanisms are also required (5.3). As such, TEQSA will expect to see that the provider’s policies and practices collectively:

• define and engage with ‘scholarship’ in a manner consistent with the typologies and effects outlined above
• create an expectation of scholarship at the institutional level for all teaching/research staff
• create a culture of monitoring and evaluation of the outcomes of scholarship, with related monitoring instruments and processes to do so; and
• allow the provider to present an aggregate representation of the extent of its involvement in scholarship, including an overall plan that is monitored and reviewed (e.g. by an academic review committee, academic board or the like), for both progress and outcomes.

A provider’s claims of involvement in scholarship would be expected to be assessed by TEQSA in relation to the dimensions outlined above.

Diversity of approaches to scholarship across the sector

Principle 5: TEQSA will accept different approaches to scholarship that reflect the nature of the provider.

TEQSA recognises that the requirements of the HES Framework can be met in different ways according to the circumstances of the provider. Providers may approach and conduct scholarship differently depending on the nature of their operations and their primary focus within the different types of scholarship. The importance of scholarship across all types of providers has also been emphasised previously, and TEQSA will consider the different types of scholarship activities so long as they meet the requirements of Principles 1 and 2.

TEQSA notes the benchmarking discussion paper by Emeritus Professor Geoff Scott summarising the results of the Higher Education Private Provider Quality Network (HEPP-QN) scholarship workshop conducted on 29 October 2019. The workshop sought to identify approaches to scholarship that are specifically relevant to different higher education provider categories, and was attended by 11 non self-accrediting higher education providers (HEPs), two self-accrediting HEPs, two pathway colleges, and one University College.

For the purposes of brevity, the following list provides practical examples of what participants at the HEPP-QN workshop considered to be indicators of a quality approach to scholarship. This is not an exhaustive list of scholarly activities conducted across the sector. However, these examples may inform TEQSA’s development of further guidance to the sector on the different types of activities that may be accepted as scholarship if they meet the requirements of Principles 1 and 2.

Selected examples of key indicators of quality approaches to scholarship nominated by participants at the HEPP-QN workshop

• Course Development and Review policies and procedures follow ‘best practice’ research.
• Robust philosophy and policy on scholarship with clear evidence of its consistent implementation via staff induction programmes and annual performance monitoring.
• Peer review and feedback results from students are analysed and shared. TEQSA would expect this to inform further improvements to curricula, course content, and course design.
• Consistently high attendance at professional development events by both tenured and sessional staff. TEQSA expects these types of activities to have demonstrable outcomes for teaching and learning in order to be accepted as scholarship.
• The provider actively disseminates the latest research on what is taught and effective innovations in learning and teaching.
• All staff are demonstrably up-to-date with research in the area(s) they teach. TEQSA expects teaching staff to apply this knowledge to inform their teaching activities.
• Staff are researching their own practice and its impact on student outcomes and successfully addressing any improvement areas identified.
• Staff are implementing Course Development and Review policies effectively.
• There is acceptance of articles written by staff in peer-reviewed journals.

The discussions from the workshop demonstrate the wide-ranging indicators that providers consider to be most relevant to their nature of operations. TEQSA is aware that there may be specific kinds of scholarship activities and outcomes that are considered to be integral to the provider type in order to uphold its reputation and the sector’s reputation more broadly.

Nonetheless, the key indicators nominated by providers at the HEPP-QN workshop above are not mutually exclusive between provider types. For example, an independent provider may very well elect to focus its investments and efforts into research activities based on its strategic plan or operating model. TEQSA does not seek to confine the types of scholarship activities by provider type, but continues to encourage the sector to delve into a diverse range of scholarship inputs to achieve meaningful and beneficial learning outcomes for students, which work to enhance the quality delivery of higher education and uphold the reputation of the sector.

 

 

Summary of TEQSA’s proposed approach

TEQSA proposes to assess claims of individual scholarship according to the following:

1. Is the nature of a claimed scholarly activity consistent with an established typology of scholarship that is acceptable to TEQSA?
2. Is there evidence of intended outputs or outcomes of scholarship including, where relevant, dissemination of the outputs or outcomes of such activity that are likely to foster advances more broadly?

In relation to claims at the institutional level of the provider, TEQSA will look to the requirements of the HES Framework, particularly:

1. Evidence of creation of a climate of scholarship, including institutional planning
2. Mechanisms to monitor and evaluate the impact of such scholarship.

Notes

1. TEQSA thanks those that have provided their feedback on the Guidance Note thus far. These will be considered alongside responses to this discussion paper.
2. Examples include: Boyd, William E. (2013) 'Does Boyer’s Integrated Scholarships Model Work on the Ground? An Adaption of Boyer’s Model for Scholarly Professional Development,' International Journal for the Scholarship of Teaching and Learning: Vol. 7: No. 2, Article 25 [Available online]; Heinemann, Robert L (1999). "We Are Who We Are" : Repositioning Boyer's Dimensions of Scholarship. Distributed by ERIC Clearinghouse, [Washington, D.C.] [Available online].
3. A descriptive catalogue of works of art with explanations and scholarly comments.