Risk assessment framework

Introduction

Background

The Tertiary Education Quality and Standards Agency (TEQSA) risk assessments of registered higher education providers are a key component of TEQSA’s risk-based approach to assuring higher education standards. The Risk Assessment Framework (RAF) outlines the key steps and components of the risk assessment process, and provides detailed supporting information on the risk indicators used. Information on how risk assessments form part of TEQSA’s approach to quality assurance can be found in the paper Our approach to quality assurance and regulation.  

Under the ESOS Act, TEQSA regulates a number of ELICOS and Foundation Program providers that are not registered higher education providers. TEQSA’s approach established under the RAF also applies to these providers, but with a tailored set of risk indicators and information requirements. 

TEQSA is committed to continuing to refine the RAF over time, with experience of applying the RAF in its assessment processes, feedback from providers and through consultation with peak bodies. TEQSA will also continue to expand available information on the RAF and other processes through the TEQSA website.

Information sources 

TEQSA works closely with the Department of Education (Department) to access data for providers that already report data to existing collections in order to minimise reporting burden and remove any overlap in reporting through data sharing arrangements with other agencies.

TEQSA sources data through the Tertiary Collection of Student Information (TCSI) system and through the HELP Information Technology System (HITS).

TEQSA has reduced its annual reporting requirements since 2012 and is continuing to work with the Department and other stakeholders to further reduce reporting burden, while maintaining its capacity to assure standards under a risk-based approach.

Overview

Purpose of risk assessments    

TEQSA’s risk assessments provide a snapshot of providers across the sector to help prioritise TEQSA’s focus in undertaking its assurance activities. They assist TEQSA to give effect to its principles of reflecting risk, proportionality and necessity, as outlined under the Tertiary Education Quality and Standards Agency Act 2011 (the TEQSA Act). They also inform risk-based regulation of providers under the Education Services for Overseas Students Act 2000 (the ESOS Act).

Through the RAF and use of risk assessments, TEQSA aims to:

• reduce burden on the sector by using risk assessments to inform a differentiated approach to evidence and reporting requirements in assessment processes (e.g. for renewal of registration and course accreditation applications)
• strengthen the protection of students’ interests and the sector’s reputation by monitoring key aspects of providers’ operations during registration periods
• support TEQSA case managers and providers to engage in early discussion about emergent issues prior to any scheduled assessment process
• support quality improvement activities through the sharing of information with providers about potential risks and good practices in the sector.

TEQSA’s risk assessments do not draw conclusions about compliance with the Higher Education Standards Framework (Threshold Standards) 2021 or the ESOS Act and National Code¹, but rather identify potential risks of non-compliance. In other words, risk assessments may identify ‘leads’ that warrant closer consideration by TEQSA case managers, but do not confirm that there is necessarily a problem. 

The purpose of the RAF is not to identify all institutional risk or to replace or replicate a provider’s own risk management. The RAF focuses on key risks across the sector that can be readily measured on a regular basis. TEQSA’s assessment processes, such as a renewal of registration, involve a deeper assessment of evidence to determine compliance with the Standards.

Approach

The RAF enables a consistent, structured and systematic approach to assessing risk across all providers. This is achieved by using a standard format and set of risk indicators across areas of institutional practice and outcomes that are central to all providers.

TEQSA recognises, however, the breadth of diversity in the sector and the importance of provider context in assessing potential risks. TEQSA also recognises that innovation often involves a degree of risk taking and does not consider risk as necessarily negative or that all risk must be controlled or eliminated. To support this in practice, TEQSA’s approach allows for expert judgement and consideration of providers’ history, context and own risk management within the risk assessment process. Dialogue between TEQSA’s case managers and providers about potential risks also enables TEQSA to better understand where risks may reflect strategic decisions taken by the provider for innovation and growth, and where risk controls are in place. TEQSA will look for evidence that the provider’s risk-taking is well managed, for example, through evidence of careful planning and using of pilots of the proposed innovation.

TEQSA’s risk assessments are predominantly focused at the institutional level, but may also consider risks relating to specific aspects of a provider’s operations, such as particular cohorts of students and/or areas of course offerings.

TEQSA’s approach to risk assessments is informed by the ISO Risk Management Standards, while adapted for TEQSA’s regulatory context and purpose.

Risk assessment process

Overview

TEQSA undertakes an annual cycle of risk assessments of all providers, following TEQSA’s PIR and acquisition of data from existing annual national collections where available.

TEQSA may choose to update a risk assessment outside of the annual cycle in response to emerging information. An overview of key steps in TEQSA’s risk assessment process is reflected in Figure 1 below and further outlined in the following sections.

Figure 1 – Key steps in risk assessment process

Steps in the assessment

Key steps in the risk assessment process, as reflected in the figure above, are:

1. Risk assessment

• TEQSA gathers existing information from various sources, mainly TCSI and HITS, national survey data, findings from TEQSA assessment processes, and information from the previous TEQSA risk assessment cycles
• TEQSA considers the history and context of the provider, its approach to delivery, and findings from previous assessment processes (positive and adverse)
• TEQSA undertakes an analysis of risk indicators, guided by risk indicator thresholds, trends, and other relevant context
• TEQSA undertakes a holistic evaluation of the history, context and indicator analysis to determine overall risk ratings, with explanatory notes where significant risk is identified.

2. Dialogue with provider 

• TEQSA will usually share a risk assessment with all providers each year, except for newly registered providers that have insufficient data to conduct a risk assessment
• each provider will have the opportunity to respond to its risk assessment which may lead to adjustments in the risk assessment
• if significant risks are identified, the TEQSA case manager will invite the provider to discuss the risk assessment and provide any broader context and information on its strategies and any risk controls in place. 

3. Next steps

• the finalised risk assessment is used to inform the scope of scheduled assessment processes (e.g. renewal of registration application processes) and, in some cases, may lead to further interaction with a provider ahead of an assessment process
• in cases where a provider has ongoing regulatory matters, the distribution of the risk assessment will be coordinated with TEQSA’s regulatory decisions to ensure consistency across all findings.

What the process involves for providers

It is optional for providers to respond to TEQSA’s annual risk assessments, unless specifically requested by TEQSA.

TEQSA undertakes the necessary data calculations and analysis to prepare risk assessments. TEQSA will issue a risk assessment to all providers.  A provider will receive a risk assessment with an invitation to comment. The provider may choose to provide additional information. If TEQSA considers it necessary to take further steps as a result of a final risk assessment, a meeting with the provider will be organised to discuss the providers risk assessment. 

A provider will receive a copy of its latest risk assessment where there is a forthcoming renewal of registration process.

Key components of risk assessments

Overview

An overview of the key components of a risk assessment is reflected in Figure 2 below and further detailed in the following sections. 

Figure 2 – Key areas considered in risk assessment

Overall risk evaluation

TEQSA makes an overall evaluation against: ‘Risk to Students’ and ‘Risk to Financial Position’. The evaluation uses a high, moderate or low rating (represented with traffic light colours). This is a qualitative expert judgement taking into consideration the provider’s context, history and standing, and analysis of risk indicators.

Where an overall evaluation is not able to be established due to lack of information or track record, conflicting information, or unreliable data, the overall rating may be suspended, or a rating of ‘No Confidence in Data’ applied.

Four key risk areas

TEQSA focuses on four key areas in risk assessments to support the overall evaluation:

1. regulatory history and standing
2. students (load, experience and outcomes)
3. academic staff profile
4. financial viability and sustainability.

Considered together, these areas provide coverage across key aspects of providers’ operations and all contribute to a view of potential risks to academic standards. In particular, the role of regulatory history in the risk assessment highlights any risks to academic standards identified through previous TEQSA assessment processes. This may include, for example, findings relating to:

• quality assurance processes in a past renewal of registration process
• admission practices in a past renewal of course accreditation process
• professional accreditation status through a material change notification.

The other key risk areas are informed through the assessment of a set of risk indicators.

Risk indicators

Risk indicators have been identified giving consideration to data availability (on an annual basis), applicability across the sector, and to different provider circumstances. The indicators, with descriptions and links to the Standards, are set out at Appendix 1. Supporting technical information on the indicators is provided at Appendix 2. The indicators are rated using a traffic light system.

A combination of input and output/outcome indicators are used, recognising that relying solely on output/ outcome indicators would mean a focus on the detection of confirmed failure, but not prevention. A combination of indicators also provides a more holistic view of a provider’s operations noting the limitations of individual indicators.

The assessment of indicators using student data includes a specific focus on any onshore and offshore international student populations (where possible). This allows a view of organisation-wide risk, as well as risk to these cohorts of students. An integrated approach of this kind is consistent with the considerable overlap between the ESOS Act and National Code, and the TEQSA Act and Threshold Standards.

Risk thresholds

In assessing risk indicators, TEQSA considers a set of risk thresholds, while taking into account provider context and risk controls (where information is available).

TEQSA has adopted a systematic approach to developing its risk thresholds, which includes consideration of the following dimensions:

1. Reference material

Documentation such as past regulatory and quality assurance reports, and providers’ risk management and strategic plans, can provide views on common issues such as attrition and student-staff-ratio.

2. Statistical analysis of the sector

Status quo and trends in the sector can shed light on the discriminating power of a risk threshold.

3. Experience from previous risk cycles

TEQSA’s experience of applying the risk thresholds can help to ascertain their efficacy.

4. The nature of indicators

Consideration of the different nature of indicators can inform whether the indicators lend themselves to a more absolute setting of risk thresholds or whether more emphasis is given to levels that vary from sector trends. For example, if the sector average attrition rate was significantly increasing, TEQSA may take a view that this does not alter the level considered to indicate a risk to standards.

These dimensions are evaluated holistically, based on available information, and there is no single consideration that would automatically overrule others. Professional judgement is used, with regard to the specificities of each indicator, in determining the levels which may represent potential risk.

Further information about TEQSA’s approach to determining risk thresholds is made available on its website and will be updated as needed. The risk thresholds themselves are held confidentially within TEQSA. Risk thresholds are considered in the context of other information and are not the sole determinant of risk ratings.

In the risk assessment and in any communication with the provider, TEQSA will explain the basis for an overall moderate or high risk rating in the context of the provider’s particular circumstances.

Risk controls 

As noted earlier, if a risk assessment identifies potential concerns that may warrant further consideration by TEQSA, a provider is invited to comment on the assessment, on a voluntary basis. The provider may comment on the factual accuracy underpinning the observations, provide relevant information about risk controls that it has in place in relation to the potential risks identified, or any other information that the provider considers relevant.

TEQSA’s consideration of the provider’s response may lead to an adjustment of the risk assessment. Examples of evidence and context that may lead to adjustments of risk ratings are available in a published information sheet on TEQSA’s website, and may be updated from time to time.

Noting that innovation often involves a degree of risk, a provider may choose to demonstrate that the level of risk is acceptable in its circumstances.

Outcomes of risk assessments

Actions in response to risk assessments

A final risk assessment will typically identify action in line with the following:

No action	If no significant risks are identified overall, or risks are already known to TEQSA with a response already in place (such as additional reporting requirements), then TEQSA will not take any action in response to the risk assessment. The risk assessment will continue to be updated annually.
Recommendation	TEQSA may recommend that the provider closely monitor identified risks and/or put in place appropriate controls or improvement strategies. A recommendation arising from a risk assessment does not constitute a formal condition on registration.
Request for information	TEQSA may identify risks that require further consideration by TEQSA. In such cases, TEQSA may seek additional information from the provider so that TEQSA may determine if further action is necessary. Requests for information may also be used to monitor identified risks between risk assessment cycles.
Regulatory action (e.g. compliance assessment or conditions)	If TEQSA identifies significant risks, it may determine that regulatory action is necessary outside a scheduled assessment process. This may include, for example, undertaking a compliance assessment to satisfy TEQSA that the provider continues to comply with the Threshold Standards, or imposing formal conditions on registration.
To be considered in scheduled assessment process	If the provider has a scheduled assessment process (e.g. re-registration), TEQSA may indicate that risks identified in the risk assessment will be considered further in that process rather than identify additional action at that time.

Links with scheduled assessment processes

A risk assessment is one input to inform the scope of evidence required in renewal of registration or course accreditation processes.

If a provider is evaluated as low risk overall in relation to Risk to Students and Risk to Financial Position (and satisfies other criteria), then the application and assessment process focuses on reduced core evidence requirements. If a provider is evaluated as presenting a high or moderate risk overall in relation to Risk to Students and/or Risk to Financial Position, then the scope of the assessment process may be expanded. In exceptional circumstances, an expansion may be considered necessary where a provider is evaluated as low risk overall, but a significant trend or specific issue is identified. TEQSA case managers determine the scope and discuss requirements with the providers.

Further information about TEQSA’s approach to tailored renewal of registration and course accreditation processes is available on the TEQSA website.

It should be noted that, as scheduled assessment processes are more in-depth and consider wider evidence, it is possible for a scheduled assessment to identify compliance issues that had not previously been identified as potential risks in a TEQSA risk assessment.

Administration

Privacy and confidentiality

Given the potential sensitivity of risk assessments and associated documents, provider risk assessments are treated confidentially by TEQSA. Risk assessments and associated documents relating to individual providers are not publically released by TEQSA or shared with other providers. Similarly, a TEQSA risk assessment is to be treated confidentially by the provider, noting that the provider may not publish a risk assessment or make it available to any person other than those employed by the provider. TEQSA may share risk assessments with other government agencies (refer to ‘Information Sharing’).

While TEQSA has certain statutory obligations of confidentiality, pursuant to Division 2 of Part 10 of the TEQSA Act, providers should note that TEQSA also operates within a public accountability framework. This includes obligations:

• to provide information to Ministers, the Parliament or Parliamentary Committees
• under the Freedom of Information Act 1982, the Auditor-General Act 1997, and the Ombudsman Act 1976
• to provide reasons for TEQSA’s decisions, or details about TEQSA’s activities, including in the context of court or tribunal proceedings.

If TEQSA receives a request to provide an applicant’s confidential information, TEQSA will endeavour to consult the applicant, and to provide the applicant with an opportunity to make submissions on whether TEQSA should release the information. However, in certain cases this course of action may not be possible.

Freedom of information

TEQSA is subject to the Freedom of Information Act 1982 (the FOI Act). TEQSA will respond to requests for access in accordance with the requirements of the FOI Act.

Further details on freedom of information

Information sharing 

A key function of TEQSA’s establishment as the national higher education quality assurance agency includes disseminating information about higher education providers and their awards. This function is specified in paragraph 134 (1) (e) of the TEQSA Act, which notes that TEQSA may collect, analyse, interpret and disseminate information relating to higher education providers, regulated higher education awards and for quality assurance practice and improvement in higher education.

To provide a broad overview on risks in the higher education sector and to share information on good practices, TEQSA may publish high-level sector analyses. Any analysis that is published will be at a high level only and will not contain any provider level risk information. Information on risks in the sector and good practices may also be shared through information sheets on TEQSA’s website and presentations at TEQSA provider roundtables.

TEQSA may share risk assessments, or components of risk assessments, with other Commonwealth agencies (e.g. Australian Skills Quality Authority and the Department of Education and Training) where there is an established need and where it reduces the reporting or compliance burden on providers. Any sharing of risk assessments with other Commonwealth agencies will be established under appropriate arrangements (eg Memoranda of Understanding or Information Sharing Protocols) with the relevant agency.

In considering any requests to share risk assessments or their components, TEQSA will give due regard to all confidentiality provisions through which the agency obtained this information from a provider. This means that risk assessments would not contain identifying personal information on individual members of organisations, their staff or students.

Appendix 1 and 2

• PDF (167 KB) (please contact comms@teqsa.gov.au if you have any problems accessing the document)

 

¹ National Code of Practice for Registration Authorities and Providers of Education and Training to Overseas Students 2018.