Gen AI strategies for Australian higher education: Emerging practice (Q&A)

Questions and answers from TEQSA webinar on 28 November 2024

1. What is TEQSA’s position on students using generative artificial intelligence (gen AI) during their studies for preparing work or conducting research?

It is each provider’s responsibility to determine appropriate and permissible use of gen AI within specific disciplines, courses, units or assessment items. Thoughtful integration of gen AI tools can facilitate new avenues for collaboration between students and educators, foster a more interactive learning environment and enhance the student experience.

Providers need to establish consistent, high-level messaging regarding gen AI use in teaching, learning and research. Legitimate differences in approaches to gen AI use across programs, courses or assessment items, need to be clearly communicated to students to avoid confusion.

Additionally, if students are required to use gen AI, providers need to be mindful of the potential to disadvantage those who cannot afford multiple gen AI subscriptions.

2. Is TEQSA able to provide general advice to Registered Training Organisations (RTOs) and higher education providers who might not have a culture of adjusting and amending their assessment design?

Regularly reviewing and renewing their suite of assessments enables providers to safeguard the fairness and validity of their approach and the integrity of awards.  Providers with robust and responsive mitigation processes will be best placed to adapt to the impact of new or evolving risk.

To keep up to date with industry requirements and protect the integrity of awards, providers must genuinely engage with the risks and opportunities gen AI technology poses to their course offerings. This includes adjusting and amending assessment types that are identified as high risk and updating learning outcomes to ensure students are developing industry-relevant knowledge and capabilities.

The toolkit reflects emerging practice in assessment security and transformation in its Practice section, providing examples of how some institutions are assuring that students meet the learning outcomes and required capabilities for their award.

3. Are there any concerns with using gen AI tools for assessment marking purposes?

TEQSA’s request for information did not specifically include information relating to institutional use of gen AI for marking.

The extent to which providers choose to engage with gen AI – including as a tool to assist with marking – is an institutional decision, which reflects their strategic goals and stance towards this technology.

Where providers incorporate gen AI technology into assessment or administrative processes, they need to ensure they are aligned to institutional and legislative requirements regarding data security, privacy and protection of intellectual property.

If, following a comprehensive risk assessment, a provider decides to make gen AI permissible for marking purposes, they need to ensure this decision is reflected in relevant policies, and that quality assurance and oversight measures are in place.

Clear messaging around specific instances of when and how gen AI may be used, such as marking or administrative functions, must be communicated to all staff and students.

4. Were international comparisons undertaken in preparation of the published resource?

TEQSA did not undertake international comparisons when compiling the toolkit. The aim of the toolkit is to highlight emerging practices within the Australian higher education sector and share examples of the different ways providers are integrating gen AI into their operations.  However, TEQSA continues to engage with its international counterparts to share knowledge and exchange information to inform our own work.

5. Does the toolkit provide advice on how to adapt to ongoing technological advancements?

To maintain both their institutional momentum for reform and the relevance of their courses, providers need to commit to staying abreast of gen AI’s rapid development. This includes, conducting periodic reviews to assess whether processes are keeping pace with the opportunities and risks presented by ongoing technological advancements.

Given the broad applicability of gen AI technologies, providers will need to critically review the integrity of the full breadth of their course offerings and delivery modes. The way each provider achieves this will be guided by their institutional strategy and have regard to their existing processes for managing risk.

To assist providers, the toolkit presents examples of different approaches which have been taken to embrace the opportunities and mitigate the risks gen AI poses to assessment security and award integrity.

6. Institutions seem to be restrained in their adoption of gen AI. In TEQSA’s opinion, are we falling behind industry in the delayed uptake?

With the increasing integration of gen AI tools in various professions, graduates will need to be equipped with the knowledge to effectively and ethically use this technology in their future careers. Additionally, as more industries adopt the use of gen AI tools, professional accreditation bodies may adapt their requirements. Maintaining effective bilateral communication and staying abreast of industry requirements will help providers update their courses and learning outcomes to ensure the skills and knowledge their graduates are developing align with industry expectations.

The toolkit offers examples of providers’ strategies to engage with industry in its People section, and examples of assessment, unit and course revision measures in its Practice section.

7. How is TEQSA’s work relating to gen AI affected by the Senate’s Select Committee on Adopting Artificial Intelligence’s final report, which was published recently?

8. What is TEQSA’s view on programmatic assessment to assess student learning?

Many of the traditional forms of assessments which have been used as proxy evidence of student learning can now be successfully and quickly produced by gen AI. Therefore, it is important for providers to review their current teaching and assessment practices and, where necessary, transform their assessment regimes to ensure students have demonstrated attainment of the skills and knowledge reflected by their award.

There is no single form of assessment that can enable students to both demonstrate achievement of all their learning outcomes and support them in developing skills to ethically and effectively use gen AI. Similarly, no single tool or technology can guarantee assessment security. Providers must therefore look at how the different methods of assessment across a course can be used holistically, with the inclusion of highly resourced and authentic assessments at key points over the course of a degree.

Providers are encouraged to revisit the principles and propositions contained in the document Assessment Reform for the Age of Artificial Intelligence and consider the most effective way to prioritise assessment security at key moments of a student’s program.

9. Do you think institutions need a dedicated role responsible for gen AI, to drive initiatives, monitor developments and keep up with rapid changes?

Given the huge diversity in the size and scale of Australian higher education providers, there will be range of institutional strategies to manage gen AI responsibilities based on operational requirements, constraints and opportunities.

To establish, implement and monitor an institution-wide approach to gen AI it is important that staff have clarity about roles and responsibilities with clearly defined oversight mechanisms at the appropriate level of delegated authority. If no person or group is formally tasked, as part of their official workload, to implement, monitor and document proposed changes, providers may fail to implement the necessary actions required to maintain award integrity and provide assurances that students have met their learning outcomes.

10. What is the role of Communities of Practice in integrating gen AI into teaching practices?

Communities of Practice (CoP) are an effective way to utilise internal and external expertise and create opportunities for staff to exchange knowledge and practical approaches to teaching and learning. Although CoPs do not directly affect institution-wide decision making and strategy, they are an excellent way to foster collaboration and support a culture of academic integrity.

Providers should encourage staff to interact across their institutions to share information as well as joining external networks such as the Australasian Academic Integrity Network.

11. Why does the toolkit seem to focus more on gen AI risks rather than opportunities?

TEQSA acknowledges that gen AI tools offer both opportunities and risks to providers’ course offerings and operations. However, as the national regulator TEQSA’s purpose is to deliver quality assurance that protects the interests of students and the reputation and standing of Australian higher education. With that in mind, the aim of the toolkit is to support providers in improving their processes and approaches to teaching and learning.

By mitigating the risks gen AI poses to the security of existing assessment types and teaching students to be effective and ethical users of these tools, providers can have assurance that their students have met their learning outcomes and attained the relevant skills needed for their chosen profession.

12. Are you able to provide information on how many providers still rely on gen AI detection software, given that it is problematic and outdated?

While TEQSA’s analysis of submissions included providers’ use of detection tools, any quantifying information would only represent a snapshot in time and would not account for the accompanying measures to protect award integrity. TEQSA’s position is to understand and collaborate with the sector on utilising gen AI technology ethically while protecting award integrity.

Providers who choose to use gen AI detection software, as one of their risk mitigation strategies, need to be mindful that these tools alone do not provide sufficient assessment security. By installing a ‘humanise’ plugin or writing the appropriate prompt, gen AI created content can be manipulated to bypass detection tools, making it difficult to identify. Additionally, providers need to be aware of, and seek to mitigate, the risks of false positives and unintended bias associated with AI detection software.

The toolkit provides details on assessment security and transformation in its Practice section.

13. How can TEQSA and the sector provide agility in their responses to gen AI to ensure that students are not left behind?

Frequent monitoring and shortened cycles for the review of course offerings, assessments, and teaching and learning approaches, will allow providers to prioritise risks and inform the staged roll out of intervention strategies. A robust policy framework with responsive processes will assist providers in maintaining institutional momentum to offer students enhanced teaching and learning experiences relevant to their chosen field.

It is also important to understand student and staff expectations, requirements and concerns about the use of this technology.  Active participation of staff and students in advisory committees or working groups ensures that their perspectives are considered in decisions about gen AI that impact them.

14. Will there be a way to capture continual learning or evaluation of the effectiveness of measures arising from the toolkit?

Given the speed of technological innovation institutional strategies can quickly become outdated. Providers need to ensure their review and evaluation cycles are frequent enough to keep pace with the ongoing developments of gen AI and other emerging technologies. Ongoing monitoring and evaluation of measures to mitigate risks to course and assessment offerings, and teaching and learning practices is good governance and a requirement under the Higher Education Standards Framework (Threshold Standards) 2021.

Providers need to periodically examine whether their processes are keeping pace with the evolving opportunities and risks presented by ongoing developments of gen AI technologies. Staying up to date with gen AI advances will help ensure awards are legitimately conferred, and meet the expectations and needs of students, industry and employers.