The Significance of the EU AI Regulation for the University of Regensburg

Initial guidance on various matters can be found in the following documents:

• Examination regulations regarding the use of AI at the Faculty of Physics
• Guidelines on the use of artificial intelligence in the preparation of written examination papers at the Faculty of Law
• Guidelines on the use of artificial intelligence at the Faculty of Catholic Theology
• Data protection regulations of the UR Data Protection Office
• Support from ZHW
• GRIPS course ‘Toolbox for Digital Teaching’
• Guide to Generative Artificial Intelligence

Practical, useful AI tools for various application scenarios.

The AI Regulation is therefore highly relevant to the UR in two respects:

• On the one hand, the UR provides its members with AI systems ready for use – for example, Microsoft Copilot, a tool that utilises ChatGPT and can therefore be used to draft or summarise texts, or DeepL as a translation aid within the TYPO3 system – and is therefore, within the meaning of the regulation, a controller of AI systems (Article 3 of the AI Regulation).
• On the other hand, UR members may, as part of their research or teaching activities, independently develop AI systems, some of which may also utilise existing AI systems (such as ChatGPT). If these in-house developments are used in a productive capacity – for example, to implement an AI-based chatbot as part of a commercial collaboration or to assign students to study groups – then the UR acts as a provider of AI systems within the meaning of the regulation (Article 3 of the AI Regulation).

The EU Regulation came into force upon its publication on 1 August 2024 and will be implemented in phases. Six months after the AI Regulation comes into force, on 2 February 2025, the first bans and regulations will take effect.

At present, members of the UR

• acquire the necessary skills in the use and handling of AI systems if they (potentially) use such systems. Relevant training courses specifically on the subject of AI can be found on this page.
• ensure that no AI systems involving prohibited practices are put into productive use. It is irrelevant whether the AI system is operated solely within the university (see the first example of prohibited practices) or whether it is an AI system that is publicly available and usable.

With the phased implementation of the AI Regulation, members of the UR will face further obligations in future. Information on this will be published on this page. A set of guidelines for the introduction of new AI systems will also be made available shortly.

Consequences will follow if the regulations are disregarded. Specifically,

• if the training requirement is not met and damage is caused to the UR as a result of an AI system being used by an unqualified person,
• if systems employing prohibited AI practices are used in live operations and process real-world data.

Failure to comply with the training requirement or the prohibitions is particularly problematic in the event of damage, as this is often the trigger for administrative fines in practice. However, damage is not a mandatory requirement. Breaches of the duties of conduct may also trigger a sanction under the AI Regulation.

Finally, it should be noted that the AI Regulation supplements existing regulations, such as the GDPR in particular, which govern the protection of personal data. Breaches of these regulations when using an AI system may also lead to sanctions.

The use of AI systems also carries risks, as their decisions may be based on algorithms that are, in some cases, opaque, on training data that may be flawed or biased, and on decision-making structures that cannot be verified; as a result, they may be incomprehensible or incorrect. This can lead to injustice, discrimination and manipulation, particularly in sensitive areas such as the justice system, the workplace or public spaces. As erroneous or biased decisions made by AI often cannot be traced or corrected, there is a risk that individuals will be treated unequally or unfairly without a fair and transparent assessment process being guaranteed. To avoid these risks, the EU’s AI Regulation specifically prohibits certain AI practices that threaten fundamental rights and freedoms.

Consequently, prohibited AI practices relate only to applications of AI in operational settings, i.e. where harm to individuals may occur.

• Explicitly excluded from the AI Regulation are AI systems developed and used solely for the purposes of scientific research and demonstration.
• AI systems used in teaching, which are expressly employed solely for the purpose of illustrating the functionality of AI (including, where applicable, prohibited practices), are not explicitly excluded from the scope of the AI Regulation. Their use is not prohibited across the board. They are subject to obligations of conduct, including, where applicable, the transparency obligations mentioned above.

This is without prejudice to the ethical responsibility of researchers and teachers.

The following AI practices and systems are, in principle, prohibited under the AI Regulation in the listed areas of application (see Chapter II, Article 5, AI Regulation). The examples are provided for illustrative purposes and are not exhaustive.

Subliminal influence, deliberately manipulative techniques

Example: An AIprovides personalised recommendations for learning materials or courses. The AI suggests specific courses or content based on economic or institutional interests rather than on the students’ actual needs.

Exploitation of a person’s vulnerability

Example: An university uses AI-powered exam software that employs facial recognition and eye-tracking to check whether students are cheating during online written exams. Students with certain disabilities (e.g. visual impairments or motor disabilities) have a higher error rate in this context because they behave differently and may therefore exhibit patterns that lead the AI-powered software to suspect them of cheating. The use of AI in examination software thus leads to significant discrimination against this group of students and exploits their vulnerability.

Discriminatory classification based on social behaviour, personal characteristics or personality traits

Example: An AI system is used to assess applications for a funding programme or a specific degree program. In addition to academic qualifications, the AI also takes personality traits into account when assessing applications in order to allocate students to the funding programme or degree program. However, due to the ‘black box’ nature of AI, it remains unclear exactly how these classifications are arrived at. Furthermore, bias within the training data could lead to certain personality traits being systematically rated more poorly. For example, less extroverted individuals could be disadvantaged by the AI algorithm, even though this personality trait has no negative impact on the task to be performed.

Detection of emotions in individuals in the workplace and in educational institutions

Example: An AI system is used to assess performance during a presentation. The AI measures pitch, tone of voice and rate of speech to detect nervousness or self-confidence and, based on this, to assess whether someone appeared convincing, uncertain or anxious. This creates a sense of surveillance for the individual, as misinterpretations by the AI can also lead to incorrect assessments of the person.

Database for automated facial recognition

Example: An AI-powered facial recognition system is being used, which utilises cameras across the entire campus and accesses a database containing the facial features of all enrolled students. Every movement made by the students is recorded and any conspicuous behaviour is analysed.

Biometric categorisation of individuals in sensitive areas

Example: An AI system is used to analyse employees’ typing behaviour or facial expressions and gestures (visual analytics) in order to draw conclusions about their mental health or possible disabilities. Such information could be misused for personnel decisions.

Predictive Policing

Example: Video footage from CCTV cameras is analysed using AI to predict potential security risks. The AI identifies ‘potentially suspicious’ movement patterns or groups on campus and uses comparative data from previous incidents to predict who might potentially commit offences such as vandalism or theft. This could lead to individuals being wrongly accused of having malicious intentions.