AI Incident Monitor - Sep 2025 List

AI BREACHES (1)

1- AI Incident: Deloitte Refunds Australian Government After GPT-4o Hallucinations in Official Report

The Briefing
Deloitte issued a partial refund to the Australian government after its AI-assisted report on the welfare compliance system was found to contain fabricated citations and factual errors caused by hallucinations from OpenAI’s GPT-4o. While Deloitte stated the report’s key findings were unaffected, the incident raises serious concerns about AI reliability, quality assurance, and accountability in official documentation. The AI system’s direct role in producing false information that influenced a government deliverable qualifies this as an AI Incident involving realized reputational and financial harm.

Potential AI Impact!!

 ✔️ It affects the AI Principles of Robustness & digital security, Safety,, Transparency & explainability, Accountability

 ✔️ Industries Affected are Government, security, and defence.

 ✔️ The Severity is AI Incident

  ✔️ Harm Type Economic/Property, Reputational

💁 Why is it a Breach?

This incident involves the use of a Gen LLMs in preparing a government report. The AI system’s outputs directly resulted in the inclusion of fabricated references and quotes, constituting misinformation within an official document. This misinformation represents realized harm - undermining public trust, breaching accuracy and transparency obligations in public administration, and potentially misleading policymakers and stakeholders. Because the harm is both realised and directly attributable to the AI system’s output, the event meets the definition of an AI Incident, rather than a hypothetical risk. It highlights the accountability challenges and due diligence required when integrating generative AI into government reporting and decision-making processes.

AI BREACHES (2)

2 - RealPage Rent-Setting Algorithm Leads to $141M Settlement Over Price-Fixing Allegations

The Briefing

Greystar and 25 other property management companies in the USA have agreed to pay over $141 million to settle a class action lawsuit alleging collusion through RealPage’s AI-driven rent-setting algorithms. The lawsuit claimed that the AI system facilitated coordination and rent inflation across competing property owners, resulting in economic harm to tenants and violations of competition law. The settlement includes an agreement to cease sharing nonpublic rental data with RealPage.

Potential AI Impact!!

 ✔️ It affects the AI Principles of Accountability, Fairness, Privacy & data governance, Transparency & explainability

 ✔️ The Severity is AI Incident

✔️ Industries Affected is Real Estate.

 ✔️ Harm Type is Economic/Property

💁 Why is it a Breach?

The RealPage software functions as an AI-driven revenue management and rental pricing system. Its use allegedly enabled unlawful coordination and rent inflation among property managers, leading to economic harm to renters and violations of antitrust laws protecting legal rights. The harm is realized, demonstrated by the class action lawsuit and resulting $141 million settlement. The AI system’s role in generating coordinated pricing recommendations and facilitating data sharing was central to the incident. The case demonstrates realized economic harm caused by the use of an AI system to manipulate market outcomes, fulfilling the criteria for an AI Incident due to both harm to consumers and breach of legal obligations under antitrust and consumer protection laws.

AI BREACHES (3)

3 - NSW Reconstruction Authority Data Breach After Sensitive Information Uploaded to ChatGPT

The Briefing

A former contractor for the NSW Reconstruction Authority uploaded a spreadsheet containing personal and health data of approximately 3,000 flood aid applicants to ChatGPT, an AI system not approved for handling sensitive information. This action resulted in a major data breach, exposing highly confidential personal data and prompting an investigation by Cyber Security NSW. While no public data access has been confirmed, the unauthorized transfer and processing of sensitive personal data itself constitutes realized harm and a breach of privacy obligations under applicable law.

Potential AI Impact!!

 ✔️ It affects the AI Principles of Accountability, Privacy & data governance

 ✔️ The Industries affected Government, security, and defence

 ✔️ Harm Type is Human or fundamental rights

✔️ The Severity AI Incident

💁 Why is it a Breach?

The event involves the misuse of ChatGPT, an AI system, by a contractor who uploaded sensitive personal and health data belonging to thousands of flood aid applicants. This action resulted in an unauthorized disclosure of private information - a realized harm affecting individuals’ privacy and data protection rights. The breach constitutes a violation of legal obligations protecting fundamental rights under privacy and cybersecurity laws.

AI BREACHES (4)

4 - AI-Generated Fake Recruiters Fuel Surge in US Employment Scams

The Briefing

The event explicitly involves the use of generative AI systems to create fake recruiter identities and job offers. These AI-generated materials are used to deceive job seekers, resulting in realised financial and emotional harm. The AI systems are central to the scam’s effectiveness, enhancing deception and scalability. This constitutes a violation of rights and legal obligations related to fraud and consumer protection. Therefore, this qualifies as an AI Incident due to direct, AI-enabled harm to individuals.

Potential AI Impact!!

 ✔️ It affects the AI Principles of Accountability, Safety, Transparency & explainability, Democracy & human autonomy, Respect of human rights

 ✔️ The Affected Industries Business processes and support services

 ✔️ Harm Type Economic and Property

✔️ Severity AI Incident

💁 Why is it a Breach?

The event involves the use of generative AI systems to fabricate realistic job offers and recruitment processes, resulting in realized harm, notably financial losses to individuals and heightened cybersecurity risks for employers. The AI’s role is central, enabling the scale, believability, and automation of these fraudulent schemes. The incident represents harm to individuals (financial injury) and harm to communities (systemic fraud and security threats), fulfilling the criteria for an AI Incident.

AI BREACHES (5)

5 U.S. Senate Report Warns of AI-Driven Job Displacement, Proposes ‘Robot Tax’

The Briefing

The event involves projected risks from AI adoption rather than realised harm. The report warns that AI could replace up to 100 million jobs within a decade, affecting diverse sectors. Proposed measures, such as a ‘robot tax,’ aim to mitigate potential economic and social impacts. Since this reflects anticipated consequences and not actualised harm, it does not meet the criteria for an AI Incident but serves as a critical AI hazard warning for policymakers and businesses.

Potential AI Impact!!

 ✔️ It affects the AI Principles of Human Welbeing

 ✔️ The Affected Stakeholders Workers

✔️ The Affected Industries Other

 ✔️ Severity AI Hazard

💁 Why is it a Breach?

The event describes a plausible future harm scenario in which AI adoption could displace up to 100 million jobs across white- and blue-collar sectors in the U.S. over the next decade. As no actual job losses have yet occurred, and the article primarily serves as a warning and strategic guidance, this does not constitute a realised harm or AI Incident. It represents a credible AI Hazard, highlighting potential economic and societal risks that may require mitigation measures such as retraining programs or policy interventions like a proposed ‘robot tax’.

AI BREACHES (6)

6 - California AG Sues El Cajon Over Unlawful ALPR Data Sharing

The Briefing

The event involves automated license plate reader (ALPR) systems, AI-driven tools that collect and process sensitive personal data. The California Attorney General alleges that El Cajon’s police department shared ALPR data with out-of-state and federal agencies, violating state privacy laws and endangering the rights of individuals, particularly vulnerable populations. The unauthorized use and dissemination of AI-processed data constitutes realized harm and legal violations, fulfilling the criteria for an AI Incident rather than a potential hazard or informational update.

Potential AI Impact!!

✔️ It affects the AI Principles of Privacy & data governance, Respect of human rights, AccountabilityTransparency & explainability

 ✔️ The Affected Stakeholders General Public

✔️ The Affected Industries Government, security, and defence

 ✔️ Severity AI Incident

💁💁 Why is it a Breach?

This unauthorised data sharing constitutes realized harm, breaching legal obligations and endangering individuals’ fundamental rights. The event is classified as an AI Incident, as it involves the misuse of an AI system leading directly to legal and privacy violations.