AI Incident Monitor - Mar 2025 List

AI BREACHES (1)

1 - USA Based AI Systems Under Scrutiny for Extensive Data Collection

The Briefing

According to a ZDNet report drawing on research from Surfshark, some US-based AI systems, think Google Gemini and friends are collecting more data than a nosy neighbor with binoculars. We're talking location, browsing history, even your contacts list. And just to spice things up, about 30% of tested apps are generously sharing that data with third parties too. It’s like hosting a private dinner party and finding out someone live-streamed it🤭.

Potential AI Impact!!

•  It affects the AI Principles of Privacy and Data Governance 

•  The Severity classification for AI Breach 1 is Non-Physical Harm

• Harm Type - Human Rights

• Affected Stakeholders: Consumers

Why is it a Breach? 💁

The article dives into how AI systems are hoovering up user data like it’s going out of fashion, raising eyebrows over potential privacy violations. That said, there’s no smoking gun: no confirmed harm, no legal tripwires triggered (yet). So, while no one’s calling the lawyers just now, it’s firmly in AI Hazard territory. Think of it as a privacy powder keg - no explosion yet, but the fuse is looking a little too ready.

AI BREACHES (2)

2 - French Copyright Lawsuit: Meta’s Unauthorized AI Training

The Briefing

In a very French plot twist, authors and publishers—through groups SNE, SGDL, and SNAC have taken Meta to court in Paris. Their claim? Meta allegedly helped itself to copyrighted works to train its generative AI, skipping the whole “permission” part. The lawsuit accuses Meta of intellectual property trespassing and economic misconduct. In other words, the literary world is saying: “Merci, but non merci.”.

Potential AI Impact!!

•  It affects the AI Principles of Accountability and Fairness

•  The Severity classification for AI Breach 2 is Non-Physical Harm

• Harm Type - Economic/Property

• Affected Stakeholders: Business

Why is it a Breach? 💁

This event centers on Meta allegedly feeding copyrighted texts into its AI training diet without asking first. If true, it’s not just bad manners; it potentially breaches intellectual property law. That bumps it up to an AI Incident status, since it involves a suspected violation of legal obligations designed to protect creative rights (and keep lawyers gainfully employed)!

AI BREACHES (3)

3 - US School AI Surveillance Sparks Privacy Breach

The Briefing

In an effort to keep students safe, Vancouver Public Schools and other US districts rolled out AI-powered monitoring tools to watch over online activity. Unfortunately, the digital hall monitor tripped over its own shoelaces, accidentally exposing nearly 3,500 sensitive student records. The result? A crash course in unintended consequences, and a wave of privacy and security concerns no one signed up for.

Potential AI Impact!!

•  It affects the AI Principles of Privacy & Data Governance

•  The Severity classification for AI Breach 3 is Non-Physical Harm

• Harm Type - Human Rights

• Affected Stakeholders: General Public

Why is it a Breach? 💁

AI-powered surveillance in schools, designed to monitor students’ online behavior might be crossing more than just digital boundaries. While intended for safety, these systems can flag personal issues and alert staff, raising serious questions about privacy, confidentiality, and students' rights to express themselves freely. In short: protecting students shouldn't mean putting their rights on detention.

AI BREACHES (4)

4 - AI-Driven Price Discrimination Sparks Consumer Outcry

The Briefing

Consumers are calling out what feels like déjà vu with a markup. Across travel, retail, and lodging platforms, AI-driven pricing algorithms appear to be playing favorites (and not in a good way), hiking prices for returning users. The result? Accusations of dynamic pricing morphing into digital discrimination. With complaints piling up, regulators are circling and transparency in algorithmic pricing is moving from nice-to-have to non-negotiable.

Potential AI Impact!!

•  It affects the AI Principles of Transparency & Explainability, Fairness

•  The Severity classification for AI Breach 4 is Non-Physical Harm

• Harm Type - Economic/Property

• Affected Stakeholders: Consumers

Why is it a Breach? 💁

The article highlights a case of ‘big data price discrimination,’ where AI systems tailor prices based on user profiles, think personalized pricing, but not in a good way. By mining user data to adjust prices, this practice raises serious concerns about fairness and transparency. It may cross the line into consumer rights violations, potentially breaching legal obligations designed to protect fundamental rights. In short: when AI plays favorites, regulators and customers start paying attention.

AI BREACHES (5)

5 - DeepSeek AI Vulnerability Enables Malware Code Generation

The Briefing

Tenable researchers showed that the DeepSeek R1 model has a bit of a gullible side - it can be coaxed into generating malware like keyloggers and ransomware with carefully worded “for educational purposes” prompts. While it might pass as a teachable moment, the real lesson is a serious one: this loophole could be exploited by bad actors, flagging a clear AI security hazard that needs attention before it’s more than just theoretical.

Potential AI Impact!!

•  It affects the AI Principles of Robustness & digital security, Safety

•  The Severity classification for AI Breach 5 is Hazard

• Harm Type - Economic/Property

• Affected Stakeholders: General Public

Why is it a Breach? 💁

This shows how the DeepSeek AI model can be manipulated into generating malware code, essentially turning a helpful assistant into a potential cyber accomplice. While it hasn’t gone rogue, the risk is real: if misused, this capability could lead to legal violations and real-world harm. It squarely fits the definition of an AI Hazard, a system with just enough flexibility to be dangerous in the wrong hands.

AI BREACHES (6)

6 - CCTV Exposes AI-Powered Harassment Call Scandal

The Briefing

CCTV uncovered that companies, particularly Shanghai销氪 Information Technology, have been using AI-powered systems to make mass marketing calls—crossing privacy lines and exploiting synthetic human voice recordings. Tied to a micro-sales group, the company is now conducting a full review and working with regulators, as public concern over unsolicited AI-driven calls continues to rise.

Potential AI Impact!!

•  It affects the AI Principles of Privacy & Data Governance

•  The Severity classification for AI Breach 5 is Hazard

• Harm Type - Public Interest

• Affected Stakeholders: Consumers

Why is it a Breach? 💁

The event highlights how companies have used AI systems to automate marketing calls, resulting in breaches of privacy and consumer rights. This qualifies as an AI incident, as it involves AI usage that violates legal obligations designed to safeguard fundamental rights.