---
title: "AI Incidents Policy"
source: https://www.jemoka.com/posts/kbhai_incidents_policy/
---

Talk by Lisa Soder
AI Regulation EU GPAI Code of Practice: makes fronter model providers track, document, and report AI incidents SB 53 Large Developer: identify and respond to safety incidents + internal governance Problem: define what it means to &ldquo;track, document, and report&rdquo;?
Methods Goals help companies with compliance help regulators for assessing compliance technical tooling Approach legal analysis best practices in AI best practices in other domains What is an AI incident? &ldquo;In house evaluation is not enough.&rdquo;
proactive monitoring is important for things that don&rsquo;t have external signals (i.e. cyber espionage) methods activation probes output classifiers LLM-as-judge human labeling for certain cases clio (https://arxiv.org/abs/2412.13678) Good Rood-Cause Analysis Similar outcomes can come from very different causes, so you need to good data to contextualize.
&hellip;but data retention processes don&rsquo;t support incident analysis (because people don&rsquo;t retend data for more than 30 days; models maybe self hosted)
