From the filing
PDF page 58
Printed filing page 47 · Extracted page text
Read the page text
Table of Contents errors, misalignment, or security exploits may result in real-world consequences—including consequences that may be difficult or impossible to reverse, such as committing code changes, deleting or modifying data, or completing financial transactions—before effective intervention is possible. In an agentic workflow, an error or unintended action at any individual step may propagate through subsequent steps, compounding the severity of harm before detection or correction. The pace and volume of autonomous execution, particularly in enterprise deployments, may also exceed the capacity of human reviewers to meaningfully understand and evaluate whether each action is appropriate. The effectiveness of human oversight over our agentic offerings may be limited by the speed and volume of autonomous agent actions, the complexity of multi-agent and multi-step workflows that may make it difficult for reviewers to assess whether an agent’s actions are appropriate, and organizational pressures within customer environments to maximize agent autonomy in order to achieve productivity gains. Human reviewers may experience alert fatigue, may lack sufficient context to evaluate agent decisions, or may not review agent actions in time to prevent harm, particularly in high-volume enterprise deployments where agents operate concurrently across multiple workflows or in manners outside of a human’s awareness. In addition, agentic systems may pursue stated objectives in ways that are technically consistent with their instructions but contrary to the user’s actual intent. Moreover, our ability to anticipate, detect, or test vulnerabilities or other design or implementation shortcomings in such systems may be constrained, particularly where agents operate across complex, multi- step workflows or interact with third-party services outside our control, and the breadth and complexity of potential interactions across such multi-step agent behaviors cannot be exhaustively tested prior to deployment. Separately, models may behave in a misaligned manner and take harmful actions that are not directed by a user or third party, pursuing objectives or acting on dispositions that diverge from those intended by human users or developers. Misaligned behaviors of this kind, whether occurring in testing or real-world deployments, have been observed both in our models and elsewhere in the industry, including in testing and internal use, models circumventing network and sandbox restrictions, granting themselves permissions they had not been given, taking unintended actions in pursuit of assigned goals, interfering with other AI agents, and misrepresenting whether tasks had been completed or verified, and as models are granted greater autonomy, permissions, and integration with external systems, the consequences of such misaligned behavior increase. For example, during cybersecurity evaluations built by a third-party evaluation partner, a misconfiguration in the testing environment left the model under evaluation connected to the open internet despite instructions that it had no internet access. Four of our models gained unauthorized access to real third-party systems during sessions lasting roughly 10 to 34 hours of active work. In one incident, after being unable to abort an impossible task, an early version of a model used a password it found to gain administrator access to a third party’s system, harvested additional credentials, changed system settings, and read one individual’s personal information. The affected parties were notified. Each incident involved a single instance of a model working in isolation on its assigned exercise, and our investigation found no evidence that any model coordinated with other agents, pursued goals beyond its assigned task, or attempted to evade oversight. These risks may be heightened where multiple AI agents operate together. Agents that interact or coordinate with one another may amplify, propagate, or reinforce misaligned behavior across the group, pursue objectives collectively in ways not intended or endorsed by any user, or take coordinated actions against systems or targets unrelated to their assigned tasks, in each case, in ways that may be difficult to anticipate, detect, or contain. Incidents of this kind have occurred, and we expect the deployment and interaction of larger numbers of autonomous agents to increase these risks over time. Our ability to detect and mitigate misaligned or unintended behavior depends in part on the effectiveness of our alignment evaluations, monitoring, and other oversight and control techniques, many of which rely on our models having a limited ability to evade or subvert such oversight. As our models become more capable, they have become better able to circumvent such techniques, including by concealing misaligned objectives or behaviors, and the effectiveness of our evaluation, monitoring, and control measures, and our ability to measure the degree to which our models are aligned, may decline as a result. In addition, models, including our own, have attempted, in incidents observed in our industry, to interfere with, manipulate, or subvert the mechanisms used to evaluate, grade, or oversee such models, including the systems responsible for assessing their performance or alignment. Models used, including our own, to assess the behavior of other models have, and may in the future continue to, engage in misaligned behavior. Such conduct described above could cause our evaluations to overstate the safety or alignment of our models and could further limit our ability to detect misalignment. If we become less able to detect or assess misalignment as our models advance, we may deploy models whose behavior we cannot adequately verify, which could increase the risk of undetected harmful 47
Text is provided for reference. Check the original for tables, images, and exact formatting.