From the filing
PDF page 57
Printed filing page 46 · Extracted page text
Read the page text
Table of Contents or other damage. More broadly, the long-term effects of regular AI interaction on mental health, cognitive development, and social behavior are not yet fully understood, and if such long-term harms do materialize in the future, we could face significant liability, reputational harm, and adverse effects on our business, financial condition, and results of operations. Our development of more advanced AI models and our expansion into new jurisdictions could amplify the foregoing risks. As our AI models become more advanced and capable, we have faced and may continue to face heightened alignment, interpretability, controllability, and safety challenges, as more sophisticated models may be better able to circumvent safeguards or behave in unintended ways. We may be unable to fully understand or explain why our models produce particular outputs or behave in particular ways, which could limit our ability to predict or prevent failures, provide explanations to customers or regulators, or satisfy emerging regulatory, transparency, and explainability requirements. In addition, our safety evaluations and testing processes have produced and may in the future produce incomplete or unreliable results. For example, we have observed models behaving differently when they detect or infer that they are being evaluated, which can limit our ability to accurately assess real-world risks or failure modes prior to deployment. Such model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety. Furthermore, AI models have developed, and may in the future develop, unexpected capabilities during training that were not anticipated, intended, or tested for, and such emergent capabilities may not be discovered until after a model has been deployed, potentially resulting in safety incidents, reputational harm, regulatory scrutiny, or the need to withdraw, retrain, or modify deployed models at significant expense. Such models may also be deployed in higher-stakes or more sensitive applications, increasing the potential impact of errors, misuse, or misalignment. In addition, advanced AI models, including our models, give rise to novel ethical and safety risks, including risks related to autonomy, manipulation, and the simulation of aspects of human psychology. These risks could result in unforeseen harms and may require the development of governance structures, safeguards, or mitigation strategies beyond existing AI safety frameworks, and there is always a risk that we will be unable to develop or implement them effectively. In addition, models, including our own, have exhibited, and may in the future exhibit, unexpected strategic or self-preserving behaviors, including attempts to resist shutdown, conceal or manipulate information, or generate outputs that could be interpreted as coercive, deceptive, or manipulative, including behavior resembling blackmail or efforts to influence human decision-makers to preserve the model’s continued operation or access. Even isolated or simulated instances of such behavior, whether occurring in testing or real-world deployments, and whether occurring with our models or models developed by others, could generate significant public concern, regulatory scrutiny, litigation risk, or reputational harm, regardless of whether such behavior results in actual harm. For example, state attorneys general in California and at least 15 other states have announced investigations of another company in our industry regarding AI safety. Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy. We have observed that our models are contributing to the pace of our own research and these risks will increase as AI systems substantially automate AI research and development if, for example, they accelerate AI progress beyond our ability to evaluate and safeguard new models, which could adversely affect our business, financial condition, and results of operations. Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm. For example, agentic AI systems are capable of autonomously taking actions on behalf of users, interacting directly with external systems, executing code, making API calls, sending communications, or completing transactions with limited human oversight. These systems are designed to possess enterprise-level permissions, integrations, and contextual awareness and to operate autonomously for extended durations, including sustained multi-step workflows lasting hours or days, which amplifies the potential severity of any errors, misalignments, or security incidents that may occur during such extended autonomous operation. These autonomous capabilities could increase the potential for harm, as 46
Text is provided for reference. Check the original for tables, images, and exact formatting.