Research

Focus Areas

Our broad purpose is to address emergent risks from advanced AI systems. We welcome a variety of interests in this area. Here are a few prominent areas of interest:

• Specification. Focuses on precisely defining what we want an AI system to do. This involves ensuring that the objectives and behavior of AI models are in line with human intentions and values, eliminating any ambiguity that could lead to unintended and potentially harmful outcomes.

• Robustness. Pertains to the resilience and stability of AI systems in the face of adversarial attacks, novel inputs, or changing environments. The goal is to ensure that AI models can consistently operate safely and effectively, even under unexpected conditions or when exposed to malicious intents.

• Interpretability. Seeks to make the decision-making processes of AI systems transparent and understandable to humans. By uncovering the black box nature of AI, we can ensure that AI decisions can be explained, validated, and trusted, fostering more responsible and accountable AI deployments.

• Governance. Delves into the frameworks, policies, and regulations guiding AI development and deployment. This area emphasizes creating structures that ensure AI systems are developed ethically, responsibly, and in alignment with societal values and legal norms.

Current Projects

Supervised Program for Alignment Research

Organized by groups at UC Berkeley, Georgia Tech, and Stanford, the Supervised Program on Alignment Research (SPAR) is an intercollegiate project-based research program for students interested in AI safety running this fall. SPAR matches students around the world with advisors to do guided projects in AI safety.

Past Projects

We have a few papers under review that we will share soon!