GPT-6 Needs ARC Evals

GPT-6 Needs ARC Evaluation

Vincent Chung, Carson Ellis, Jakub Kraus, Vidya Silai

The world in which GPT-6 is deployed will be far different from that of GPT-4. Current infrastructure may support alignment of large language models in 2023, but these approaches may break down against future waves of powerful, large, multimodal, widely available models. We propose ideal AI governance structures, identify structures which already exist, and posit that without structural change, the near future will not guarantee Safety in AI. Using GPT-4 and the pre-deployment testing conducted by ARC, we argue for the necessity of mandatory alignment testing for SOTA Language Models. This report was written for the GPT-6 Release Considerations case of the AI governance hackathon. It is a compilation of some ideas our team had on this topic, but it is certainly not a complete or polished report.