Jsevillamol

We at Epoch AI have recently released a new composite AI capability index called the Epoch Capabilities Index (ECI), based on nearly 40 underlying benchmarks. Some key features... * Saturation-proof: ECI "stitches" benchmarks together, to enable comparisons even as individual benchmarks become saturated. * Global comparisons: Models can be compared,...

Anticipating the future of AI necessarily requires anticipating results in performance. Ideally, we would like to understand the dynamics of improvements. This would help us preempt capabilities and understand how plausible sudden improvements are. However, this problem is notoriously difficult. Among other reasons, Machine Learning benchmarks use many different metrics...

Developments in Machine Learning have been happening extraordinarily fast, and as their impacts become increasingly visible, it becomes ever more important to develop a quantitative understanding of these changes. However, relevant data has thus far been scattered across multiple papers, has required expertise to gather accurately, or has been otherwise...

Epoch is a research group forecasting the development of transformative Artificial Intelligence. We try to understand how progress in AI happens and what economic impacts we might see from advanced AI. We want to enable better governance during this economic transition by gathering information about the timing of new developments,...

We summarize and compare several models and forecasts predicting when transformative AI will be developed. Highlights * The review includes quantitative models, including both outside and inside view, and judgment-based forecasts by (teams of) experts. * While we do not necessarily endorse their conclusions, the inside-view model the Epoch team...

I found this article useful context for Boaz Barak's more recent article about the probability of AI Doom. In this article, the author (through very rough BOTECs): 1. Estimates the number of parameters needed for AGI (tentative conclusion: x10-100 more than GPT-3) 2. Estimates the cost of training AGI of...

Overview * The linked document contains a collection of AI Forecasting research ideas, prepared by some Epoch employees in a personal capacity * We think that these are interesting and valuable projects that research interns or students could look into, though they may vary in difficulty (depending on your background/experience)...