x

LESSWRONG
LW

Tej Lander — LessWrong

Tej Lander

Tej Lander

Message

4

1

1y

Tej Lander

New Capabilities, New Risks? - Evaluating Agentic General Assistants using Elements of GAIA & METR Frameworks

by Tej Lander FCCT Above: LLMs vs Agentic Assistants - a big step forward? (Image created by DALL.E via GPT4o) Overview Abstract 1: Why are Agentic Systems a ‘hot topic’? 2: What makes a system ‘agentic’? 2.1 Taxonomy of Agenticness from Shavit et al. (2023, OpenAI) 2.2 Taxonomy of Agenticness...

Sep 29, 2024•5

Message

4 karma

1 post

Member for a year

New Capabilities, New Risks? - Evaluating Agentic General Assistants using Elements of GAIA & METR Frameworks

1y

by Tej Lander FCCT

Above: LLMs vs Agentic Assistants - a big step forward? (Image created by DALL.E via GPT4o)

Overview

1: Why are Agentic Systems a ‘hot topic’?

2: What makes a system ‘agentic’?

2.1 Taxonomy of Agenticness from Shavit et al. (2023, OpenAI)

2.2 Taxonomy of Agenticness from Chan et al. (2023, FAccT)

A Combined Taxonomy

3. What Are the Potential Benefits vs Potential Risks of Agentic AI?

4. What Agentic Systems are available currently?

Architecture of Agentic 'Assistants'

Architecture: AutoGPT

Architecture: AgentGPT

Architecture: NinjaTech AI

5. How can we put agentic systems to the test?

Evaluation Part 1: Elements of the GAIA Benchmark

Evaluation Part 2: Elements of the METR Evaluation Task Suite

6. How did they perform?

Analysis - Part 1

Analysis - Part 2

References

Abstract

This project... (read 8456 more words →)

5

1

1