694

LESSWRONG
LW

693

Prodding ChatGPT to solve a basic algebra problem — LessWrong

14

Prodding ChatGPT to solve a basic algebra problem

12th Dec 2022

1 min read

14

This is a linkpost for https://twitter.com/shminux/status/1602140709204955142

14

Prodding ChatGPT to solve a basic algebra problem

4the gears to ascension

New Comment

6 comments, sorted by

Click to highlight new comments since: Today at 3:13 AM

Got it in one thinking step by step.

yeah, looks like "steb by step" is a magic incantation:

https://twitter.com/ESYudkowsky/status/1602177149401989120

... which makes it even more human. I used to almost yell at the students I used to tutor: DON'T SKIP STEPS!

Incantation rankings, for GPT-3 on math word problems

Didn't seem to work for me. It still seems to get confused trying to match similar words together even when they shouldn't be. Again quite, dumb/young human.

It tries to continue its dialogue in ways that fit with its previous replies, so prompts like "think step by step" might fail to get it back on track mid-conversation even when they would've worked in the initial prompt.

Also, there is some randomness.

[-]the gears to ascension3y42

[humor] teenagers, amiright?

More from Shmi

Curated and popular this week

TL;DR: It behaves amazingly like a human would!

It understands the word problem, but struggles to put it in equations, takes multiple tries, screws up again, eventually muddles through to get the right answer. Given a similar problem it does slightly better, but still messes up. On the third problem it messes up again, then realizes that something went wrong... and promptly blames the problem setup, unable to question its own reasoning.