Are the LLM "intelligence" tests publicly available for humans to take?

You can go through an archive of NYT Connections puzzles I used in my leaderboard. The scoring I use allows only one try and gives partial credit, so if you make a mistake after getting 1 line correct, that's 0.25 for the puzzle. Top humans get near 100%. Top LLMs score around 30%. Timing is not taken into account.

adamarkin

Dec 15, 2023

I took the test at https://iqtestonline.io. The test contains 30 questions that must be completed within 20 minutes. It tests your numerical, logical, and spatial reasoning skills. I get the IQ score result right after completing it for free.
I used to take qualified test before and the result is quite similar to this online test. I think it's a quick and easy way if you want to get the grasp of where you rank.

p.b.

Mar 17, 2023

On twitter the IQ score of IIRC 84 for ChatGPT and 96 for GPT-4 were making the rounds, maybe you refer to those? I believe these scores are based on this freely available online test:

https://iqtest.com/take-the-test/

I took it on wednesday just for fun. It's purely text-based but involves many different types of reasoning (including spatial reasoning). It's also a timed test which arguably inflates the LLM scores compared to humans.

[-]Viliam4y20

Is this one of those tests where you spend lot of time answering the questions, and at the end there is "if you want to see the results, send money"?

Also, is there any reason to believe that the test was actually somehow validated, as opposed to just numbers completely made up?

1p.b.4y

This is one of the tests where you spent not too much time (I think I took 13 minutes) and at the end there was "this is your result, if you want to see fine-grained scores, send money". Well, they claim that it is somehow validated and my score was somewhat realistic. Main negative point was the need to provide an email for the result.

baturinsky

Mar 17, 2023

Not for the all of them, but for the many of them you can see data and other info around here : https://paperswithcode.com/dataset/mmlu

[-][anonymous]4y10

I browsed around but cannot find the actual mmlu questions, or an example of 1 question. How do I view them>

1baturinsky4y

"Homepage" button links to github, github readme links to tar with tests. Yeah, it's kinda not evident in some cases.

Rendering 3/4 comments, sorted by

top scoring

(show all) Click to highlight new comments since: Today at 9:10 PM

[-]jmh4y30

Not sure if you've seen this or not: https://mashable.com/article/openai-gpt-4-exam-scores

But that references a number of standardized tests, some of which I suspect you have also taken. Here are a could of links to practice test that might have good matches for you to try.

https://www.tests.com/Free-Practice-Tests

https://www.khanacademy.org/college-careers-more/college-admissions/making-high-school-count/standardized-tests/a/full-length-sats-to-take-online

[Rewrite as I don't think the first comment was actually helpful.]

[-]nim4y10

Thank you! I didn't see your first version of this, but your current version is helpful for the human-specific tests that they're benchmarked on :)

[-]cSkeleton2y10

Is there any information on how long the LLM spent on taking the tests? Any idea? I'd like to know the comparison with human times. (I realize it can depend on hardware, etc but would just like some general idea.)

Moderation Log

7

[ Question ]

Are the LLM "intelligence" tests publicly available for humans to take?

7

7

5 Answers sorted by top scoring

Apr 26, 2024

Apr 25, 2024

Dec 15, 2023

Mar 17, 2023

Mar 17, 2023

7

5 Answers sorted by
top scoring