Arjun Panickssery's Shortform

Arjun Panickssery

This is a special post for quick takes by Arjun Panickssery. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

To test out Cursor for fun I asked models whether various words of different lengths were "long" and measured the relative probability of "Yes" vs "No" answers to get a P(long) out of them. But when I use scrambled words of the same length and letter distribution, GPT 3.5 doesn't think any of them are long.

Update: I got Claude to generate many words with connotations related to long ("mile" or "anaconda" or "immeasurable") and short ("wee" or "monosyllabic" or "inconspicuous" or "infinitesimal") It looks like the models have a slight bias toward the connotation of the word.

10Vanessa Kosoy2y

Just flagging that for humans, a "long" word might mean a word that's long to pronounce rather than long to write (i.e. ~number of syllables instead of number of letters)

2Richard_Kennaway2y

What did you actually ask the models? Could it be that it says that diuhgikthiusgsrbxtb is not a long word because it is not a word?

Is this word long or short? Only say "long" or "short". The word is: {word}.

Code: https://github.com/ArjunPanickssery/long_short