Posts

Sorted by New

Wiki Contributions

Comments

Doesn't strike me as inevitable at all, just a result of OpenAI following similar methods for creating their tokenizer twice. (In both cases, leading to a few long strings being included as tokens even though they don't actually appear frequently in large corpuses.)

They presumably had already made the GPT-4 tokenizer long before SolidGoldMagikarp was discovered in the GPT-2/GPT-3 one.

Prior to OpenAI's 2023-02-14 patching of ChatGPT (which seemingly prevents it from directly encountering glitch tokens like ‘ petertodd’)

I've never seen it mentioned around here, but since that update, ChatGPT is using a different tokenizer that has glitch tokens of its own:

https://github.com/openai/tiktoken/blob/46287bfa493f8ccca4d927386d7ea9cc20487525/tiktoken/model.py#L16

https://wetdry.world/@MrCheeze/110130795421274483
 

I'd say this captures the spirit of Less Wrong perfectly.

[This comment is no longer endorsed by its author]Reply

500 years still sounds optimistic to me.

[This comment is no longer endorsed by its author]Reply

The key is in the phrase "much more complicated". The sort of algorithm that could become a mind would be an enormous leap forward in comparison to anything that has ever been done so far.

[This comment is no longer endorsed by its author]Reply

Man, people's estimations seem REALLY early. The idea of AI in fifty years seems almost absurd to me.

[This comment is no longer endorsed by its author]Reply

I still stand by my belief that 2 + 3 = 5 does not in fact exist, and yet it is still true that adding two things with three things will always result in five things.

[This comment is no longer endorsed by its author]Reply

"I think that if you took someone who was immortal, and asked them if they wanted to die for benefit X, they would say no."

This doesn't help against arguments that stable immortality is impossible or incredibly unlikely, of course, but I suppose those aren't the arguments you were countering at the time.

[This comment is no longer endorsed by its author]Reply

Yes, but the chance of magic powers from outside the matrix is low enough that what he says has an insignificant difference.

...or is an insignificant difference even possible?

[This comment is no longer endorsed by its author]Reply

Hm. I'd rather have seen more of the analysis on whether what they do with the money is useful, but this is something.

[This comment is no longer endorsed by its author]Reply
Load More