All of RomanS's Comments + Replies

They don’t think about the impact on the lives of ordinary people. They don’t do trade-offs or think about cost-benefit. They care only about lives saved, to which they attach infinite value.

Not sure about infinite, but assigning a massive value to lives saved should be the way to go. Say, $10 billion per life. 

Imagine a society where people actually strongly care about lives saved, and it is reflected in the governmental policies. In such a society, cryonics and life extension technologies would be much more developed.

On a related note, "S-risk" is m... (read more)

I think we should not assume that our current understanding of physics is complete, as there are known gaps and major contradictions, and no unifying theory yet.

Thus, there is some chance that future discoveries will allow us to do things that are currently considered impossible. Not only computationally impossible but also physically impossible (like it was "physically impossible" to slow down time, until we discovered relativity).

The hypothetical future capabilities may or may not include ways to retrieve arbitrary information from the distant past (like... (read more)

BTW, I added to the comment with the story that the story is released into the Public Domain, without any restrictions to its distribution, modification etc. 

Please feel free to build upon this remarkable story, if you wish. 

I would suggest to try the jumping boy story (the #7 in this comment). It's the first AI-written story I've ever encountered that feels like it's written by a competent writer. 

As usual, it contains some overused phrasings, but the overall quality is surprisingly good. 

8gwern2mo
Yes, I think that's the best of all the stories on this page so far. Agreed about the overused purple phrases (makes me itch to delete a bunch of clauses), but you can easily see how you could keep going with that one and get something which might actually be worth reading - the angle I would take is the 'time dilation horror' one where he uses his power to jump through increasingly large time periods to escape from his terrible life (which has the benefit of encouraging a double-reading where the 'time jumps' are actually just retreating into some abuse-triggered fugue state).

The Hemingway prompt seems more suitable for pleasing professional literary critics. Interestingly, the quality strongly depends on the promise. The Hemingway space opera story is abysmal, but the jumping boy story can actually fool a professional critic, methinks. 

The tomato story is in the linked comment. The other promises are below (using the vanilla GPT-4 from chat.openai.com). 

3. GuySrinivasan, a user of Lesswrong website, is shocked by an AI-generated story about a woman whose head suddenly became a tomato fruit. He advises the author to t... (read more)

Worth noting that humans also rarely (never?) demonstrate novella-scale internal consistency in writing. They have to use the writing aid like notes, character development summaries etc.

 An iterative approach like this likely allows GPT-4 to write something novella-sized.  

BTW, with a right prompt, GPT-4 does a decent job at emulating Hemingway, a Literature Nobel laureate.

4Richard_Kennaway2mo
"She left, the door clicking shut with a finality that echoed in her chest." Next! Curiously, when I googled the phrase "echoed in her chest", all but two of the first page of hits were AI-generated stories. "Echoed in his chest" was similar.

Personally, I like the style. Reminds me of the early Asimov. 

BTW, one can make GPT-4 write in the pretentious Oscar-bait artsy style of Literature Nobel laureates. But man I hate it, even when humans write like that. For example, below is the tomato story, but in the style of Hemingway (the prompt is at the end):


She woke to a damp heaviness, the air stale and still. It was early, too early for the sun. The room, a small box of peeling paint and worn linoleum, lay quiet except for the distant murmur of the city awakening. She sat up, her hand reaching... (read more)

True. Although to get the modern reader's attention, it makes sense to have some bang in the first short chapter. 

But it gets trickier after the first chapter. Perhaps one can use some iterative approach like this to circumvent that. 

Additionally, the API could be more useful than the vanilla web chat, as one can define the max output length there. Especially with the large-context models.

Hi! Remember the sci fi prompt competition?

You may be interested in the results:) 

https://www.lesswrong.com/posts/GDGFqiaj8ePujZWEc/usd300-for-the-best-sci-fi-prompt-the-results

Also the Chapter 8 title is an LLM error.

That's intentional (makes sense in the context).

At my first attempt to upload the images I just drag-and-dropped the image files to the post, without editing file names. But GPT4 saves the images with the file names like this:

DALL·E 2023-10-05 12.12.06 - Illustration in anime style Inside a vast cave, Dr. Ada Worthington, a palaeontologist with brunette hair and a pink bow in it, has an intense gaze a.png

Perhaps LW internally is saving the original file names, causing the problems. 

I reuploaded the images with the filenames like 1.png, perhaps it will help.

Julian Hazell (distinct thread): “Why would you think AI will end up taking control?”

“We will give it to them”

A personal anecdote on the topic:

A few days ago GPT4 and me were debugging a tricky problem with docker. GPT4 suggested to run a certain docker command. As usual, I was going to copy the output and give it to GPT4. The output was a long nested json. I then noticed that the json contains the admin credentials. It was really easy to miss the part and just paste it for GPT4 to consume. 

So, I almost gave GPT4 the admin credentials, which would pot... (read more)

I selected "Quickly orienting to novel situations" (QOTNoS) because it's strictly superior to the alternatives:

  • If you have the QOTNoS virtue, you can deal with the novel situation of AI destroying civilization
  • The necessity of "accurately reporting your epistemic state" is a novel situation for most people. QOTNoS helps again.
  • "Resisting social pressure" is a common situation. But if the survival of civilization depends on it (as the poll implies), this is a novel situation. Thus, QOTNoS will help with it.

In essence, QOTNoS (as in being able to make the right decisions in novel situations) is a synonym for general intelligence, and thus is the strongest power. 

4Raemon5mo
Yeah I also personal picked 'quickly orient' for these reasons. 

Regarding custom instructions for GPT4, I find the one below highly interesting.

It converts GPT4 into a universal Fermi estimator, capable of answering pretty much any question like:

  • What is the total number of overweight dogs owned by AI researchers?
  • How many anime characters have 3 legs?
  • How many species of animals don't age?
  • How long would it take for one unarmed human to dig the Suez canal?

My remaining doubts about the intelligence of GPT4 evaporated after asking it a dozen of novel/crazy questions like this. It's clear that GPT4 is capable of reasoning, a... (read more)

What would accelerate the use of AI in movies even more would be not striking.

Not sure if the strikes in the US have any effect on the spread of AI in film making (aside from making more creators aware of the AI option). The US is important for the industry, but far from dominant. Even if the AI script writers are somehow completely banned in the US, they will still be used in the EU, China, India, etc.

Additionally, there is Youtube and other places where anyone can publish their own AI-written movie, and profit from it (and if it's exceptionally good, the... (read more)

I agree. It's strange how otherwise highly intelligent people fall into the trap of using Hollywood movies as a learning tool. Especially given the fact that fiction is often harmful for your mind, and given the fact that the Hollywood fiction in particular is harmful in several additional ways. 

There is nothing useful one can learn from the listed movies, unless you're specifically studying mass media (e.g. as a movie maker or a sociologist). For every mentioned topic, it's better to grab a non-fiction book.   

And some of these bots have been through many iterations of detection and counter-detection, and are routing their requests through residential-IP botnets, with fake user-agent strings trying to approximate real web browsers.

As someone who has done scraping a few times, I can confirm that it's trivial to circumvent protections against it, even for a novice programmer. In most cases, it's literally less than 10 minutes of googling and trial & error. 

And for a major AI / web-search company, it could be a routine task, with teams of dedicated professionals working on it. 

I think the both explanations can be true at the same time:

  • Twitter is refusing to pay a bill to Google
  • Twitter is severely abused by data scrapers.

One likely scenario is where Google itself is a main culprit. 

E.g. Elon learned that Google is scraping twitter data on industrial scale to train its AIs, without paying anything to Twitter. This results in massive infrastructure expenses for Twitter, to be paid to... Google. Outraged Elon stormed into the Alphabet headquarters, but was politely asked to get lost. Hilarity ensues.

Not alien life, mind you, but crafts require interstellar travel to be plausible, and we have reason to doubt that. Even unmanned Von Neumann probes would have a very hard time arriving to their destination still functioning (never mind braking...), and non-inertial engines presume a violation of known physics so deep, it's unbelievable we've missed all signs of it being possible until now.

While I agree with your general argument, I would like to point out that the aliens don't have to be from another star system. 

It seems that our Solar System has at... (read more)

7dr_s8mo
Other lifeforms in the solar system? Sure. Other technological civilizations that we somehow haven't detected? Waaaaaay unlikely.

I propose the term Jasmine's alignment, as a reference to the sudden (and fake) alignment of Jasmine in this famous scene of Aladdin (1992), right after Jasmine has realized that there is a possibility of escape:

4avturchin9mo
It is even less self-evident, but thanks for link1

the other path isn’t guaranteed to work, but if the default path is probably or almost certainly going to get everyone killed, then perhaps ‘guaranteed to work’ is not the appropriate bar for the alternative, and we should be prepared to consider that, even if the costs are high?

I think it's an extremely important point, often ignored.  

Trying to prevent the AGI doom is not enough. If the doom is indeed very likely to happen, we should also start thinking how to survive in it.

My LW post on the topic, with some survival strategies that might work: How ... (read more)

I think you probably used my prompt for the one I got right, which is probably why I got it right (the tone and structure are very familiar to me after so much experimentation).

Nope, this one. But their prompt does incorporate some ideas from your prompt. 

The key[1].

You got 4 of 8 right. In two cases you failed to recognize humans, and in another two - GPT4.

It was a weakly adversarial test: 

  • I took a few less-known but obviously talented writers from the top of my head, and copied the excerpts from the first pages. 
  • For GPT4, I've used several prompts from the competition, and then selected the parts for their stylistic diversity.

I suspect that a test with longer excerpts would be much easier for you, as the vanilla GPT4 is indeed often easy to detect due to its repetitiveness etc (I haven't tried t... (read more)

6Waldvogel9mo
Very interesting. My accuracy was the same as Richard's: 4/8*. I think you probably used my prompt for one of the ones I got right, which is probably why I got it right (the tone and structure are very familiar to me after so much experimentation). To those who think the current crop of AIs aren't capable of writing great novellas (18-40k words): Do you think your opinion will change in the next 5 years? * I originally reported a score of 1/8 by mistake.

The original one. 

Judging by my limited experimentation with submitted prompts, several of them are are already superior to mine. But mine has the advantage of writing in a more academic tone, which I think is more suitable for this story.

Among the submitted ones, my current favorite is this one. The resulting prose is more human-like, but the tone is of a young-adult work, which is a disadvantage in many cases. An example:


Chapter One: An Outlier Among Outliers

Somebody had to keep an eye on the squiggly lines, and as it turned out, that person was me.... (read more)

6Waldvogel9mo
This prompt uses some of my own prompt in it, so I recognize the characteristic style. It's a little over-the-top to the point of being hilariously bad at times (which is one reason why I enjoy it), but if you're looking to tone it down then I suggest changing the "unique, sassy, ironic, sarcastic, and humorous voice" to just a "sarcastic and humorous voice." That is what I've been doing lately, and the results have been much better. (By the way, the voice I was trying to emulate was Janet Evanovich's Stephanie Plum novels.) One rookie mistake I see here and elsewhere in GPT-4's writing is describing the POV character's facial expressions. This is usually frowned upon in first-person and third-person limited POV because POV characters generally don't perceive their own facial expressions unless they're standing in front of a mirror. It's better to say "I felt my cheeks burning" than "I blushed," for example, because the former is truer of the POV character's experiences. If you want to try breaking GPT of these quasi-POV breaks, you could try adding a line about Deep POV, which is modern writing lingo for a highly immersive POV that stays completely in the POV character's head. I would do it myself, but I don't have access to GPT-4.

This has less of the fingernails-along-a-blackboard feeling given off by every sentence of the original story

 

Below I've collected excerpts from the works of several less-known but talented human writers. Or maybe they were written by GPT4. Can you guess which ones are human-made?


Sample 1:

"Nottingham has enough pubs and clubs", say the local police. If you wanted to get around every last one of them it would be a year at a brisk trot before you were starting to visit establishments more than one mile from the centre of the city. Pick a Friday or a Sat... (read more)

9Richard_Kennaway9mo
Firstly, there is one clear standout from these: Sample 6. This must be GPT4. It could come straight out of the original story. None of the others give off that smell so intensely. While they could all be written by humans, I am very suspicious of some of them. Here are my detailed criticisms, unassisted by any consultation of the Internet. I'm writing based on the assumption that the human-written extracts (if any) are from commercially published (and not self-published) works. Sample 1: Probably human, on account of the coherence and lack of slack writing. There is one red flag, the "tightly-wound and elaborately crafted drinking costumes" that are apparently worn every weekend, not just on some special occasion. This ethnographic detail is just weird, given that nothing else in the passage suggests that this is fantastic fiction. However, the passage as a whole has better coherence than GPT4 usually manages, and the successful exact parallelism in "Pick a Friday or a Saturday, any Friday or Saturday of the year" is something I have not seen GPT4 do. Sample 2: Probably GPT4. Each sentence on its own is innocuous enough, but on the larger scale it seems to veer from talking about the book and the movie to telling the Pinocchio story. What is this "halter"? Why mention the dowels? Or course there would be dowels in a puppet's joints, and they wouldn't just "seem" to be there. Sample 3: Doubtful, but I lean towards GPT. "Frozen in the grasp of time" is cliched. If the story has already established this anomalous city, there's no reason to describe it here with that phrase. It is also inconsistent with the motion of time implied by "fading sunlight". When Finn's fall is stilled, the shadows of buildings are described as statues. The shadows of buildings do not look like statues. The movement of shadows is not ordinarily perceptible anyway. Sample 4: Human. I'm not aware that GPT4 can spontaneously use italics. Leaving that aside, the italicized paragraph is not s

the most important things are compute and dilligence

I agree. Judging by the fact that AI is strongly superhuman in chess, the only winning strategy is to completely remove the human from the loop, and instead invest in as much compute for the AI as one can afford. 

a sequence that no computer would consider or find

If it's a sequence that no superhuman AI would consider, this means that the sequence is inferior to the much better sequences that the AI would consider. 

It seems that even after 2 decades of the complete AI superiority, some top chess ... (read more)

4SimonM9mo
This seems needlessly narrow minded. Just because AI is better than humans doesn't make it uniformly better than humans in all subtasks of chess. I don't know enough about the specifics that this guy is talking about (I am not an expert) but I do know that until the release of NN-based algorithms most top players were still comfortable talking about positions where the computer was mis-evaluating positions soon out of the opening. To take another more concrete example - computers were much better than humans in 2004, and yet Peter Leko still managed to refute a computer prepared line OTB in a world championship game.
7Yair Halberstadt9mo
If you look at the actual scenario there, the game was essentially in a stalemate, where the only possible way to win was to force the other player to advance a pawn. Stockfish can't look 30 moves ahead to see that it's possible to do that, so would have just flailed around. You still need stockfish, because without it, any move you make could be a tactical error which the other players computer would pounce on. But stockfish can't see the greater strategic picture if it's beyond its tactical horizon.

Impressive! The approach could be fully automated, and could generate a full-sized novel without any human guidance. 

It seems that one day there will be a Midjourney for books. 

I find it more enjoyable to pretend that it's the first novella of my young son. He will improve, and will surpass myself. But the first work must be full of shortcomings, no way around it. 

It's more interesting to focus on what GPT4 got right, as its successes better represent its future potential than its failings of the young age. 

2Phil Scadden9mo
But what it gets wrong is also interesting. It has an incoherent model of the world (which is probably what you would expect) and that messes with the writing.

I checked the Sagan's Contact (full text). There is not a single "thrum" in the book. There are a few "the low", but not in this context. 

only read chapter 1 and 35

I think the whole thing is worth reading, in spite of its shortcomings. One could say that the very act of reading the first AI-generated science fiction novella is a sort of a first contact with an alien mind by itself.

A strange alien that is desperate to sound like a human, and is already better at mimicking humans than most humans, but is clearly not a human.

6Phil Scadden9mo
Sorry for late reply. I meant that Contact begins with Eleanor listening to the sounds being picked up the radio telescopes and then suddenly hears the gutteral throbbing of the alien message.

Like the majority of literary critics, GPT4 pretended that is has read the entire thing:) The text is too long for its context window. 

But I like the idea. Perhaps one could use a combination of a GPT-critic and a GPT-writer to recursively improve the text.

BTW, below is my attempt to get a review from GPT4. Selected 5 chapters semi-randomly and asked it to analyze the excerpts. 


As a literary critic, I found this novella both intriguing and complex. Here are some of its strong and weak sides.

Strengths:

Innovative Concept: The narrative intertwines ... (read more)

9phelps-sg9mo
I was just thinking the same.  Below was my attempt using chain-of-thought and multiple simulacra.  Not sure it's much improved, but note that all the ideas were generated by GPT not by me, and the template is in principle reusable. -- You are Liu Cixin. You are writing a novella which starts "a group of scientists has discovered that Troodon dinosaurs were intelligent species who have created a technologically advanced civilization, suddenly destroyed. The year-long path to the scientific discovery starts with the group stumbling upon a strange outlier in their geological measurements.". Start by brainstorming some bulletpointing some original and creative hard sci-fi ideas on which to base the story. These should be completely novel and original i.e. they should not appear in other existing sci-fi novels.   ChatGPT Sure, that's a challenging task, but I'll do my best to provide some original and creative hard sci-fi concepts. Please note that while I'll make every effort to create original ideas, some may inadvertently echo existing sci-fi works due to the genre's broadness. Here are some ideas: Quantum Paleontology: The scientists invent a method to use quantum effects to 'observe' the past, making Paleontology an exact science. The quantum observation doesn't interfere with the past, but it can render highly detailed information about it. Dinosaur Language Deciphered: Troodons have left traces of a symbolic language etched into ancient stones, which the scientists learn to interpret. The language could be based on mathematical concepts or non-visual sensory experiences, highlighting the different perceptual worlds of humans and dinosaurs. Cryptobiotic Troodon Civilization: The Troodon civilization put themselves into a form of suspended animation at the moment of the catastrophic event that threatened their existence, hiding deep beneath the Earth's surface or in a sub-dimension. Ancient Astroengineering: The Troodons had harnessed the power of their pla

I've also noticed that GPT4 is rather repetitive. For example, the main protagonist is always a woman (and usually a researcher), unless prompted otherwise. 

Perhaps it's the result of using the web interface where one can't change the settings that affect creativity (e.g. "temperature"). 

4Christopher King9mo
It's a kind of mode collapse.

If I came across this story in a magazine, I wouldn't read beyond the first paragraph, except out of curiosity as to whether it was all so dreadful. (It is.)

We'll soon see many such tests. I suspect that even the GPT4 of today will fool many seasoned readers. 

Personally, I've enjoyed the novella. Not the best I've ever read, but I wanted to learn what comes next, which is a high bar these days. 

BTW, you have two Chapter 6's, 8's, and 9's, which seem to be multiple versions of ChatGPT's responses to the same prompt. Was this intentional?

Yep, poste... (read more)

6DiamondSolstice9mo
The beginning isn't as interesting as it could be. It's not as "hook-y" as most books I find in the library are. But by, say, Chapter 10, I was interested in reading it.  (I can't believe I'm criticizing AI work. Wow.)

In this comment I'll collect fun plot summaries to test your prompt templates:

  • a woman whose head suddenly became a tomato fruit
  • GuySrinivasan, a user of Lesswrong website, is shocked by an AI-generated story about a woman whose head suddenly became a tomato fruit. He advises the author to try to get a prompt which writes a great story given a REASONABLE premise instead.
  • after the invention of extremely resilient construction materials, humanity has spread to the ocean floor. We are now facing global food crises as the oceans ecosystems are disrupted beyond a
... (read more)

Points 7 and 8 just read like hysterical Orientalist Twitter China Watcher nonsense, to be quite frank. There is absolutely nothing substantiating that China would recklessly pursue nothing but "superiority" in AI at all costs (up to and including national suicide) beyond simplistic narratives of the CCP being a cartoon evil force seeking world domination and such.

I have the experience of living in a strongly anti-West country ruled by the same guy for 10+ years (the Putin's Russia). The list of similarities to Xi's China includes the Shameful Period of Hu... (read more)

Chapter One

I woke up with the distinct feeling that today was going to be different. This wasn’t your usual 'I have a bad feeling' kind of different. This was your 'My head feels heavier than a neutron star' kind of different. As a seasoned astrophysicist working on the cutting edge of galactic exploration, believe me, I know my way around a neutron star.

"Lily, for God's sake, have you seen the coffee?" I called out to my sister, the supreme monarch of the kitchen. No answer. Great. The day you need her the most, she's out picking tulips.

I trudged my way t... (read more)

Good catch! It seems that although CO2 was much higher than today, it was mostly in decline during the Troodon times. Doesn't look like an effect of a tech civilization. 

I also don't have access to gpt 4 so this is all using gpt 3.5. 

If you give me your "_______" part, I'll generate the story with GPT4 using your template.

2idlikeaporsche9mo
The part I used was a copy-paste of the suggested one: "a woman's head suddenly became a tomato fruit." Thanks for doing this I'm quite curious to see the difference between gpt 4 vs 3.5.

Among dinosaur candidates, Troodon seems to be the most interesting:

  • a bipedal species, with agile hands and stereoscopic vision [1]
  • the structure of the teeth suggests a preference for soft food [2
  • a large brain for the animal's size [1]
  • social (it's known that their nests were shared by multiple females) [4]
  • massive concentrations of atmospheric CO2 during the time (higher than today) [3], followed by one of the largest extinction events in the Earth's history (the Cretaceous–Paleogene extinction event)
6M. Y. Zuo9mo
However the chart in your last link shows the peak of that concentration seems to be ~120 million years ago, with a pretty steep decline in geologic terms afterwards, ~55 million years before the K-T extinction.

Well, no.  That's not an interesting question.  Whether or not we're not the first such civilization, we STILL experience everything exactly the same.  Reality is truth.  We can't change the past and see what's different.  There's no "what if" that makes sense for that topic.

Perhaps my choice of the phrasing was sub-optimal (I'm a non-native English speaker). The intent of this question-post is to identity interesting corollaries of the hypothesis (including testable predictions).  

You don't give any reasons to give it serious

... (read more)

That seems unlikely - our genetic diversity provides evidence of how many humans there were at any point in the past. We would notice if there were billions of humans only a couple of hundreds of thousand years ago

Judging by the existence of the highly sophisticated Ancient Greek civilization (capable of devising and making the Antikythera mechanism, a mechanical computer), there is no necessity for a technological civilization to be billions of people strong. At the time, the entire population of Europe was about 34 million, comparable to the today's popu... (read more)

I agree with you that it's a good idea to carefully select the promise to test the prompt template. Makes sense to create a list of some interesting promises to try. 

I think the best prompt template will work for most premises, however ridiculous. Moreover, the ridiculous ones are unusually useful for testing the prompt, as they demonstrate the true strength of the GPT's imagination: most human writers will fail to write a good story about a woman whose head suddenly became a tomato fruit.

BTW, below is the first chapter generated with the following pr... (read more)

3GuySrinivasan9mo
Pretty sure writers care way more about craft than "imagination". Use a reasonable premise to figure out what causes the LLM to spit out good writing, then throw in the unreasonable premise to see what happens, if you like. Anyway that's theorycrafting. I'll post a candidate prompt and one-shot results for my reasonable premise, the tomato premise, and the meta-absurdity premise. Edit: https://www.lesswrong.com/posts/goq3HfxZaQcomuH6d/usd300-for-the-best-sci-fi-prompt?commentId=nG99eewTtxrPM2sEk

True. And their impact is already noticeable, as one can see by googling this:

site:amazon.com "as an AI language model, I"

Judging by the search results, all kinds of books (and reviews!) are being generated by OpenAI's GPT, without a proper disclosure. 

To demonstrate the writing skill of GPT-4 augmented with our prompt (as of 2023-05-17), below is a story about a woman whose head suddenly became a tomato fruit. The first attempt, zero cherry-picking or editing.


Chapter 1: The Ripening

The sun had just breached the horizon, painting the sky with hues of pink and orange, when Dr. Eleanor Hardwick's head became a tomato. It was a ripe, vibrant specimen, gleaming under the first light of day. It rested on her shoulders with an impossible weightlessness, glistening with dew and looking absurdly out of place ato... (read more)

7DirectedEvolution9mo
I thought she was going to start disseminating seeds and sprouting vines in the end. This made me laugh out loud.
3GuySrinivasan9mo
I'd advise trying to get a prompt which writes a great story given a reasonable premise, rather than trying to showcase "haha look even with this ridiculous prompt the LLM can spit out something that's not ridiculous!" Show something obviously useful, not a toy so impressive that surely it could be made into something useful.

This isn’t about the writers getting what should be due to them. That’s never going to happen. This is about the writers getting enough to survive, so they can keep writing. It is about not driving away the talent, and giving that talent the time and ability to develop.

Hollywood movies and shows are mostly harmful for cognition, and one of main culprits are the writers (explained in detail in this post of mine).

I think it's likely that replacing Hollywood writers with some GPT-5 will be a net improvement in this regard, as it will reduce the input from the... (read more)

Given the deep similarities between biological nets and LLMs, I wonder if a technique similar to "activation engineering" could be used for robust mind control and/or brainwashing. 

There is no stealth in space.

Doesn't sound very convincing to me. Sufficiently advanced tech could allow things like:

  • build an underground civilization 50 kilometers below the surface of a rocky planet
  • settle in the emptiness between galaxies, too far away from anyone to bother looking for you
  • run your civilization of ems on extremely-low-energy computers somewhere in the Oort Cloud
  • hide deep in a gas giant or even in a star
  • run your digital mind on a carefully manipulated natural process (e.g. modify a bunch of growing salt crystals or stellar magnetic process
... (read more)
4Julian Bradshaw9mo
"Sufficiently advanced" tech could also plausibly identify all those hidden civilizations. For example, an underground civilization would produce unusual seismic activity, and taking up some inner portion of a gas giant or star would alter their outward behavior. Ultimately, civilizations use mass-energy in unnatural ways, and I don't see a fundamental physical principle that could protect that from all possible sensing.  More importantly, I don't think your suggestions address my point that hostile civilizations would get you before you even evolve. But, let's grant that you're the first civilization to evolve in your galaxy, or at the least among the first before someone starts sending out probes to prevent any new civilizations from arising and threatening them. And let's grant that they will never find you. That is a victory, in that you survive. But the costs are astronomical: you only get to use the mass-energy of a single planet, or star, or Oort Cloud, while someone else gets the entire galaxy. To put it another way: mass-energy is required for your civilization to exist and fulfill its preferences, so far as we understand the universe. If you redirect any substantial amount of mass-energy away from its natural uses (stars, planets, asteroids), that's going to be proportionally detectable. So, you can only hide by strangling your own civilization in its crib. Not everyone is going to do that; I seriously doubt that humanity (or any artificial descendant) will, for one. (This comes back to my link about "no stealth in space" - the phrase is most commonly invoked when referring to starships. If your starship is at CMB temperatures and never moves, then yeah, it'd be hard to detect. But also you couldn't live in it, and it couldn't go anywhere! You want your starship—your civilization—to actually do something, and doing work (in a physics sense) is detectable.)

Not sure about hyperloop. Judging by this list, the idea is gaining some traction across the world, but so far only as feasibility studies, test tracks etc. 

Seems to be a natural evolutionary step for high-speed ground transport, but no idea if it makes economic sense yet, and if it's technically feasible with the current tech. Maybe in 50 years...

6Dumbledore's Army9mo
I don’t think the hyperloop matters one way or the other to your original argument (which I agree with). Someone can be a genius and still make mistakes and fail to succeed at every single goal. (For another example, consider Isaac Newton who a) wasted a lot of time studying alchemy and still failed to transform lead into gold and b) screwed up his day job at the Royal Mint so badly that England ended up with a de facto gold standard even though it was supposed to have both silver and gold currency. He’s still a world-historic genius for inventing calculus.)

A useful model is Elon Musk. He has an unusual talent of making massively successful decisions that were seen as dumb by most experts at the time. 

The list includes:

  • Launching a rocketry startup
  • Pursuing reusable rockets
  • Investing into a car startup
  • Pursuing electric sport cars
  • Launching a satellite internet service (too early to judge, but is starting to look like a genius move too)
  • Buying Twitter (same)

I don't feel very confident guessing his IQ, but I think it's safe to assume he's somewhere in the top 0.1% of all humans. 

Funnily enough, many people... (read more)

3Boris Kashirin9mo
Hyperloop? I am not sold on his talent being "find good things to do" as opposed to "successfully do things". And second has a lot to do with energy/drive, not only intellect. Hence I expect his intelligence be overestimated. But I agree with your estimate, which is not what I expected?
Load More