I am pretty sure there are honest and well-meaning financial advisors too. E.g. there are valid and professional blogs/youtube channels (e.g. "The Plain Bagel"). There are some confounding factors though, in particular, pretty much anyone can call himself a financial advisor. Even if they do not know anything about finance and their only goal is to sell overpriced insurance policies.
"Essentially, the Culture must have value lock-in for the values of the Minds that were present at its founding."
Probably at least some value lock-in is somewhat required, unless you want the particular civilization to fracture. If it is allowed to create your own cult of space nazis intent on exterminating everyone else, your post scarcity utopia may not live long. Even "live and let live" is a value, and many people do not subscribe to it.
"But I think it is more likely that it was achieved by genetic changes, so that it’s safe to raise full Culture citizens in other cultures"
I agree, or at least that genetic change is also a strong contributor. However, if you think about sociopathy as a disability that in most cases makes one's and their fellows life worse, this genetic modification is a good thing.
"In other words, these superhuman minds have not solved alignment"
I think they did kinda solve it, as long as the other system is somewhat dumber than they are. Just as we are able to more-or-less align dumb systems, probably Minds can do that too. As long as they are not another mind or comparable level.
I think it really is a somewhat large disempowerment to humanity, but I see this as a better alternative. It might sound great to make all the important decisions, but in the end, humans are just too limited and we would likely fail and ruin ourselves. A Culture where humans make the final decision could not have won the Idiran War. If I am to play chess against Kasparov and my life depends on it, I would much rather let Stockfish make the decisions.
It sounds so weird I must be missing something. Any idea from someone what is the chain of thought behind it?
Adding context/(kind-of) counter argument from reddit (the link has a link to the main article and contains a summary of it):
I think the comments are also worth a read. I want to share one particular comment here, which I think has a good explanation/hypothesis regarding the situation:
The scaling up of FPV drones for the Ukrainians was definitely the result of artillery and mortar ammo shortages. But that can't be the only answer, as Russia never suffered that degree of shortage and they've gone as hardcore into FPV drones, if not more so, than the Ukrainians.
I think the biggest problem relying on artillery and mortars in THIS war is the ultra static nature of it. With the lines barely moving, it's very hard to create an artillery or mortar firing position that has decent survivability. Enemy recon drones, which can't be jammed or shot down easily (as most use freq hopping, fly at altitude, have limited radar signatures, etc), they are prowling the tactical rear areas. Since the start of the war, indirect fire has had to greatly disperse, especially artillery, which operates as single guns now. They can't even do "shoot and scoot" for survivability, since moving is where most of them will be caught. And if they fire, then counterbattery radars will detect their location. What most are doing is having to dig into treelines, at the least using maximum camouflage, if not building overhead cover, making it much more difficult to take them out with counterbattery.
However, there is only a certain number of hiding spots they can do that. I heard a report last fall about how the Russians in the Pokrovsk sector, despite having ample artillery ammo, had their fire rates drop, because as they were advancing, their artillery could not find and occupy enough hidden firing points to adequately support ground operations. It will be just as hard with mortars, if not harder, because even more enemy recon drones (including those belonging to the enemy's small unit level) can reach into their range to see.
Whereas, drone operators are much harder to perform the equivalent of counterbattery, especially destructive (actually hitting them). While artillery and mortars are next to impossible to stop once they're airborne, for drones its often the opposite, it's not even worth trying to go after the drone operators, they're too hard to kill, whereas its much easier to try to down the drone using EW especially, or using some sort of passive system, like C-UAS cage. Or active systems, to include dudes armed with shotguns, other drones, or even hard kill remote gun systems that the Ukrainians and Russians haven't developed/fielded much of.
I'm personally convinced that the ultra static nature of the Russo-Ukraine War is mostly responsible for most of these novel TTPs. And why a lot of this isn't applicable outside Ukraine.
Thank you for putting in the effort required to review this. Post like this help a lot in interpreting hyped literature. I am also skeptical myself whether LLMs are the path to AGI, and likely would have counted the paper as an additional (small) datapoint in favour of my conclusion if not for your detailed summary (I did not read the original paper myself and had no intention to, hence only 'small' data point).
"There’s a common assumption in many LLM critiques that reasoning ability is binary: either you have it, or you don’t."
I agree, and would even push it further: I think the crux of the whole issue is our lack of good understanding of the concept space we refer to with words such as "reasoning", "thinking", "intelligence" or "agency".
I do have a hunch that we have some inherent limitations while trying to use our own "reasoning"/"intelligence"/etc to understand this space itself (something like Gödel's incompleteness theorem), but I do not have a proof. Than again, maybe not and we will figure it out.
Whatever the case is, we are not good at it right now. I imagine an analogy for this if we had the same (lack of) understanding for moving around in physical space:
Car-3.5 is invented, and is being used to carry things from one location to another. Some people claim it cannot "move on its own", as Car-3.5 could not move over hill #1 or muddy field #1, so it is just a road following engine, not a general artificial mover. Car-4 is created, with stronger engine, being able to climb over hill #1, Car-o1 has better transmission and wheels, being able to cross muddy field #1. It still cannot cross hill #2 or creek #1, so some people claim again, that it cannot actually move on its own. Other people show that just increasing engine power and doing tricks like adjusting wheel structure will help overcome this, and even most humans would be unable to cross hill #2 or creek #1, we would just go along the roads and use the tunnel or bridge to cross these, just as Car-o1 does it. Are we not general movers after all either? Do we need to increase only engine power and get some scaffolding in place for creek crossing to get a something that can move at least as well as a human in all spaces?
Replace the "Car-" string with "Humanoid_robot-", and think about it again. Change back to "Car-" but imagine this is the thought process of a horse, and think about it again.
We do not know which of the three variant describes our situation best.
Well, the disturbed protagonists in the Culture series (as in: books, and in the whole of the fictional universe) are usually not from the "Culture" (one particular civilizations within the whole fictional universe), but outsiders hired to act as agents.
In books about the Culture sci fi universe such things are described a couple of times. E.g. in the novel "Use of Weapons" the "crew" (the ship is fully automated, so more like permanent passengers) of a ship deliberately weaken their immune system to basically get a seasonal cold just for the experience, which otherwise could not happen due to their genetically enhanced immune system.
1Is the data reliable? I just did a search on PornHub, xHamster and iXXX for the following keywords:
violence, incest, rape, torture
Results:
Pornhub returns a warning "Your search could be for illegal and abusive sexual material...etc" (it is a longer description), returning no videos for 'violence', 'incest' and 'rape'. It does return results for 'torture' though.
The cases is somewhat similar for xHamster and iXXX, except they do not return any special message for the first three, but also do not return any results. They do work fine with violence though.
It does seem there is some official or unofficial policing.
Any opinion on this regarding being a somewhat good solution?
https://www.lesswrong.com/posts/Q3huo2PYxcDGJWR6q/how-to-corner-liars-a-miasma-clearing-protocol
How likely do you think this quality aspect to stay there long-term? Are you able to allocate more time to quality due to having been sped up on the "core" part of development, but expectations haven't been increased accordingly? When organizations realize they can push out more productivity, speed of development timeline might be forced to increase and quality may drop back to current levels.
Alternatively, do you think paying/preventing technical debt is quicker with LLM assistance than otherwise? I mean as relative cost compared to building out the specific features.
By the way, what IDE are you using with Claude?