This is a special post for quick takes by RHollerith. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
rhollerith_dot_com's Shortform
29RHollerith
8Vladimir_Nesov
7RHollerith
23RHollerith
8RHollerith
15gwern
2habryka
1cubefox
6RHollerith
4RHollerith
6Richard_Kennaway
4cubefox
2Nathan Helm-Burger
2Nathan Helm-Burger
4RHollerith
2Nathan Helm-Burger
2RHollerith
2Nathan Helm-Burger
4RHollerith
4RHollerith
2RHollerith
6Garrett Baker
10Viliam
2RHollerith
2Mateusz Bagiński
2Garrett Baker
2Mateusz Bagiński
2Garrett Baker
2Mateusz Bagiński
-6RHollerith
2irarseil
-10RHollerith
4gwern
2[comment deleted]
Many believe that one hope for our future is that the AI labs will makes some mistake that will kill many people, but not all of us, resulting in the survivors finally realizing how dangerous AI is. I wish people would refer to that as a "near miss", not a "warning shot". A warning shot is when the danger (originally a warship) actually cares about you but cares about its mission more, with the result that it complicates its plans and policies to try to keep you alive.
While not similar to bounded AI disasters, warning shots don't require caring about the other side. True warning shots still make sense as signaling, a standard rung on the ladder of escalation, cheaper than the full cost of violence.
Good point. Change my final sentence to, "A warning shot is made by the entity capable of imposing damaging consequences on you -- to alert you and to give you a way to avoid the most damaging of the consequences at its disposal."
I appreciate it when people repost here things Eliezer has written on Twitter or Facebook because it makes it easier for me to stay away from Twitter and Facebook.
(OTOH, I'm grateful to Eliezer for participating on Twitter because posting on Twitter has much higher impact than posting here does.)
1I'm tired of the AI-generated art that writers here put in their posts and comments. Some might not be able to relate, but the way my brain works, I have to exert focus for a few seconds to suppress the effects of having seen the image before I can continue to engage with the writer's words. It is quite mentally effortful.
2I agree. The problem with AI-generated images is that any image you can generate with a prompt like "robot looking at chessboard" is going to contain, almost by definition, no more information than that prompt did, but it takes a lot longer than reading the prompt to look at the image and ascertain that it contains no information and is just AI-generated imagery added 'to look nice'. This is particularly jarring on a site like LW2 where, for better or worse, images are rarely present and usually highly-informative and dense with information when present.
Worse, they usually don't 'look nice' either. Most of the time, people who use AI images can't even be bothered to sample one without blatant artifacts, or to do some inpainting to fix up the worst anomalies, or figure out an appropriate style. The samples look bad to begin with, and a year later, they're going to look even worse and more horribly dated, and make the post look much worse, like a spammer wrote it. (Almost all images from DALL-E 2 are already hopelessly nasty looking, and stuff from Midjourney-v1--3 and SD1.x likewise, and SD2/SD-XL/Midjourneyv4/5 are ailing.) It would be better if the authors of such posts could just insert text like [imagine 'a robot looking at a chessboard' here] if they are unable to suppress their addiction to SEO images; I can imagine that better than they can generate it, it seems.
So my advice would be that if you want some writing to still be read in a year and it to look good, then you should learn how to use the tools and spend at least an hour per image; and if you can't do that, then don't spend time on generating images at all (unless you're writing about image generation, I suppose). Quickies are fine for funny tweets or groupchats, but serious readers deserve better. Meaningless images don't need to be included, and the image generators will be much better in a year or two anyway and you can go back and add them if you really feel the need.
For Gwern.net, I'm satisfied with the images I've generated for my dropcap fonts or as thumbnail previews for a couple of my pages like "Suzanne Delage" or "Screwfly Solution" (where I believe they serve a useful 'iconic' summary role in popups & social media previews), but I also put in a lot of work: I typically generate scores to hundreds of images in both MJv5/6 & DALL-E 3, varying them heavily and randomizing as much as possible, before inpainting or tweaking them. (I generally select at a 1:4 or less ratio, and then select out of a few dozen; I archive a lot of the first-stage images in my Midjourney & DALL-E 3 tag-directories if you want to browse them.) It takes hours. But I am confident I will still like them years from now.
1I like them sometimes, but a lot of them are IMO not executed well-enough. I did like them in Owen CB's latest posts.
I also find them irksome for some reason. They feel like pollution. Like AI generated websites in my Google results.
An exception was the ghost cartoon here. The AI spelling errors added to the humor, similar to the bad spelling of lolcats.
We will soon learn how to make machines that are better at planning and better at reality than we are. That is a big problem.
The utility function is an abstraction that does not capture the richness of the behavior of agents in the real world, but the better an agent is at rationality, the more accurate (and comprehensive) the abstraction of the utility function becomes IMHO at describing the agent. I suspect that it never makes sense to model a utility function as changing over time.
Or maybe it makes sense only as a mental shortcut to be used only when we do not have time to make a proper analysis. People of course discover new wants and new preferences as they go through life, but this can be taken into account by saying (or noticing) that the person does not know what their (unchanging) utility function is in every detail, and every now and then he or she learn a previously-unknown detail (or fact) about their utility function.
A baby is not discovering who it is as its mind develops. It is becoming who it will be. This process does not stop before death. At no point can one say, “THIS is who I am” and stop there, imagining that all future change is merely discovering what one already was (despite the new thing being, well, new).
IMHO, utility functions only make sense for “small world” problems: local, well-defined, legible situations for which all possible actions and outcomes are known and complete preferences are possible. For “large worlds” the whole thing falls apart, for multiple reasons which have all been often discussed on LW (although not necessarily with the conclusion that I draw from them). For example, the problems of defining collective utility, self-referential decision theories, non-ergodic decision spaces, game theory with agents reasoning about each other, the observed failures of almost everyone to predict the explosion of various technologies, and the impossibility of limiting the large world to anything less than the whole of one’s future light-cone.
I do not think that any of these will yield merely to “better rationality”.
It's possible to view utility functions just like probability functions ("probability distributions"), namely as rational restrictions on a subjective state of mind at a particular point in time. Utilities can describe desires, just as probabilities can describe beliefs. That doesn't cover multi-agent rationality, or diachronic changes over time, but that isn't much different from probability theory. (Richard Jeffrey's axiomatization of utility theory is expressed for such a "subjective Bayesian" purpose, but unfortunately it isn't well known.)
Yeah, when I started studying neuroscience and the genetics of neurons I was kinda mind-blown by just how much change there is throughout the lifetime. There are certain things which are fairly static, like the long-range axons in your brain (aka spanning more than a millimeter). Other things, like the phenotype (the set of expressed genes) and the synapses change from second to second.
Indeed, it caused a bit of a fuss in the neuroscience community when enough evidence was gathered that we had to finally admit that the synapses/dendritic spines in the brain fluctuate too fast and chaotically to be the storage site of learned information that they were long thought to be. Other things may be, such as proteins that remain in place in the cell while the dendritic spine grows and collapses, or certain patterns of gene expression (triggered by reinforced synaptic activity during learning) which code for a propensity to form a synapse in a particular location... we just don't know at this point.
I used to think similarly, but my friend Max Harms convinced me otherwise. He explained that what he, and others he was in agreement with, meant by 'utility function' was not the simple thing written down in the rules of a model doing RL. He meant a much grander thing, the description of a person's life and behavior as seen from outside time itself by an omniscient observer capable of perfectly simulating the observed agent in infinitely many contexts. A fundamental timeless truth about the state of the universe, not a mere human knowable description.
From this point of view, any utility function that can be written down about some complex real-world agent is likely just a crude approximation of the true utility function, which is potentially too complex to be written in full within the bounds of our observable universe.
I feel like we need two different terms for these two different concepts.
I think you are responding to my "is an abstraction that does not capture the richness" which on reflection I'm not attached to and would not include if I were to rewrite my comment.
Your "seen from outside time" suggests that maybe you agree with my "it never makes sense to model a utility function as changing over time". In contrast, some on LW hold that a utility function needs to change over time (for human potential to remain "open" or some such). I think that that doesn't work; i.e., if it changes over time, I think that it is incompatible with the four axioms of VNM-rationality, so these people should switch to some other term than "utility function" for the same reason that someone using the term "point", "line" or "plane" in a way that is inconsistent with the axioms of geometry should find some other term. (I have no doubt that your "much grander thing" is compatible with the four axioms.)
(In the context of RL, I'm used to hearing it referred to as a reward function.)
1Yeah, I'd more say that a utility function can be a description of an agent at a particular point in time, or across the agents entire existence, depending on how you frame it.
Like, for an instance in time (i) where you are evaluating what an agent will do next, there is some mathematical description of what they will do next based on their state of existence and the context that they are in.
If you have several moments in time, you could define such a description for each moment. Indeed, as the agent may change over time, and the context almost certainly does, the utility function couldn't be static (unless you were referring to the outside-of-time all-timepoints-included utility function).
Does that make sense?
I'm not stating this with much confidence, this doesn't feel an idea I fully grok, I'm just trying to share what I think I've learned and learn from you what you know, since it seems like you've thought this out more than I have.
My assertion is that all utility functions (i.e., all functions that satisfy the 4 VNM axioms plus perhaps some additional postulates most of us would agree on) are static (do not change over time).
I should try to prove that, but I've been telling myself I should for months now, but haven't mustered the energy, so am posting the assertion now without proof because an weak argument posted now is better then a perfect argument that might never be posted.
I've never been tempted to distinguish between "the outside-of-time all-timepoints-included utility function" and other utility functions such as the utility function referred to by the definition of expected utility (EU (action) = sum over all outcomes of (U(outcome) times p(outcome | action))).
Ok, the static nature of a utility function for a static agent makes sense. But in the case of humans, or of ML models with online (ongoing) learning, we aren't static agents. The continuity of self is an illusion. Every fraction of a second we become a fundamentally different agent. Usually this is only imperceptibly slightly different. The change isn't a random walk however, it's based on interactions with the environment and inbuilt algorithms, plus randomness and (in the case of humans) degradation from aging. Over the span of seconds, this likely has no meaningful impact on the utility function. Over a longer span, like a year, this has a huge impact. Fundamental values can shift. The different agents at those different timepoints surely have different utility functions, don't they?
The different agents at those different timepoints surely have different utility functions, don't they?
IMHO, no.
1Now that the risks of AI are getting mainstream traction, we can expect the people who want to rush forward with AI research to increase their efforts to influence public opinion. In particular, most people will come to rely heavily on large language models to get information much like they rely heavily on search engines today, and the most popular large language models (LLMs) will probably be tuned so as to downplay the risks of AI research (particularly the argument that AI research is so dangerous that it should be halted for a few decades). It is not too early to think about how to counter that.
There's a good chance that "postpone the intelligence explosion for a few centuries" is a better motto than "stop AI" or "pause AI".
Someone should do some "market research" on this question.
That's a whole seven words!, most of which are a whole three syllables! There is no way a motto like that catches on.
Maybe "motto" is the wrong word. I meant words / concepts to use in a comment or in a conversation.
"Those companies that created ChatGPT, etc? If allowed to continue operating without strict regulation, they will cause an intelligence explosion."
You can still include it in your protest banner portfolio to decrease the fraction of people whose first impression is "these people are against AI in general" etc.
I don't think Pause AI's current bottleneck is people being pro AI in general not wanting to join (but of course I could be wrong). Most people are just against AI, and Pause AI's current strategy is to make them care enough about the issue to use their feet, while also telling them "its much much worse than you would've imagined bro".
Some people misinterpret/mispaint them(/us?) as "luddites" or "decels" or "anti-AI-in-general" or "anti-progress".
Is it their(/our?) biggest problem, one of their(/our?) bottlenecks? Most likely no.
It might still make sense to make marginal changes that make it marginally harder to do that kind of mispainting / reduce misinterpretative degrees of freedom.
I suspect the vast majority of that sort of name-calling is much more politically motivated than based on not seeing the right slogans. For example if you go to Pause AI's website the first thing you see is a big, bold
and AI pause advocates are constantly arguing "no, we don't actually believe that" to the people who call them "luddites", but I have never actually seen anyone change their mind based on such an argument.
My model is that
- some of it is politically/ideologically/self-interest-motivated
- some of it is just people glancing at a thing, forming an impression, and not caring to investigate further
- some of it is people interacting with the thing indirectly via people from the first two categories; some subset of them then take a glance at the PauseAI website or whatever, out of curiosity, form an impression (e.g. whether it matches what they've heard from other people), don't care to investigate further
Making slogans more ~precise might help with (2) and (3)
Curated and popular this week