All of Ericf's Comments + Replies

Making choices between domains in pursuit of abstract goals:

Say I have an agent with the goal of "win $ in online poker" and read/write access to the internet. Obviously that agent will simulate millions of games, and play thousands of hands online to learn more about poker and get better. What I don't expect to ever see (without explicit coding by a human) is that "win $ at poker" AI looking up instructional youtube videos to learn from human experts, or telling its handlers to set up additional hardware for it, or writing child AI programs with different... (read more)

Better headline would be "I created a market on whether, in 2 months, I will believe that IQ tests measure what I believe to be intelligence" Not a particularly good market question.

Changed the title. I personally find that I often like the subjective questions because they are an opportunity for people to share their information about the topic.

What we saw when the I-15 Corridor was expanded (souther California, from Riverside to San Diego inland) was that over time people were willing to live further away from work, because the commute was "short enough," but as more people did that it got crowded again. So, total vehicle miles increased, without increasing the number of vehicle trips, since each trip was longer.

Highlighting the point in the Q&A: If you are having fun in HS or College, you don't need to leave. Put that extra energy that could be going towards graduating early into a side project (learn plumbing, coding, carpentry, auto maintenance, socializing, networking, youtubeing, dating, writing, or anything else that will have long term value regardless of what your career happens to be).

I'm a big fan of "take community College courses, and have them count for HS credits and towards your associates/bachelors" if your HS allows it.

Jave you tried playing with two (or 3 or 4) sides considered "open" - allowing groups to live if they touch those sides (abstracting away a larger board, to teach or practice tactical moves)?

"Baby sign" is just a dozen or so concepts like "more", "help", "food", "cold" etc. The main benefit is that the baby can learn to control thier hands before they learn to control thier vocal chords.

I'll just note here that "ability to automate the validation" is only possible when we already know the answer. Since the automated loom, computers have been a device for doing the same thing, over and over, very fast.

You don't necessarily need to know the correct answer beforehand to be able to validate whether or not an answer is correct. If we take Eliezer's problem of generating text that matches a given hash value, it's easy to validate whether an answer is true or not even if you don't know the answer beforehand. What's important is that the AI is sometimes able to generate correct answers. If the criteria for a correct answer are well-defined enough it can go from solving a problem 1% of the time correctly to solving it 100% of the time correctly.  ChatGPT is used by millions of people and a good portion of that will click the feedback button, especially if they optimize their UI for that. It's possible to build automated processes that will look at the problems where it currently frequently makes mistakes and learn to avoid them. It is possible to build a self-improving system around that.  If you let it do that for 10,000 different problems I would expect that it learns some reasoning habits that generalize and are useful for solving other problems as well. 

Let us introduce a third vocabulary word: Asset. An Asset is something that is consumed to provide Value, like cash in a mattress, food in a refrigerator, or the portion of a house or car that is depreciating as it is used. One of the miracles of the modern age is the ability of banks to turn Assets into Wealth many times over. It's a bit of technology built on societal trust. In the stock market example, it isn't double-counting, just different perspectives. Stock shares are a claim on the company, so the Google code is included in the Wealth of Google, a... (read more)

I distinguish between Wealth and Value as concepts. Lots of things provide Value (a croissant, a free library app, refactoring code, project management), but Wealth is specifically things that provide ongoing value when used, without being used up. For example, a code base provides ongoing value to its owners, and a refactoring code base provides more ongoing value, so that increases Wealth. Living near a beach or other nice place is a form of Wealth. Money in the bank, or in stocks, that is generating enough income to outpace inflation is Wealth. Strong relationships is Wealth. Useful knowledge is Wealth. In summary, Wealth is anything that generates (not "is convered to") Value over time.

if the code base for Google Search represents wealth, but is itself a critical component of Google-the-company’s success, then doesn’t that mean that for any financial instrument based on Google (i.e. Google stock, bonds issued by Google), to consider it also a form of wealth would be to double count that code base? i’m skeptical that money can be both a claim on wealth and also a form of wealth. it seems like it should be strictly one or the other, else you end up with a bank owning a bank owning a bank owning … with each additional layer of ownership somehow resulting in more wealth, and that seems questionable to me.
This has the consequence that in a time of no economic growth money is not wealth, which you may or may not be comfortable with. (I personally think money is a paradigmatic example of wealth, so that any definition of "wealth" that doesn't cover money is ipso facto not a definition I'd want to adopt.)
2Adam Zerner23d
Ah, I like that! I'm going to adopt that as the way I think about wealth. Thanks. Do you know if that is some sort of agreed upon way of thinking about it, or just something you came up with and find useful? Not that there's anything wrong with the latter. I think there is a remaining question about whether value is wanting or liking. If you have access to Facebook it gives you the ability to doomscroll any time you have can use a web browser. You want to doomscroll but you don't actually like it. So is the access to Facebook wealth? I guess we need two different terms, one to address wanting and the other to address liking. I don't see refactored code or a project manager on staff as providing value. I see value as the thing the end user experiences, like the ability to pay their credit card online. But it makes sense to me to consider the refactored code and PM on staff as adding wealth. Each generates value over time. And I like how this addresses my question about indirect/distal causes of value to end users: it doesn't matter that it's indirect, it still helps to generate value over time.
Answer by EricfJan 10, 202341

It turns out publishing bias is one heck of a drug. Every success of automation was touted, and every failure quietly tucked away, until one day the successes started getting smaller and less significant. We still see improvements around the edges of capability, but the big rocks, like making choices between domains in pursuit of abstract goals, remain elusive.

Hmm, can you elaborate on what you mean in the last sentence?

Having read the linked piece, I think it may be more a case of common cause then learning a new skill. People who are good at deciphering one complex system are going to be good at deciphering other complex systems. And people with experience doing that are going to be better than those without. "Seeing the meta" is just a way to ID people who have learned how to learn systems.

Depending on what level of competition and scope you are looking for, here are some suggestions:

For a tiny group (dozens of players), see

For a larger (thousands), but still moderately easy to learn,

For actual global competition (millions, good luck): or

I predict that 10-100k is the sweet spot, provided that there is some kind of competitive scene. So storybookbrawl looks like it might work.

It's not just a low-IQ human, though. I know some low IQ people and their primary deficiency is the ability to apply information to new situations. Where "new situation" might just be the same situation as before, but on a different day, or with a slightly different setup (eg task requires A, B1, B2, and C - usually all 4 things need to be done, but sometimes B1 is already done and you just need to do A, B2, and C - medium or high would notice and just do the 3 things, medium might also just do B1 anyway. Low IQ folx tend to do A, and then not know what to... (read more)

I would love it if you ran that exact test with those people you know & report back what happened. I'm not saying you are wrong, I just am genuinely curious. Ideally you shouldn't do it verbally, you should do it via text chat, and then just share the transcript. No worries if you are too busy of course.

You can also just speed-walk: quickly take full size strides, but always keep at least one foot on the ground - this keeps your torso at the same elevation for the whole journey, and eliminates the bouncing (and, added bonus, it doesn't look like you're running)

You could also say "no" if:

  1. You don't have "goals in life"
  2. Your parents are dead
  3. You don't care what your parents (or anyone else) thinks (a fairly common feeling among Autism Spectrum folx)
  4. You are focused on one or two important things (goal: get a promotion / get an A in this class / etc.), and nebulous "make my parents proud" things aren't as important.
  5. You interpret the question as referring to both or all your parents, but one or more of the previously mentioned reasons apply to some of your parents, so while you might want to make "my mom" proud, that doesn't apply to "my dad" or "my stepmom" and therefore you don't consider "my parents" a unified entity.

So, basically this comic:

Also what that other user said: the opportunity cost is only the next best thing you could have done, not all the alternatives.

1:10 management ratio is a very bad assumption as things get large. First level supervisors in Retail or Manufacturing often have 30-100 direct reports, regardless of how many levels of management exist above them (with 2-6 direct reports each).

Also, modern supply chains mean that medium to big companies often have entire additional companies "reporting into" the management structure.

There are some mainstream-ish ideas out there like Servant Leadership or Agile that try to work around this problem by dis-entangling power and prestige. If the lower level managers are organizationally "in charge" of what actually gets done (leaving the higher levels just in charge of who to hire/fire), that breaks up some of the feedback.

Another thing to do is ensure that Management is not the only (and hopefully not even the best) way to gain prestige in the company. Include the top widget makers in the high level corporate meetings. They might not care... (read more)

If you try to introduce Agile in a company that is already a deep Maze, expect to get it redefined to a version that has only superficial similarity with the original idea, and is Maze-compatible.

Basically, why most developers hate Scrum. Read the Scrum Guide, then underline the parts that your company does differently: Everyday meetings? Yes. Meetings under 5 minutes? No. Sprints? Yes, but they are two weeks instead of the recommended three. Retrospective? No. Scrum Master? There is a guy called that, but actually he is a manager. Product Owner? There is ... (read more)

The physical world is also acting continuously based on inputs it receives from people, and we don't say "The Earth" is an intelligence.

That's true. Earth doesn't act like an intelligent agent, but a model could. A current model could simulate the verbal output of a human, and that output could be connected to some actuators (or biological humans) that would allow it to act in the world. Also, Earth can't comprehend new concepts, correctly apply them and solve problems.

I think the biggest piece of an actual GI that is missing from text extenders is agency Responding to prompts and answering questions is one thing, but deciding what to do/write about next isn't even a theoretical part of thier functionality.

I’m puzzled by the apparent tension between upvoting importance of continuous learning on one hand and downvoting agreement with agency on the other hand. When transformers produce something that sounds not from humans, it’s usually because of consistency mistakes (like telling at length that it can’t speak danish… in well formed danish sentences). Maybe it’s true that continuous learning can solve the problem (if that includes learning from its own response maybe?). But wouldn’t we perceived that as exhibiting agency?
That doesn't seem like it would be a problem if it was connected to something where people constantly interacted with it. Then the model's actions would be outputted constantly, and it seems like there would be no important difference between that and it acting unprompted (heh).

If you are buying "a place to live" that can get complicated.

But buying "a building to rent out" isn't any more fraught than buying a car (or a boat, which is a more expensive vehicle; or a private jet for an even bigger "thing").

Guessing that it involved a human checking the website and/or sending an email to find out if the author had used a translation tool

Maybe, but a big player selling without explanation can also cause a panic.

There are also reputation effects with either choice.

Both examples are in "Cut time" [2÷2] - so only 2 beats to a measure.

Ooh, thanks. I knew there was a c for common time, 4:4, and I even looked it up to double check. I didn't know, or look closely enough to see, that there was also ¢.

A middle ground version of this happens (in the US) over the summer when almost all kids are out of school for 8 weeks between June 15 and Aug 15 (plus or minus), so families that can often take thier long vacations during that time.

On my 10 person team, that led to the entire 8 weeks having 1-4 people off.

I don't know, but it would depend greatly on your kettle design.

Depending on what kind of pasta dish is being made you can also:

  1. Use hot (120°F) water from the tap instead of cold.
  2. Use the Alton Brown method, where you put the pasta in the pot first, then add just enough water to cover (plus a little extra, like you would for rice), and heat from cold, stirring to keep the pasta from sticking. This reduces the total time from start to "pasta done", and you don't drain off the starchy water, so you get a creamier sauce (I use this regularly for Kraft Mac&Cheese) 2.5. Do that partially, where you still use a bunch o
... (read more)
3Stephen Bennett (Previously GWS)3mo
Does putting the pasta in the kettle gunk it up? They're a bit hard to clean and I wouldn't want starchy water in my tea.

Our family draws a line between American Football (and Boxing/MMA/etc.) Where the point of the game is to hit your opponent (and, by symmetry, be hit) and other sports where getting hit is accidental. Soccer Headers is a gray area.

While I disagree, I can see why you would make that choice. Personally I stopped doing MAs when I realized I just really didn't like getting hit in the face.
  1. Aim to explain, not persuade: the arguments against Approval Voting should at least be mentioned (eg

  2. Depressing turnout of other major candidates doesn't go down in value, so I see no support for the assertion that the campaigns would be less negative.

  1. Most or all of these vary by culture. In some places, it is expected that everyone will bring a non-dining friend to dinner, or not contribute to a wiki.

  2. Things exist because of reasons. In a counterfactual world where everyone did X, maybe Y wouldn't exist, but in reality very few people do X - usually because doing X is culturally frowned on, or requires more cash, brains, or neurodivergences than 67% of everyone.

Yep. It's perfectly consistent to say "I will do X even though, if everyone did X, bad things would happen because if enough people start doing X that it starts to become a problem, then I'll stop doing X".

Rice is the killer, though. Make sure you keep anything bad for the dishwasher out of the dishwasher.

With 2 hypothesis: die is fair/die is 100% loaded, a single roll doesn't discriminate at all. The key insight is that you have to combine Baysean and Frequentist theories. The prior is heavily weighted towards "the die is fair" such that even 3 or 4 of the same number in a row doesn't push the actionable probability all the way to "more likely weighted" but as independent observations continue, the weight of evidence accumulates.

Cautiknary tale: There was a browser game about sustainable fishing that was supposed to show the value of catch shares, but the concept was only introduced at the end of the game, so after playing for 30 minutes I hadn't even seen it (and had gotten bored with the mechanics)

Don't wait too long into the play experience to have your player start interacting with yiur key concepts.

Isn't that begging the question? If the goal is to teach why being optimistic is dangerous, declaring by fiat that an unaligned AI ends the world skips the whole "teaching" part of a game.

Yes, it doesn't establish why it's inherently dangerous but does help explain a key challenge to coordinating to reduce the danger.  
  1. If you are tall, put your feet (instead of your bag) under the seat in front of you.

  2. Every flight I've taken (circa 2000-2014) was either at 95%+ capacity, or was a 1 hour commuter leg with a half-full plane #misleadingAverages

  1. Humans who are not concentrating are not general intelligences.
  2. It's a warning label, not a categorical imperative.
  3. The warning explains why, so anyone who chooses to think about it can figure out what precautions to take, if they chose to disregard the warning.
  4. Sometimes inexperienced people become parents and/or babysitters. Everyone has to learn about uneven microwave heating at some point - better from a written warning than a trip to the ER.
It's not just the warning label, though. Even the public health sites that have multiple dense paragraphs on proper treatment of formula stick with "don't microwave it, ever".

This feels like an incomplete thought. Maybe another editing pass could add some overall structure to the essay.

I think even with search capabilities, it wouldn't accurately sort a set of real submissions (eg, if a high school English teacher have the assignment to thier ~150 students - with a note that they would be only mildly punished for plagiarism on this one assignment to ensure at least some actual plagiarized essays in the sample)

Accurately (ie 0% false positive and no more than 10% false negative) idenrify if one paragraph book reviews are plagiarized.

It might have web search capabilities a la WebGPT, in which case I wouldn’t be confident of this. Without web search I’d agree.

This result seems not surprising. Finding the most common result in a large dataset is explicitly what computers are good at, and people are not good at. That's kind of why we invented computers. There is little value in knowing that 70% of the time the sentence is "Now, the developer is planning for phase 2 trials" and not "Phase 2 trials are the next step" "Now the developer, [name], is planning phase 2 trials" "Now trials enter phase 2" Or any other variations. Our understanding is at the sentence level, and based on a tiny and biased sampling from the possible set of source data.

Agreed, while at the same time I’m very glad that this research has been done, and wish to congratulate those involved accordingly. A lot of good research is obvious in retrospect, but there were enough smart people who prior to this experiment displayed contradictory opinions that this result is not trivial. As such, kudos on the great work!

I feel like there is some vocabulary confusion in the genesis of this post. "Reward" is hard coded into the agents. The Dinosaurs of Jurrasic Park (spoiler alert) were genetically engineered to lack iodine. So, the trainers could use iodine as a reward to incentives other behaviors because be definition the dinos valued iodine as a terminal value. In humans Seratonin and Dopamine bonding to appropriate brain receptors are DNA-coded terminal values that inherently train us to pursue certain behaviors (eg food, sex). An AI is, by definition, going to take whatever actions maximize its Reward system. That's what having a Reward system means.

I think the terminological confusion is with you: what you're talking about is more like what is called in some RL algorithms a value function. Does a chess-playing RL agent make whichever move maximises reward? Not unless it has converged to the optimal policy, which in practice it hasn't. The reward signal of +1 for a win, 0 for a draw and -1 for a loss is, in a sense, hard-coded into the agent, but not in the sense that it's the metric the agent uses to select actions. Instead the chess-playing agent uses its value function, which is an estimate of the reward the agent will get in the future, but is not the same thing. The iodinosaurs example perhaps obscures the point since the iodinos seem inner aligned: they probably do terminally value (the feeling of) getting iodine and they are unlikely to instead optimise a proxy. In this case the value function which is used to select actions is very similar to the reward function, but in general it needn't be, for example in the case where the agent has previously been rewarded for getting raspberries and now has the choice between a raspberry and a blueberry. Even if it knows the blueberry will get it higher reward, it might not care: it values raspberries, and it selects its actions based on what it values.

"Divisibility" is meaningless. Your accounting ledger can use however many decimal places are desired. And unless price tags are denominated in a currency, it doesn't matter if I have 1,000,000 yen or .0000001 BTC. And if price tags are written in a currency, it helps to have common items cluster in a clean set of values, preferably near 1, but Xthousand or Xmillion or Xhundreths or Xthousandths etc. can also work.

To be fair, we did have animals that served the purpose of computers. We even called them computers - as in, people whose job it was to do calculations (typically Linear Algebra or Calculus or Differential Equations - hard stuff).

2Ege Erdil7mo
This is true, but if this level of similarity is going to count, I think there are natural "examples" of pretty much anything you could ever hope to build. It doesn't seem helpful to me when thinking about the counterfactual I brought up.

The example that springs to mind is the 19th century US. And what they did was conquer and drive off (or kill off) the existing residents west of the Mississippi to make room for the new immigrants.

It can just be a random number that is a number and not, say, a telephone dialing pattern or PIN. But it can't be a number with relevant context.

So if you're selling a used car, mention big numbers without meaningful context like "they made 123,456 of this model year." But if you mention the Milage, that has a "slot" I the buyer's brain, and won't be used as an anchor for the price.

In the Rich Dad/Poor Dad frame, some people, when they have money, spend that money multiple times (eg, with a bank balance of $500 they charge up $300 on thier credit card, and withdraw $300 in cash, and commit to a monthly $20 subscription box, and plan a party for next month that will cost $300).

No, I don't remember exactly where on LW I saw it - just wanted to aknowledge that I was amplifying so.eone else's thoughts.

My college writing instructor was taken aback when I asked her how to cite something I could quote, but didn't recall from where, but her answer was "then you can't use it" which seemed harsh. There should be a way to aknowledge plagiarism without knowing or stating who is being plagiarized - and if the original author shown up, you've basically pre-conceded any question of originality to them.

Thx for being clear about it.

I don't know that anyone has done the studies, but you could look at how winners of large lotteries behave. That is a natural example of someone suddenly gaining a lot of money (and therefore power). Do they tend to keep thier previous goals, amd just scale up thier efforts, or do they start doing power-retaining things? I have no idea what the data will show - thought experiments and amecdotes could go either way.

Let me Google that for you. []
Load More