All of magfrump's Comments + Replies

Not from OpenAI but the language sounds like this could be the board protecting themselves against securities fraud committed by Altman.

1Oliver B3mo
I doubt the reason for his ousting was fraud-related, but if it was I think it's unlikely to be viewed as securities fraud simply because OpenAI hasn't issued any public securities. I'm not a securities lawyer, but my hunch is even if you could prosecute Altman for defrauding e.g. Microsoft shareholders, it would be far easier to sue directly for regular fraud.
What kind of securities fraud could he have committed? 

I am confused about the opening of your analysis:

In some sense, this idea solves basically none of the core problems of alignment. We still need a good-enough model of a human and a good-enough pointer to human values.

It seems to me that while the fixed point conception here doesn't uniquely determine a learning strategy, it should be possible to uniquely determine that strategy by building it into the training data.

In particular, if you have a base level of "reality" like the P_0 you describe, then it should be possible to train a model first on this real... (read more)

My issue isn't with the complexity of a Turing machine, it's with the term "accessible." Universal search may execute every Turing machine, but it also takes adds more than exponential complexity time to do so.

In particular because if there are infinitely many schelling points in the manipulation universe to be manipulated and referenced, then this requires all of that computation to causally precede the simplest such schelling point for any answer that needs to be manipulated!

It's not clear to me what it actually means for there to exist a schelling point... (read more)

I'm confused by your intuition that team manipulation's universe has similar complexity to ours.

My prior is that scaling the size of (accessible) things in a universe also requires scaling the complexity of the universe in a not-bounded way, probably even a super-linear way, such that fully specifying "infinite computing power" or more concretely "sufficient computing power to simulate universes of complexity <=X for time horizons <=Y" requires complexity f(x,y) which is unbounded in x,y, and therefore falls apart completely as a practical solution (... (read more)

2Charlie Steiner2y
One antidote to feeling like simple Turing machines can't contain complicated stuff is to consider Universal Search (I forget its real name if that's not it) - this is a Turing machine that iterates over every Turing machine. Turing machines can be put in an ordered list (given a choice of programming language), so Universal Search just runs them all. You can't run them in order (because many never halt) and you can't run the first step of each one before moving onto step two (because there's an infinite number of Turing machines, you'd never get to step two). But you can do something in between, kinda like the classic picture of how to make a list of the rational numbers. You run the first Turing machine for a step, then you run the first and second Turing machines for a step, and you run the first, second, and third... at every step you're only advancing the state of a finite number of Turing machines, but for any finite Turing machine, you'll simulate it for an arbitrary number of steps eventually. And all of it fits inside Universal Search, which is quite a simple program. As for finding us in our universe, your estimate makes sense in a classical universe (where you just have to specify where we are), but not in a quantum on (where you have to specify what branch of the universe's wavefunction we're on).

Many people commute to work in businesses in San Francisco who don't live there. I would expect GDP per capita to be misleading in such cases for some purposes.

Broadening to the San Francisco-San Jose area, there are 9,714,023 people with a GDP of $1,101,153,397,000/year, giving a GDP/capita estimate of $113,357.  I know enough people who commute between Sunnyvale and San Francisco or even further that I'd expect this to be 'more accurate' in some sense, though obviously it's only slightly lower than your first figure and still absurdly high.

But the c... (read more)

tl;dr: if models unpredictably undergo rapid logistic improvement, we should expect zero correlation in aggregate.

If models unpredictably undergo SLOW logistic improvement, we should expect positive correlation. This also means getting more fine-grained data should give different correlations.

To condense and steelman the original comment slightly:

Imagine that learning curves all look like logistic curves. The following points are unpredictable:

  1. How big of a model is necessary to enter the upward slope.
  2. How big of a model is necessary to reach the plateau.
  3. How
... (read more)
I think I endorse this condensation/steelman! Thank you for making it :-) For more in this vein maybe: why forecasting S-curves is hard. The associated video is pretty great.

Seconded. AI is good at approximate answers, and bad at failing gracefully. This makes it very hard to apply to some problems, or requires specialized knowledge/implementation that there isn't enough expertise or time for.

Based on my own experience and the experience of others I know, I think knowledge starts to become taut rather quickly - I’d say at an annual income level in the low hundred thousands.

I really appreciate this specific calling out of the audience for this post. It may be limiting, but it is also likely limiting to an audience with a strong overlap with LW readership.

Everything money can buy is “cheap”, because money is "cheap".

I feel like there's a catch-22 here, in that there are many problems that probably could be solved with money, but I don't know how ... (read more)

Sounds like you want roughly the sequence Inadequate Equilibria.

I think this post does a good job of focusing on a stumbling block that many people encounter when trying to do something difficult. Since the stumbling block is about explicitly causing yourself pain, to the extent that this is a common problem and that the post can help avoid it, that's a very high return prospect.

I appreciate the list of quotes and anecdotes early in the post; it's hard for me to imagine what sort of empirical references someone could make to verify whether or not this is a problem. Well known quotes and a long list of anecdotes is a su... (read more)

Your model of supporters of farm animal welfare seems super wrong to me.

I would predict that actually supporters of the law will be more unhappy the more effect it has on the actual market, because that reveals info about how bad conditions are for farm animals. In particular if it means shifting pork distribution elsewhere, that means less reduction in pig torture and also fewer options to shift consumption patterns toward more humanely raised meat on the margins.

Those costs can be worth paying, if you still expect some reduction in pig torture, but obviously writing laws to be better defined and easier to measure would be a further improvement.

70% compute, 30% algo (give or take 10 percentage points) over the last 25 years. Without serious experiments, have a look at the Stockfish evolution at constant compute. That's a gain of +700 ELO points over ~8 years (on the high side, historically). For comparison, you gain ~70 ELO per double compute. Over 8 years one has on average gained ~400x compute, yielding +375 ELO. That's 700:375 ELO for compute:algo

Isn't that 70:30 algo:compute?

Yes, sorry, I got that the wrong way around. 70%=algo

I'm curious about what the state of evidence around long covid is now, and especially how protective vaccines are against it. I imagine there still isn't much data about it yet though.

A friend of mine on Facebook notes that the instances of blood clots in Germany were concerning because in Germany mostly young health care workers are getting vaccinated, where it's both more possible to distinguish small numbers of blood clots from chance and more concerning to see extreme side effects.

The rate is still low enough that pausing vaccination is (obviously) a dangerous move, but dismissing the case that blood clots may be caused by the vaccine isn't a fair assessment of the evidence, and that may be important in maybe two years when supply of non-AZ vaccines is no longer a limit for the world.

Do you have any thoughts on what you'd do differently to be more personally confident doing this again?

Good question.

One thing I looked into was obtaining fast antibody tests (basically strips of paper with some proteins and colloidal gold soaked into them). They're "research use only" and hard to get your hands on if you're in the US, but if you're outside the US (or have a friend outside the US willing to help) it should be easier. They would make it dramatically easier and less error-prone to test, and they'd also test for binding against full COVID proteins directly (rather than the radvac peptides). If I were going to invest much more effort into this ... (read more)

Maybe but the US number lines up with 1% of the population lines up with the top 1% figure; if people outside the US are ~50x as likely to be top-1% at various hobbies that's a bold statement that needs justification, not an obvious rule of thumb!

Or it could be across all time, which lines up with ~100 billion humans in history.

I think "a billion people in the world" is wrong here--it should only be about 75 million by pure multiplication.

3HDMI Cable3y
They might have been talking about the total amount of people with the potential to become better than you at the specific thing rather than the pure percentage of people who would be if everyone tried.

I see, I definitely didn't read that closely enough.

Looks like the initial question was here and a result around it was posted here. At a glance I don't see the comments with counterexamples, and I do see a post with a formal result, which seems like a direct contradiction to what you're saying, though I'll look in more detail.

Coming back to the scaling question, I think I agree that multiplicative scaling over the whole model size is obviously wrong. To be more precise, if there's something like a Q-learning inner optimizer for two tasks, then you need the cross product of the state spaces, so the size of ... (read more)

If you mean to suggest this post has a positive result, then I think you're just mis-reading it; the key result is  which says that under some assumptions, there exists a task for which the minimal circuit will engage in deceptive behavior (IE is a malign inner optimizer). The comment with a counterexample on the original post is here.

I'm replying on my phone right now because I can't stop thinking about it. I will try to remember to follow up when I can type more easily.

I think the vague shape of what I think I disagree about is how dense GPT-3's sets of implicit knowledge are.

I do think we agree that GPT-5000 will be broadly superhuman, even if it just has a grab bag of models in this way, for approximately the reasons you give.

I'm thinking about "intelligent behavior" as something like the set of real numbers, and "human behavior" as covering something like rational numbers, so we ca... (read more)

There was indeed a post posing this question a while back, and discussion in the comments included a counterexample: a construction of a minimal circuit that would be malign. To my eye, the whole crux of the inner alignment problem is that we have no results saying things like: * The simplest program which solves a problem is not an inner optimizer * The minimal circuit which solves a problem is not an inner optimizer * The fastest program solving a problem is not an inner optimizer Or any such thing. If we had such a result, then we'd have a grip on the problem. But we don't currently have any result like that, nor any plausible direction for proving such a result. And indeed, thought on the problem suggests that these hypotheses are probably not true; rather, it seems surprisingly plausible, once you think about it, that indeed minimal solutions may sometimes be inner optimizers. My thinking is that it's probably somewhere between the two. Multiplicative complexity suggests memorizing a lookup table. But there is regularity in the universe. There is transfer learning. Right. I think transfer learning speaks pretty strongly against this multiplicative model.

I think this is obscuring (my perception of) the disagreement a little bit.

I think what I'm saying is, GPT-3 probably doesn't have any general truth+noise models. But I would expect it to copy a truth+noise model from people, when the underlying model is simple.

I then expect GPT-3 to "secretly" have something like an interesting diagnostic model, and probably a few other narrowly superhuman skills.

But I would expect it to not have any kind of significant planning capacity, because that planning capacity is not simple.

In particular my expectation is that co... (read more)

Thanks for trying further to bridge the gap! (It would be nice if you flagged a little better which things you think I think / which things you think I disagree with) OK, that makes sense. So you're not saying that GPT contains useful diagnostic models in the overall statistics of its models of Reddit users (EG that someone complaining of one symptom will often complain of another), nor are you saying that GPT contains a good model of disease which it then feeds through noise (EG it decides that a particular user is a diabetic, which shapes how it plays that character going forward, but the character itself doesn't know it is diabetic, so may say some confused things); indeed, you are denying the latter. But what you are saying is that GPT plays the role of users who do have their own internal models, so it must mimic those models (in cases where that's not too hard to learn). I find this hard to square with your earlier statement: Where it sounds like you think GPT will know something medical science does not know. As for me, I find all of these to be broadly possible. I'd have to think more to give a meaningful plausibility ranking. How many? I am thinking of "medical diagnostics" as just one example of many many areas of expertise which border on GPT's competence. I wasn't thinking there was any special reason to single out medicine in particular as something GPT might have implicit knowledge about. On my model, if GPT contains implicit medical competence, it probably contains similar competence in "every area", although I'm not sure how to quantify. Maybe a similar hidden competence in at least 50% of professions at least as numerous as, say, physicist? (Really, what matters is how much discussion of a profession there is online, not how numerous that profession is, but maybe it's an OK proxy.) My crux would be something special about medical diagnosis such that we especially expect GPT to have implicit talent there. It seems like you think planning cap

This seems like it's using the wrong ontology to me.

Like, in my mind, there are things like medical diagnostics or predictions of pharmaceutical reactions, which are much easier cognitive tasks than general conversation, but which humans are specialized away from.

For example, imagine the severity of side effects from a specific medication. can be computed by figuring out 15 variables about the person and putting them into a neural network with 5000 parameters, and the output is somewhere in a six-dimensional space, and this model is part of a general model... (read more)

Let's see if I can properly state the nature of the disagreement. I stated that there's a spectrum between "GPT knows more than the average human across a broad variety of domains, but only uses this knowledge to imitate humans, so it's not obvious" and "GPT really knows very little, and its apparent stupidity is stupidity-in-fact". I somewhat operationalized the difference as one of internal representation: to what extent is GPT using a truth+noise model (where it knows a lot of stuff about reality, and then filters it through the biases of particular perspectives) vs a model where everything is thrown together and it's not very possible to extract truth without having more information yourself to know what is truth vs noise. This model has an implication, that Ajeya's project will work to the extent that we're toward the smart-GPT end of the spectrum and won't work to the extent that we're toward the other end. I think you're disagreeing with this implication? So you're saying: even if GPT doesn't internally use anything like a truth+noise model, it's possible to extract a great deal of useful information about the world by observing the statistics of GPT's imitation of internet users. For example, because people talk a lot about diseases online, it should be possible to extract statistics about this from GPT. This can produce a useful diagnostic model, even if GPT isn't internally representing something so useful. Is this roughly what you are saying? If that's what you're saying, then I agree that such a thing could be possible, but I am unsure if this should count as success in Ajeya's terms.  If GPT knows a lot of stuff but isn't telling us because it's not trying to be helpful, that's misalignment. Getting it to try to communicate those things to us would be a kind of alignment work. If the statistics of GPT's text model can be used to infer useful things about the world, this doesn't seem related to alignment. But maybe I'm totally mis-identifying t

This post matches and specifies some intuitions I've had for a while about empirical research and I'm very happy it has been expanded.

Upcoming this comment because it helped me understand why nobody seems to be engaging with what I think the central point of my post is.

After reading some of this reddit thread I think I have a better picture of how people are reacting to these events. I will probably edit or follow up on this post to follow up.

My high level takeaway is:

  • people are afraid to engage in speech that will be interpreted as political, so are saying nothing.
  • nobody is actually making statements about my model of alignment deployment, possibly nobody is even thinking about it.

In the edit or possibly in a separate followup post I will try to present the model at a further disconnect from the specific events and... (read more)

This seems pretty unfortunate insofar as some genuinely relevant real-world details might not survive the obfuscation of premature abstraction. Example of such an empirical consideration (relevant to the "have some members that keep up with AI Safety research" point in your hopeful plan): how much overlap and cultural compatibility is there between AI-ethics-researchers-as-exemplified-by-Timnit-Gebru and AI-safety-researchers-as-exemplified-by-Paul-Christiano? (By all rights, there should be overlap and compatibility, because the skills you need to prevent your credit-score AI from being racist (with respect to whatever the correct technical reduction of racism turns out to be) should be a strict subset of the skills you need to prevent your AGI from destroying all value in the universe (with respect to whatever the correct technical reduction of value turns out to be).) Have you tried asking people to comment privately?

I appreciate the thread as context for a different perspective, but it seems to me that it loses track of verifiable facts partway through (around here), though I don't mean to say it's wrong after that.

I think in terms of implementation of frameworks around AI, it still seems very meaningful to me how influence and responsibility are handled. I don't think that a federal agency specifically would do a good job handling an alignment plan, but I also don't think Yann LeCun setting things up on his own without a dedicated team could handle it.

I would want to see a strong justification before deciding not to discuss something that is directly relevant to the purpose of the site.

Noted that a statement has been made. I don't find it convincing, and even if it did I don't think it changes the effect of the argument.

In particular, even if it was the case that both dismissals were completely justified, I think the chain of logic still holds.

I think this makes sense, but I disagree with it as a factual assessment.

In particular I think "will make mistakes" is actually an example of some combination of inner and outer alignment problems that are exactly the focus of LW-style alignment.

I also tend to think that the failure to make this connection is perhaps the biggest single problem in both ethical AI and AI alignment spaces, and I continue to be confused about why no one else seems to take this perspective.

1Carlos Ramirez3y
Necroing. "This perspective" being smuggling in LW alignment into corps through expanding the fear of the AI "making mistakes" to include our fears?

I am currently writing fiction that features protagonists that are EAs.

This seems at least related to the infrastructure fund goal of presenting EA principles and exposing more people to them.

I think receiving a grant would make me more likely to aggressively pursue options to professionally edit, publish, and publicize the work. That feels kind of selfish and makes me self-conscious, but also wouldn't require a very large grant. It's hard for me to unwrap my feelings about this vs. the actual public good, so I'm asking here first.

Does this sounds like a good grant use?

3Jonas Vollmer3y
I largely agree with Habryka's perspective. I personally (not speaking on behalf of the EA Infrastructure Fund) would be particularly interested in such a grant if you had a track record of successful writing, as this would make it more likely you'd actually reach a large audience. E.g., Eliezer did not just write HPMoR but was a successful blogger on Overcoming Bias and wrote the sequences.

I am reasonably excited about fiction (and am on the Long Term Future Fund). I have written previously about my thoughts on fiction here:

The track record of fiction

In a general sense, I think that fiction has a pretty strong track record of both being successful at conveying important ideas, and being a good attractor of talent and other resources. I also think that good fiction is often necessary to establish shared norms and shared language.

Here are some examples of communities and institutions that I think used fiction very centrally in their function.

... (read more)

Any preliminary results on side effects so far?

Nothing unusual that I've noticed. A bit of congestion a few hours after taking it is expected, but air quality is not great here so I'm mildly congested pretty often anyway.

How were you able to find someone who would give you an antibody test?

I made some effort to get an antibody test a few weeks ago but multiple sources refused to order or run one, even after I had an appointment that I showed up for in person.

Cerascreen offers an at-home antibody test. You use the kit you buy to draw a small blood sample at home and mail it to them. They use the ELISA method to test the blood for IgG antibodies and show you the result on a webpage. Not sure if this is available outside Germany, though maybe a different company offers something like this where you live. Abbott also produces a blood test for IgG/IgM antibodies, except that it comes with a small test cassette that gives you the result directly, without sending it to a lab. Maybe importing this (or something similar) is an option for you.
Huh. I believe we ordered from the labcorp website, didn't have any problems.

Welp, I spent five minutes plus trying to switch to the markdown editor to fix my spoilers and failed. Giving up now.

I would expect the prior to be to end up with something similar to the flu vaccine, which we try to get everyone to take approximately yearly and have more safety concerns about people not taking it.

I find both directions plausible. I do agree that I don't see any existing institutions ready to take it's place, but looking at secular solstice, for example, I definitely expect that better institutions are possible.

There might be a sufficiency stagnation following similar mechanics to crowding out, since people have a "good enough" option they don't try to build better things, and centralized leadership causes institutional conservatism.

I would bet this is supported by worse outcomes for more centralized churches, like unitarians vs megachurches or orthodox catholics, but that's a weakly held belief.

The sufficiency stagnation point is a good one, especially given that is suggests that the people becoming religious on the margin are likely to be the best individuals of the population not currently committed to strong social institutions, to start better ones than religions.  Potentially a crux is that the ideas that really broad social institutions can be based around may mostly be based around certain types of really strong emotions like tribalism and faith, the crux being if 'mostly' means 90%, 99% or 99.999%. 

I think I find this plausible. An alternative to MichaelBowbly's take is that religion may crowd out other community organization efforts which could plausibly be better.

I'm thinking of unions, boys and girls clubs, community centers, active citizenship groups, meetup groups, and other types of groups that have never yet existed.

It could be that in practice introducing people to religious practices shows them examples of ways to organize their communities, but it could also be that religious community efforts are artificially propped up by government subsi... (read more)

I think it's unlikely that there'd be a crowding out effect currently on the margin (although I expect you would as some point if you're attracting progressively less sociable people), as you say because it builds know how, but also because it builds social capital and maybe breaks the negative feedback loop of loneliness.  My second claim is that religion is much much better as a community organising force than any other institution other than unions. I think this is because it can attract a very high percentage of a population, it persists through gennerations, and there aren't the same types of barriers you get with groups organised around a specific interest, and they don't skew middle class (often at least.) 

A toy model that makes some sense to me is that the two population distinction is (close to) literally true; that there's a subset of like 20% of people who have reduced their risk by 95%+, and models should really be considering only the other 80% of the population, which is much more homogeneous.

Then because you started with effectively 20% population immunity, that means R0 is actually substantially higher, and each additional piece of immunity is less significant because of that.

I haven't actually computed anything with this model so I don't know whether it is actually explanatory.

I did some calculations of basic herd immunity thresholds based on fractal risk (without an infection model) a few months back, and the difference between splitting the population into high exposure vs low exposure captures more than half the change from the limit of infinite splits. The threshold stopped changing almost entirely after three splits, which was only 6 subpopulatuons.

With many other variables as exist here I'm not confident that effect would persist but my default guess is that adding fractal effects to the model will less than double the cha... (read more)

One could certainly split into low/high with a larger-than-actually-estimated division and call that close enough, or do something continuous in the middle with the assumption that the super-risky top is already spoken for, or something.  To me there's still a big mystery of why it seems like herd immunity hasn't done more work than it did.

I'd like to use this feature, especially to keep track if I meet a user in the walled garden or IRL but need consistency to remember which user they are. This is a common feature in video games and without it I would have no idea who most of my friends in League of Legends are.

I wouldn't be that worried about privacy for the notes, since I'd expect few of them to contain sensitive information, though they might contain some awkward information.

Yeah I think my main disagreements are 4 and 5.

Given stories I've heard about cryonics orgs, I'd put 10-50% on 5. Given my impression of neuroscience, I'd put 4 at 25-75%.

Given that I'm more pessimistic in general, I'd put an addition 2x penalty on my skepticism of their other guesses.

That puts me around 0.01%-20% spread, or one in ten thousand lower bound, which is better than I expected. If I was convinced that a cryo org was actually a responsible business that would be enough for me to try to make it happen.

Yes - it's hard to perform the calculations and end up with a high probability that cryonics works. I think cryonics overall is much less feasible than many Less Wrongers tend to assume. Overall, I think anti-aging has a much higher chance of working to keep us alive much longer than cryonics does. 

Even 0.2% seems quite optimistic to me. Without going into detail, anything from 3-8 seems like it could be 10% or lower and 12-14 seem nearly impossible to estimate. I wouldn't be surprised to find my personal estimate below one in a million.

Yeah. For your interest, here are the calculations from Alcor:  Steve Harris, MD: 1) Materialism is correct: 0.95-0.99 2) Identity encoded in structure: 0.95-0.99 3) Favorable conditions for suspension: 0.75-0.95 4) Suspension preserves enough information: 0.50-0.90 5) Mishap-free storage: 0.95-0.99 6) Cryonics organization survives: 0.20-0.60 7) Sufficient social stability: 0.70-0.90 8) Cryonics is continuously legal: 0.70-0.90 9) Nanotechnology is physically possible: 0.90-0.98 10) Nanotechnology is perfected: 0.95-0.98 11) Nanotechnology is non-catastrophic: 0.20-0.50 12) Cryonic revival is "cheap enough": 0.85-0.95 13) Cryonic revival is permitted: 0.50-0.80 The social problem is non-catastrophic: 0.008-0.18 Technologically, will cryonics work? 0.29-0.81 Overall, will it work? 0.002-0.15  That is, a 0.2-15% probability that cyronics works overall.  Mike Perry, PhD: Note: his calculation lumps 7 of the 13 parameters as 'the social problem' which he calls condition n.  1) Materialism is correct 1.00-1.00 2) Identity encoded in structure:  1.00-1.00 3) Favorable conditions for suspension: 0.75-0.95 4) Suspension preserves enough information: 0.50-0.90 5) Mishap-free storage: 0.90-0.99 6) Cryonics organization survives: n-n 7) Sufficient social stability n-n 8) Cryonics is continuously legal n-n 9) Nanotechnology is physically possible 1-1 10) Nanotechnology is perfected n-n 11) Nanotechnology is non-catastrophic n-n 12) Cryonic revival is "cheap enough" n-n 13) Cryonic revival is permitted n-n The social problem is non-catastrophic: 0.39-0.86 Technologically, will it work? 0.34-0.89 Overall, will it work? 0.13-0.77 That is, a 13-77% probability that cyronics works overall.   

I was trying to do a back of the envelope calculations of total cost of work and total value created (where I'm using cost of rent as a (bad) proxy for (capturable) value created).

I definitely wouldn't assume that the government or any single agent would be doing the project, just that the overall amount of capturable value must be worth it for the investment costs, then different parties can pay portions of those costs in exchange for portions of or rights to that value, but I doubt adding in the different parties involved would make my estimates more accurate.

Do you have a source for cost of similar projects? My estimates are definitely very bad for many reasons.

I want to have this post in a physical book so that I can easily reference it.

It might actually work better as a standalone pamphlet, though. 

I like that this responds to a conflict between two of Eliezer's posts that are far apart in time. That seems like a strong indicator that it's actually building on something.

Either "just say the truth", or "just say whatever you feel you're expected to say" are both likely better strategies.

I find this believable but not obvious. For example, if the pressure on you is you'll be executed for saying the truth, saying nothing is probably better that saying the truth. If the pressure on you is remembering being bullied on tumblr, and you're being asked if you... (read more)

The factual point that moderate liberals are more censorious is easy to lose track of, and I saw confusion about it today that sent me back to this article.

I appreciate that this post starts from a study, and outlines not just the headline from the study but the sample size. I might appreciate more details on the numbers, such as how big the error bars are, especially for subgroups stats.

Historical context links are good, and I confirm that they state what they claim to state.

Renee DiResta is no longer at New Knowledge, though her previous work there is st... (read more)

it's not a simple enough question for easy answers. 

It's also plausible to me that it requires enough intersections (owns a house; rents the house out on AirBnB; in a single metro area; measures success in a reasonable way; writes about it on the internet) gets small enough that there are no results.

Looking for general advice (how to succeed as an AirBNB host) might give a model that's easy to fill in, like "you will succeed if the location is X appealing and there are f(X) listings or fewer."

That still seems like a pretty easy answer to me, but it co... (read more)

I think you're misunderstanding my analogy.

I'm not trying to claim that if you can solve the (much harder and more general problem of) AGI alignment, then it should be able to solve the (simpler specific case of) corporate incentives.

It's true that many AGI architectures have no clear analogy to corporations, and if you are using something like a satisficer model with no black-box subagents, this isn't going to be a useful lens.

But many practical AI schema have black-box submodules, and some formulations like mesa-optimization or supervised amplification-d... (read more)

"Principal-agent problem" seems like a relevant keyword. Also, how does nature solve this problem? How are genes aligned with the cell as a whole, cells with the multicellular organism, ants with the anthill? Though I suspect that most (all?) solutions would be ethically and legally unacceptable for humans. They would translate as "if the company fails, all employees are executed" and similar.

As remizidae points out, most of these restrictions are not effectively enforced by governments, they are enforced by individuals and social groups. In California, certainly, the restaurants and bars thing is enforced mostly by the government, but that's mostly a "governments can't act with nuance" problem.

But for things like gatherings of friends, I think this question still applies. The government cannot effectively enforce limits on that, but your group of friends certainly can.

And I think in that context, this question remains. That is, I think groups ... (read more)

I think this misunderstands my purpose a little bit.

My point isn't that we should try to solve the problem of how to run a business smoothly. My point is that if you have a plan to create alignment in AI of some kind, it is probably valuable to ask how that plan would work if you applied it to a corporation.

Creating a CPU that doesn't lie about addition is easy, but most ML algorithms will make mistakes outside of their training distribution, and thinking of ML subcomponents as human employees is an intuition pump for how or whether your alignment plan might interact.

2Donald Hobson3y
Modelling an AI as a group of humans is just asking for an anthropomorphized and probably wrong answer. The human brain easily anthropomorphizes by default, thats a force you have to actively work against, not encourage.  Humans have failure modes like getting bored of doing the same thing over and over again, and stopping paying attention. AI's can overfit the training data and produce useless predictions in practice.  Another way of seeing this is to consider two different AI designs, maybe systems with 2 different nonlinearity functions, or network sizes or whatever. These two algorithms will often do different things. If the algorithms get "approximated" into the same arrangement of humans, the human based prediction must be wrong for at least one of the algorithms. The exception for this is approaches like IDA, which use AI's trained to imitate humans, so will probably actually be quite human like. Take an example of an aligned AI system, and describe what the corresponding arrangement of humans would even be. Say take a satificer agent with an impact penalty. This is an agent that gets 1 reward if the reward button is pressed at least once, and is penalised in proportion to the difference between the real world and the hypothetical where it did nothing.How many people does this AI correspond to, and how are the people arranged into a coorporation?

I like this post and would like to see it curated, conditional on the idea actually being good. There are a few places where I'd want more details about the world before knowing if this was true.

  • Who owns this land? I'm guessing this is part of the Guadalupe Watershed, though I'm not sure how I'd confirm that.

This watershed is owned and managed by the Santa Clara Valley Water District.

  • What legal limits are there on use of the land? Wikipedia notes:

The bay was designated a Ramsar Wetland of International Importance on February 2, 2012.

I don't know what that ... (read more)

It looks to me like you're comparing the cost of constructing a unit to the price of renting a bedroom? I don't think you're counting the cost of infrastructure construction? Bulk fill is massively cheaper than the quantities you would use for pool removal. Since this is a seismic area you can't just use bulk fill: you need to do some amount of reinforcement/stabilization. You're talking about this as if the government would pay for all construction and then own all units and rent them out? I agree that governments are not typically interested in that kind of financing, especially in the US. If you sell the units, or the buildings, however, there is a thriving real estate market that does operate on these time scales.

It looks like for pool removal there's a cost of between $20-$130 per cubic foot.

Minor correction: that source says filling in a pool is $20-80 per cubic yard, which would only be ~$1-3 per cubic foot. The higher numbers are for demolition, but that's presumably dominated by the cost of the demolition rather than the fill - jackhammers are a pain in the ass.

Load More