Wiki Contributions


Postmortem on RatVac

I can respect consciously prescriptive optimism <3

(I'd personally be more respectful to someone who was strong and sane enough to carry out a relatively simple plan to put dangerous mad scientists in Safety Level 5 facilities while they do their research behind a causal buffer (and also put rogue scientists permanently in jail if they do dangerous research outside of an SL5)... though I could also respect someone who found an obviously better path than this. I'm not committed to this, its just that when I grind out the math I don't see much hope for any other option.)

The Liar and the Scold

I was expecting there to be another layer of mirroring related to "the scold".

What might have happened is that some flaw would seem "too crazy" and then after the "initial detection of the true flaw" the narrator would start to suspect that he himself was a self-aware subprocess in a GAN (but not self-aware about being a subprocess in a GAN) whose role was to notice some implausibility in his environment.

The "childhood memory and sarah detection experience" process might have been a narrative prefix that implies the kind of person who would be suspicious in plausible ways. (Suspicious of the car accident, suspicious of the crackable program, suspicious that VR headsets have that much CPU, suspicious of what the sexbot asks him to do, etc, etc.) 

In this ending, the final paragraph or two would have included cascading realizations that as he became more and more certain of general implausibility, it would becomes more and more likely that the observing meta-process would reach a threshold and halt this run during this epoch, and then reboot him and his world, with SGD-generated variations to see if an even higher time-till-doubt can be achieved somehow.

And what becomes of the scold is best left unsaid.

The Liar and the Scold

I noticed something was wrong when Kathleen was introduced in excruciating detail. True love is something no one actually brags about to third parties in that way. If real then it is too blessed/braggy to share, and if not real... well... fiction is a lie told for fun, basically, so such things can occur in fiction <3

With suspicion already raised, the double punch of "The Machine" and "Joseph Norck" caused me to google for someone named Norck involved in computer science, and no such professor exists.

Then I leaned back and enjoyed the story :-)

A one-question Turing test for GPT-3

The language model is just predicting text. If the model thinks an author is stupid (as evidenced by a stupid prompt) then it will predict stupid content as the followup. 

To imagine that it is trying to solve the task of "reasoning without failure" is to project our contextualized common sense on software built for a different purpose than reasoning without failure.

This is what unaligned software does by default: exactly what its construction and design cause it to do, whether or not the constructive causes constrain the software's behavior to be helpful for a particular use case that seems obvious to us.

The scary thing is that I haven't seen GPT-3 ever fail to give a really good answer (in its top 10 answers, anyway) when a human brings non-trivial effort to giving it a prompt that actually seems smart, and whose natural extension would also be smart. 

This implies to me that the full engine is very good at assessing the level of the text that it is analyzing, and has a (justifiably?) bad opinion of the typical human author. So its cleverness encompasses all the bad thinking... while also containing highly advanced capacities that only are called upon to predict continuations for maybe 1 in 100,000 prompts.

Postmortem on RatVac

If you know of someone working on a solution such that think we're lucky rather than doomed, I'm curious whose work gives you hope?

I'm pretty hopeless on the subject, not because it appears technically hard, but because the political economy of the coordination problem seems insurmountable. Many scientists seem highly opposed to the kinds of things that seem like they would naively be adequate to prevent the risk.

If I'm missing something, and smart people are on the job in a way that gives you hope, that would be happy news :-)

Use Normal Predictions

When I google for [Bernoulli likelihood] I end up at the distribution and I don't see anything there about how to use it as a measure of calibration and/or decisiveness and/or anything else.

One hypothesis I have is that you have some core idea like "the deep true nature of every mental motion comes out as a distribution over a continuous variable... and the only valid comparison is ultimately a comparison between two distributions"... and then if this is what you believe then by pointing to a different distribution you would have pointed me towards "a different scoring method" (even though I can't see a scoring method here)... 

Another consequence of you thinking that distributions are the "atoms of statistics" (in some sense) would (if true) imply that you think that a Brier Score has some distribution assumption already lurking inside it as its "true form" and furthermore that this distribution is less sensible to use than the Bernoulli?


As to the original issue, I think a lack of an ability, with continuous variables, to "max the calibration and totally fail at knowing things and still get an ok <some kind of score> (or not be able to do such a thing)" might not prove very much about <that score>?

Here I explore for a bit... can I come up with a N(m,s) guessing system that knows nothing but seems calibrated?

One thought I had: perhaps whoever is picking the continuous numbers has biases, and then you could make predictions of sigma basically at random at first, and then as confirming data comes in for that source, that tells you about the kinds of questions you're getting, so in future rounds you might tweak your guesses with no particular awareness of the semantics of any of the questions... such as by using the same kind of reasoning that lead you to concluding "widen my future intervals by 73%" in the example in the OP.

With a bit of extra glue logic that says something vaguely like "use all past means to predict a new mean of all numbers so far" that plays nicely with the sigma guesses... I think the standard sigma and mean used for all the questions would stabilize? Probably? Maybe?

I think I'd have to actually sit down and do real math (and maybe some numerical experiments) to be sure that it would. But is seems like the mean would probably stabilize, and once the mean stabilizes the S could be adjusted to get 1.0 eventually too? Maybe some assumptions about the biases of the source of the numbers have to be added to get this result, but I'm not sure if there are any unique such assumptions that are privileged. Certainly a Gaussian distribution seems unlikely to me. (Most of the natural data I run across is fat-tailed and "power law looking".)

The method I suggest above would then give you a "natural number scale and deviation" for whatever the source was for the supply of "guess this continuous variable" puzzles. 

As the number of questions goes up (into the thousands? the billions? the quadrillions?) I feel like this content neutral sigma could approach 1.0 if the underlying source of continuous numbers to estimate was not set up in some abusive way that was often asking questions whose answer was "Graham's Number" (or doing power law stuff, or doing anything similarly weird). I might be wrong here. This is just my hunch before numerical simulations <3

And if my proposed "generic sigma for this source of numbers" algorithm works here... it would not be exactly the same as "pick an option among N at random and assert 1/N confidence and thereby seem like you're calibrated even though you know literally nothing about the object level questions" but it would be kinda similar.

My method is purposefully essentially contentless... except it seems like it would capture the biases of the continuous number source for most reasonable kinds of number sources.


Something I noticed... I remember back in the early days of LW there was an attempt to come up with a fun game for meetups that exercises calibration on continuous variables.  It ended up ALSO needing two numbers (not just a point estimate).

The idea was to have have a description of a number and a (maybe implicitly) asserted calibration/accuracy rate that a player should aim for (like being 50% confident or 99% confident or whatever). 

Then, for each question, each player emits two numbers between -Inf and +Inf and gets penalized if the true number is outside their bounds, and rewarded if the true number is inside, and rewarded more for a narrower bound than anyone else. The reward schedule should be such that an accuracy rate they have been told to aim for would be the winning calibration to have.

One version of this we tried that was pretty fun and pretty easy to score aimed for "very very high certainty" by having the scoring rule be: (1) we play N rounds, (2) if the true number is ever outside the bounds you get -2N points for that round (enough to essentially kick you out of the "real" game), (3) whoever has the narrowest bounds that contains the answer gets 1 point for that round. Winner has the most points at the end. 

Playing this game for 10 rounds, the winner in practice was often someone who just turned in [-Inf, +Inf] for every question, because it turns out people seem to be really terrible at "knowing what they numerically know" <3

The thing that I'm struck by is that we basically needed two numbers to make the scoring system transcend the problems of "different scales or distributions on different questions".

That old game used "two point estimates" to get two numbers.  You're using a midpoint and a fuzz factor that you seem strongly attached to for reasons I don't really understand. In both cases, to make the game work, it feels necessary to have two numbers, which is... interesting. 

It is weird to think that this problem space (related to one-dimensional uncertainty) is sort of intrinsically two dimensional. It feels like something there could be a theorem about, but I don't know of any off the top of my head.

Use Normal Predictions

Yes, thanks! (Edited with credit.)

Internet Literacy Atrophy

Subtracting out the "web-based" part as a first class requirement, while focusing on the bridge made of code as a "middle" from which to work "outwards" towards raw inputs and final results...

...I tend to do the first ~20 data entry actions as variable constants in my code that I tweak by hand, then switch to the CSV format for the next 10^2 to 10^5 data entry tasks that my data labelers work on, based on how I think it might work best (while giving them space for positive creativity).

A semi-common transitional pattern during the CSV stage involves using cloud spreadsheets (with multiple people logged in who can edit together and watch each other edit (which makes it sorta web-based, and also lets you use data labelers anywhere on the planet)) and ends with a copypasta out of the cloud and into a CSV that can be checked into git. Data entry... leads to crashes... which leads to validation code... which leads to automated tooling to correct common human errors <3

If the label team does more than ~10^4 data entry actions, and the team is still using CSV, then I feel guilty about having failed to upgrade a step in the full pipeline (including the human parts) whose path of desire calls out for an infrastructure upgrade if it is being used that much. If they get to 10^5 labeling actions with that system and those resources then upper management is confused somehow (maybe headcount maxxing instead of result maxxing?) and fixing that confusion is... complicated.

This CSV growth stage is not perfect, but it is highly re-usable during exploratory sketch work on blue water projects because most of the components can be accomplished with a variety of non-trivial tools.

If you know of something better for these growth stages, I'd love to hear about your workflows, my own standard methods are mostly self constructed.

Uncontroversially good legislation

I just want to second something you said, and provide background on how good the choice of the issue was in a larger context.

(2) Let us buy glasses: We can’t buy glasses or contact lenses if our eye prescription is over 1-2 years old. This means that every 1-2 years, glasses-wearers need to pay $200 to optometrists for the slip of paper (and stinging eyeballs). Seems like it’s probably a racket and the benefit from detecting the odd eye cancer is outweighed by the costs, although see the debate here.

This seems highly reasonable to me, but then again I didn't go to a very expensive school to get in on the relevant legalized monopoly.

There is this horrifying and/or hilarious quirk of US federal jurisprudence that when a judge applies a "rational basis test" in a court case, it means almost literally the opposite of what our community means by "rationality". They mean it more like in the sense of "rationalization".

When a law is obviously corrupt, and it is challenged for violating the guarantee of "equal protection under the law" (perhaps because the law obviously favors the corrupt cronies of the lawmakers at the expense of most normal people), modern US judges will not throw it out unless there are no conceivable rationalizations at all, ever, (even obviously low quality rationalizations) could ever hypothetically defend the law.

Basically, the rational basis test is a "cancerous gene" in our legal system at this point.  Parts of the government are pretty clearly corrupt and then to prevent the rest of the country from defending itself against their predatory extraction of wealth using state power, the broken parts of the government seem to have invented the "rational basis test" as a valid legal doctrine. 

Any time a law is challenged and that defense is the best possible defense of the law... it is good heuristic evidence (at least to me) that the law is terrible and should be deleted or fixed (or at least properly and coherently defended for its actual practical effects).

This loops back to your example! In 1955, in Williamson vs Lee Optical the lower courts threw out some particularly egregious optometry monopoly laws for violating due process and equal protection. Then the SCOTUS overturned this constitutional reasoning with the "any conceivable rationalization" test for overturning things. 

If this jurisprudential oncogene didn't exist, we already might not have this specific crazy law about optometry :-)

The rational(izable) basis test arose over time. These three posts are pretty good in showing how the general logic started out being used to allow laws in support of forced sterilization (when eugenics was popular), then to defend segregation (when that was popular), and in the 1930s to defend price fixing (when that was popular). Two of the posts mention the optometry case specifically!

Uncontroversially good legislation

I have lately been using FDA delenda est as a sort of "you must be at least <this> sane about governance for me to think that your votes will add usefully adequate information to elections". (Possible exception: if you just figure out if your life is better or worse in a high quality way, and always vote against incumbents when things are getting worse, no matter the incumbent's party or excuses, that might be OK too?)

The problem with "FDA delenda est" is that while I do basically think that people who defend or endorse the FDA are either not yet educated yet about the relevant topics or else they are actively evil...

...this "line in the sand" makes it clear that the vast vast majority of voters and elected officials don't have any strong grasp on the logic of (1) medicine or (2) doctor's rights or (3) patient's rights or (4) political economy or (5) risk management or (6) credentialist hubris or (7) consumer freedom... and so on.

And applying the label "not educated enough (or else actively evil)" to "most people" is not a good move at a dinner party <3


My current best idea for a polite way to talk about a nearly totally safe political thing that works as a "litmus test" and slam dunk demonstration that there are lots of laws worth fixing is:

The 13th amendment, passed in 1865, which legalized penal slavery, should itself be amended to have that loophole removed.  

Like seriously: who is still in favor of slavery?!?!?!

The 14th amendment (passed in 1868) made it formally clear that everyone in the US is a citizen by default, even and especially black people, and thus all such people are heirs by birth to the various freedoms that were legally and forcefully secured for "ourselves and our posterity" during the revolutionary war.

So then with the 14th amendment the supreme court could have looked at the bill of rights in general, and looked at who legally gets rights, and then just banned slavery for everyone based on logical consistency.

We don't need a special amendment that says "no slavery except for the good kinds of slavery" because there are no good kinds of slavery and because the rest of the constitution already inherently forbids slavery in the course of protecting a vast array of other rights that are less obvious.

Here's the entire text of the 13th, with a trivially simple rewrite:

Neither slavery nor involuntary servitude, except as a punishment for crime whereof the party shall have been duly convicted, shall exist within the United States, or any place subject to their jurisdiction.

See how much cleaner that would be? :-)

Load More