The Open Agency Architecture ("OAA") is an AI alignment proposal by (among others) @davidad and @Eric Drexler. .. (read more)
Open Threads are informal discussion areas, where users are welcome to post comments that didn't quite feel big enough to warrant a top-level post, nor fit in other posts... (read more)
AI Risk Skepticism is the view that the potential risks posed by artificial intelligence (AI) are overstated or misunderstood, specifically regarding the direct, tangible dangers posed by the behavior of AI systems themselves. Skeptics of object-level AI risk argue that fears of highly autonomous, superintelligent AI leading to catastrophic outcomes are premature or unlikely.
Slowing Down AI refers to efforts and proposals aimed at reducing the pace of artificial intelligence advancement to allow more time for safety research and governance frameworks. These initiatives can include voluntary industry commitments, regulatory measures, or coordinated pauses in development of advanced AI systems.
Aligned AI Proposals are proposals aimed at ensuring artificial intelligence systems behave in accordance with human intentions (intent alignment) or human values (value alignment)... (read more)
Reinforcement Learning from Human Feedback (RLHF) is a machine learning technique where the model's training signal uses human evaluations of the model's outputs, rather than labeled data or a ground truth reward signal.
| User | Post Title | Wikitag | Pow | When | Vote |
Thanks, edited that section with a link.
we generally recommend using this Google Docs add-on to convert your Google Docs drafts (formatted as you like) into Markdown syntax.
btw google docs supports this natively now! Select some text, press Option+/ and search for "Copy as Markdown".
"The X-Men comics use terms like “evolution,” “mutation,” and “genetic code,” purely to place themselves in what they conceive to be the literary genre of science. The part that scares me is wondering how many people, especially in the media, understand science only as a literary genre." -- from Eliezer's post Science as Attire.
As far as I understand, * means something that something that one would want, agree, decide, etc. under ideal reflection conditions (e.g. knowing most plausibly relevant arguments, given a long time to think, etc) See, e.g. the CEV as defined not in relation to alignment targets or Wei Dai's metaethical alternatives 3-5.
Nit: What do the "*" mean? I find them slightly distracting.
"Noticing confusion" means noticing the "tiny note of discord" (the quiet sense of "something is off") that our minds produce when what we're seeing differs from what we would have expected to see.
There is much rationality skill to be found in learning to raise this subtle signal to full consciousness, and to pay attention to it.
Sequences on Shard Theory:
We also offer a Markdown editor. If you want to use Markdown, here is a guide that discusses Markdown syntax, and we generally recommend usingyou can also this Google Docs add-on to convert your Google Docs drafts (formatted as you like) into Markdown syntax.Markdown.
Related tags and wikis: Disagreement, Modesty, Modesty argument, Aumann agreement, The Aumann Game, ⟨M,N,ε,δ⟩-agreement
"The X-Men comics use terms like “evolution,” “mutation,” and “genetic code,” purely to place themselves in what they conceive to be the literary genre of science. The part that scares me is wondering how many people, especially in the media, understand science only as a literary genre." -- from Eliezer's post Science as Attire.
What does this fractional portion mean? Roughly speaking, logarithms measure not only how long a number is, but also how much that number is really using its digits. 12 and 9997 are both two-digit numbers, but intuitively, 12 is "barely" two digits long, whereas 97 is "nearly" three digits. Logarithms formalize this intuition, and tell us that 12 is really only using about 1.08 digits, while 97 is using about 1.99.
Question: log10(100)=2, but clearly, 100 is 3 digits long. In fact, logb(bk)=k for any integers b and k, but k+1 digits are required to represent bk in base b (as(as a one followed by k zeroes). Why is the logarithm making these off-by-one errors?
You may be wondering, are two 5-digits really worth more than one 10-digit? Sure, you can place them in 25 different configurations, but how do you encode "9" when none of the digits have a "9" symbol written on them? If so, see The symbols don't matter.
Are we sure that the gene doesn't just cause people who have it to believe in Functional Decision Theory?