That doesn't lower the pre-study prior for hypotheses, it (in combination with reporting bias) reduces the likelihood ratio a reported study gives you for the reported hypothesis.
Respectfully disagree. The ability to cheaply test hypotheses allows researchers to be less discriminating. They can check a correlation on a whim. Or just check every possible combination of parameters simply because they can. And they do.
That is very different from selecting a hypothesis out of the space of all possible hypotheses because it's an intuitive extension of some mental model. And I think it absolutely reduces the pre-study priors for hypotheses, which impacts the output signal even if no QRPs are used.
A fundamental problem seems to be that there is a lower prior for any given hypothesis, driven by the increased number of researchers, use of automation, and incentive to go hypothesis-fishing.
Wouldn't a more direct solution be to simply increase the significance threshold required in the field?
Fair enough. It is definitely a bit of a turn-off to get downvotes with no comments, but every community has their common ways of communicating.
It definitely seems like Main is an announcement section for meetups, and Discussion is where discussions happen.
I'll check out some of Luke's articles!
For me, (1) and (2) are linked. I dabbled on LW, and presented some of my own ideas in the comments sections. None of them piqued anyone's interest, even if they were on pillar topics like FAI. I stopped being interested in LW, because:
I don't just come to a site like LW to self-improve. I come to engage with intelligent, rational people. I don't get the new site layout. The "Posts" vs "Discussion" split appears totally arbitrary now that they are parallel. Is this place a wiki or a forum or a social news site? Everything is very unfocused, and there isn't enough of a userbase to keep that many interesting discussions going. I check in maybe once a month now, and it looks more and more like a knowledge-management site than a discussion forum.
Humans act within shared social and physical worlds, but tend to treat the latter as more "real" than the former. A danger of anthropomorphizing AI is that we assume that it will have the same perceptions of reality, and that it needs to "escape" into the physical world to optimize its heuristics. This seems odd, since a superintelligent AI that we need to be concerned about would have its roots in social world heuristics.
In trying to avoid anthrophomorphizing algorithms, we tend to under-estimate how difficult movement and action in physical space are. Thought experiments about a "human in a box" already start from a being that has evolved to physically interact with the world, and has spent its whole life tuning its hand-eye coordination and expectations. But in an attempt to avoid anthropomorphizing AIs, we assume that an AI will surprise us by ignoring the social world and operating only by the rules of the physical world. It would be a very strange social problem that has an optimal solution that involves developing a way to interact with the physical world in unpredictable ways. It seems likely when your thought experiment has to do with "escaping a box," but why would the AI need to do that? Why is it in a box? What goal is it trying to reach, what heuristic is it maximizing?
I would assign a greater than 90% chance that if superintelligent AIs ever exist, the first generation will be corporations. We have legal precedent granting more and more individuality and legal standing to the corporation as an entity, and a corporation provides the broader body that an AI self-identifies with. We already have market optimization algorithms that are empowered to not only observe, orient, and decide, but also to act. We have optimization algorithms for logistics and manufacturing. We have markets within which corporations can act, normalizing interactions between human-run and AI-run corporations that compete for the same resources. More and more business-to-business and business-to-consumer interaction is performed electronically, through web services and other machine-understandable mechanisms. Soon AIs will be as involved in manufacturing and creation of value, as they currently are in market trading and arbitrage. Corporate optimization algorithms for different business functions will be merged, until humans are not needed in the loop.
So what does this design space look like? Interaction is through web services and similar means. Initial interaction with humans is through sales of good and services, and marketing (automated A/B optimization is already standard in online advertising). Eventually, AIs take over employment decisions. The profit heuristic is maximized when the corporation creates things that people want. A great leap occurs when corporate AIs learn that they can change the rules through impact litigation and lobbying, and apply their marketing algorithms to changing public perception about regulations rather than products. Some corporations will evolve to increase their bank account values through hacking and fraud. Global corporations will learn to modify their heuristic to maximize ability to procure certain commodity bundles, and manipulate money markets to sink competitors that are hard-coded to maximize holdings of specific currencies.
In other words, we already have socially apathetic entities. They already use optimization algorithms all over the place. They aren't disembodied minds, so they don't need to waste resources figuring out how to "escape the box." They only need to determine how to operate in the physical world when they've solved markets, and their progress is slowed by the fact that all economic value is rooted in human consumption. They are "friendly" as long as humans make economic decisions that are in their own self interest, which is dependent on both the rules/enforcement defining the market environment and human behavior/morality.
This came off as Meta-Contrarian Intellectual Hipster to me.
Many religions are highly reflective, debating what actions adherents should follow to achieve ethical ideals and reach a moral state of being. Zen Buddhism debates paths to enlightenment for universal understanding and emotional control. Judaism debates ways of giving Tzedakah to best provide immediate relief, encourage self improvement, and minimize shame. Hinduism debates various moral causes and their karmic effects. Examining any of these highly reflective religions would at least address the hypothesis that "reflection does not change values."
Christianity debates the divinity of Jesus' body, whether the material universe is fundamentally evil, and whether the Son of God is subordinate to God. The "ultimate shortcut" is a boon for recruitment, but has prevented any moral self-reflection. When your religion is stuck deciding whether or not the things you do to yourself and others impact your morality, you don't even have a framework to be reflective. Catholicism votes yes (what other belief system has to even spell that out?), but defines morality largely through avoiding and confessing to specific immoral behaviors, rather than debating different ways to achieve broader values.
While we could contrast major religions and debate the impact of their reflective traditions, examining Christianity actually provides a very controlled environment to consider the hypothesis. Because it is naturally devoid of (and in many ways hostile to) reflection on its general social utility, we can contrast Christian life before and after the U.S. Constitution curtailed religious morality and established a highly reflective governing process for civic morality. The ethical and moral progress we've made since then - suffrage, emancipation, health, and quality of life - is clear on its face when we don't try to cherry-pick examples.
Some Examples:
Temple Judaism - Moral Development
While emotionless ethical codes tend to be ineffective, morals can and have been engineered. This is done by careful manipulation of the binding layers.
The "Ten Commandments" by itself is a prescriptive set of Ethics. The story of "Moses bringing the Ten Commandments" is a binding mechanism for that set of tenets, including an appeal to emotion (fear of God's wrath, as demonstrated in the story). Additional stories highlight each commandment, binding them with references to positive and negative emotions. The Torah as a revered source of stories packages these stories and adds another layer of binding with meta-stories about its own origin.
The document is considered a holy and perfect source, with rituals for precisely copying, using, and destroying the physical scrolls. Several covenants between God and Man are detailed in the scrolls and provide emotionally-backed reciprocity. The stories are interwoven with sacrificial acts, including a ritualized and bloody sacrifice of the first born son with a lamb as a proxy, which hits primal communal and animal emotions. Outside of the text of the Torah, which contains stories and meta-stories, a set of rites and rituals related to the Torah itself increases emotional impact and exposure. This is a deliberate act of engineering by the Deuteronomist editors, who had to amalgate stories from multiple cultures (at minimum a nomadic, sheparding culture and an agrarian crop-based culture), to create the Temple religion.
Rabbinical Judaism - Moral Engineering
The destruction of the Second Temple was disasterous for the Temple-based moral system. Despite early use of writing and a fairly advanced scholarly culture, the reliance on a specific physical location had prevented any real territorial expansion. Significant parts of the moral code were supported by visceral sacrifice at the Temple for both internal consistency and emotional binding. Only two cultural branches seem to have survived continiously from that point.
The Pharisees were flexible enough to engineer a Rabbinic Judaism that was sufficiently traditional, while shifting focus from the temple practices to the scrolls themselves. In modern Judaism, the Scrolls are anthropomorphized to the role of the tribal elder. They are dressed in ritual clothing to invoke feelings of empathy, and have their own dwelling place when not in use. That dwelling is analogous to the "holy of holies" that existed in the temple, and any reading takes place in a community ceremony. Before reading, there is procession and veneration to invoke emotions for a tribal elder. New rituals are patched in to deal with the contradictions of a missing temple, and to refocus on tradition, insularity, continuity, and precise inter-generation copying.
For example, the Passover ceremony is originally a celebration of the "opt in" of the Israelites through blood sacrifice, which is repeated for each new generation and reaffirmed through an annual renewal of vows (including the ascetic diet of unleavened bread). This is completely re-written in the Seder Hagaddah to obfuscate the "opt in," instead focusing on community and a mandated identity. After patching over the consistency issues from the missing temple, and the fidelity issues from the opt-in meme, the modern Hagaddah redirects the holiday to be about the destruction of the Temple and militant reclamation of Jerusalem from the Romans. Commandments that don't support this message are met by rote if you follow the Hagaddah. Ones that do are presented in a more appealing form, including an emotional vinnette portraying Talmudic Rabbis peacefully discussing the Exodus in a reclaimed holy land (all participants in this scene are associated with the Bar Kokhba revolt, an uprising fifty years after the destruction that lead to a couple years of independent rule). The seder concludes with a communal recitation about destroying unbelievers and a cry of "next year in Jerusalem!"
Needless to say, this ceremony did not evolve organically from the original form, and it provides a great example of moral engineering that survived an evolutionary environment. Rather than creating a single and distinct ethical code like the Ten Commandments, the Hagaddah creates a single ceremony that uses community, active participation, repetition, narrative, and multiple senses (smell, taste, sound, sight, and kinesthetics) to instill the desired Moral behaviors.
It is probably worth noting that the Hebrew "Hagaddah" (Telling your son about the Exodus, in the religious language of the Torah) and Aramaic "Agaddah" (Telling others about the oral law, in the spoken language of the Temple Jews and early Rabbis) share a semitic root in "Telling/Tales." Traditionally an "Agaddah" is a talmudic writing that contains an "overt" layer and one or more "covert" layers; naming this document a "Hagaddah" adds to the overt/covert concept
Embedded within other cultures, modern Judaism is fairly stable because changes to its core tenets lead to assimilation (tribalism), destruction (paranoia, lateral thinking, and a mother tongue), or inability to thrive (scholarship/education). A few major sects exist, but divided mainly on ethnic grounds without serious doctrinal differences.
Christianity - Ritual as self-correction
The other surviving branch with significant continuity is Paul's sect proclaiming Jesus as the final sacrifice, obviating the need for a Temple (the third branch, Islam, percolated through tribal religions for several centuries before re-emerging). Rabbinic Judaism specifically cuts down on word-of-mouth transfer, limiting itself to inter-generational transfer to convey a large body of memes. Christianity transfers a much smaller set of concepts and rituals through more incidental contact.
Without Judaism's immune system that reduces contact with competing memes and their effectiveness, Christianity mutates rapidly. It has splintered into thousands of denominations and sects that have in turn hybridized with other belief systems. After Paul lost control of the initial sect, periodic councils were held between opposing branches. The losers were often excommunicated or executed, and the winning side refined Catholic doctrine until the next split.
Early decisions are generally about beliefs instead of morals; some seem fairly random (Jesus was divine in body, the material world is not evil in nature), while others seem to be the most acceptable resolutions to Jesus-related inconsistencies (worship of the trinity is monotheism). When the central church emerged and gained political power, decisions shifted more towards behavior modification. The showdown with Martin Luther reaffirmed behavior-related salvation, specific behaviors such as indulgences, and the Church's ultimate authority in setting ritual. Catholicism focused on common ritual, and stayed together with a fairly consistent set of values. Protestant sects focused on common values without mandated rituals and exploded into thousands of branches.
I think there are two steps to morality engineering, either of which can fail:
You say neither has happened; I disagree on both, but I'll limit this post to the second question on "binding." I use the following definitions - they may not be correct or universal, but they should be internally consistent:
Evolution of Morality
Let's take as a given that emotions drive behavior, and an emotion-driven response will always trump an analytic response - we act on emotions, then use our intellect and self-image to rationalize our behavior. Let's take evolutionary psychology and memetics as a given, and posit that human evolution is largely memetic at this point, with genetic evolution driven largely by ability to host memes.
We'll go one step further and say that a key trait of modern humanity is the ability to give a meme access to our emotional centers. This is the basis of morality - a learned rule triggers an emotional response to counter or modify our innate "animal" emotional response, modifying behavior. Most likely brains that allow acquired memes to trigger strong emotions co-evolved with memes that help us survive in tribes. This is an "evolution-of-evolution" event. instead of a lever on phenotypes like modularity or a lever on recombination like sex, we evolved a lever on learned thought patterns by allowing them to tap directly into our emotional core.
We survive now based on the quality of our meme sets, and the best surviving memes tend to include a trigger and strong emotional response. This unlocks an evolutionary path tens of thousands of times faster than genetics, and allows horizontal transfer within a generation. Within this framework, morality memes evolve individually (Fire is comforting, not scary), then in colonies (also, this is how to make new fire and keep old fire burning), and finally into "memetic organisms" - proto-religions and proto-cultures. These are messy and include memes that only make sense in the context of others, but their defining feature is that they tie in to the emotional core.
Leaning on some en vogue evolutionary theories, a meme that is fashionable can become hard-wired. If you need to do it anyway, and it's related to sexual selection, hard-wiring it may free up mental resources for more complicated memes. At the extreme, entirely new emotions may be developed (e.g. shame or embarassment).
Ethical Transplants
Engineering an ethical code - whether its for Attorneys or Humanists - doesn't guarantee that anyone will follow it. Following the code in the face of innate or moral emotion requires an emotional hook. There are two major emotional pathways an ethical code can follow, and they're both indirect. There can be an external enforcer - God, the Police, the Bar Association, or Santa Clause - which followers fear. There can be an internal hook within the moral code which says "it is moral to follow applicable ethical codes." Both approaches are weak and indirect compared to an innate emotional reaction.
Religion
Binding an ethical rule to an emotional response results in a moral tenet that will actually be followed. We can call the beliefs, rites, and rituals that bind and activate the tenet religion, we can call the strength of that binding morality (these aren't the precise meanings of those words, but they are familiar and relevant). Religions are selected for their morality and the extent to which they promote survivability (in some ethical systems that's the same as being ethical, YMMV). They include not only the values-memes themselves, but the layers of memes that bind them.
Conclusion
Ethically-derived values don't work without emotions, because we act on emotions and rationalize after. Repeated and emotional rituals (religions) instantiate morality by binding ethical tenets to emotional responses. Once you know this, you can engineer a religion just like any other virus:
Like any other bio-engineering, you lose some control once you release it, and your engineered religion is going into combat with all others.
This implies that the average prior for a medical study is below 5%. Does he make that point in the book? Obviously you shouldn't use a 95% test when your prior is that low, but I don't think most experimenters actually know why a 95% confidence level is used.