Thanks for the link, John.
When most of SI (and a few others) discussed the issue for 10+ hours, we tentatively concluded that WBE progress should not be accelerated. (Context: see differential technological development.)
Here is more discussion of the topic:
http://www.overcomingbias.com/2011/12/hurry-or-delay-ems.html
If we are going to try to influence this decision, we might as well try to make our estimate as high quality as possible. To do this, it seems like a good idea to discuss the issue as thoroughly as possible before coming to any firm conclusions.
If we are going to try to influence this decision, we might as well try to make our estimate as high quality as possible.
If you think WBE progress negatively impacts the survival of humanity I would think you would want to choose the lowest quality group that's still plausible.
(Unless that breaks your ethical injunctions against dishonesty in which case the best course of action would seem to be to abstain from influencing the decision)
Edit: This appears to be a working link to the discussion Luke is referencing.
Some thoughts:
It doesn't look like any of the people in your discussion were academic neuroscientists.
I don't know how much we can read into the fact that the explicit aim of the project is to emulate a brain, not make brain inspired AI.
At this point, I'm pretty uncertain, and it doesn't look like there is much interest from less wrong users in constructing a better estimate. And since the discussion you describe is something of a black box, it seems like it would be awkward to improve upon.
I don't know how much we can read into the fact that the explicit aim of the project is to emulate a brain, not make brain inspired AI.
Unfortunately, the project seems to aim at emulating some sort of generic human brain, instead of a high fidelity simulation of a specific individual (which would be likely to retain their values and skills). I'm inferring this from the fact that the project site has no description of how it plans to scan an individual's brain, nor lists brain scanning as a major research topic. Also, the listed benefits for the project do not depend on having a hi-fi WBE:
Biologically detailed simulations of the brain will make it possible, for the first time, to identify the multi-level chain of interactions leading from genes to cognition and behaviour. Also to be researched, using supercomputer-based simulation technology, are new diagnostic tools and treatments for brain disease, new interfaces to the brain, new types of low-energy technologies with brain-like intelligence, and a new generation of brain-enabled robots.
It looks like they are interested in brain inspired AI (see the part I italicized above).
It happens that Carl Shulman and I have recently been discussing some issues related to your question. Have you seen that thread?
How silly of me to not read the project website. Poking around, it looks like they aren't exactly limiting themselves in scope that much.
Thanks for the link to that thread; I had not seen it! I e-mailed Stuart Armstrong to try to figure out what the current position of the FHI is.
The FHI position on WBE is by no means uniform. The key questions are whether WBE research will lead to neuromorphic AI (NAI), whether WBE makes FAI more or less likey, and whether (WBE transition) followed by (AI transition) is more survivable than the other way round (and, of course, on the problems and solutions of WBE itself, eg Robin's nightmare scenario vs effective immortality).
My position is that successful WBE will make FAI tremendously easier (we can for instance tell the AI "do what this WBE program would tell you to do, if you ran it for a thousand subjective years" (similarly to a suggestion of Paul Christiano's), and the WBE would be able to keep pace with the AI's speed, and thus make a breakout more difficult). Other people at the FHI have different opinions, consistent with their different assessement of the risk of AI and the impact of WBE, and I won't put words in their mouths. Though one relevant fact is that getting NAI from partial WBE is generally considered to be harder by those who know the most of neuro-biology (and easiest by those who know the least).
Thanks Stuart.
In your e-mail to me, you estimated that these conflicting opinions added up to a "weak consensus towards WBE" within FHI. Since SI workshop participants' opinions added up to a weak consensus against WBE, there doesn't seem to be a strong case for trying to shift probabilities in either direction at this point.
Edit (2014-05-19): I just spoke with FHI academic project manager Andrew Snyder-Beattie, and he represents FHI as having a widespread consensus towards slowing progress on all artificial intelligence. He additionally says FHI thinks ems could be less safe than mathematically constructed intelligences. So it sounds like FHI wants to slow down all research of this sort and try to increase everyone's awareness of potential dangers.
I just noticed from that document that you listed Alexander Funcke as owner of "Zelta Deta." Googling his name, I think you meant "Zeta Delta?"
I forgot to mention, there are a few other candidate projects which might impact existential risk. For example, one is an exploration of graphene based hardware:
Wait, isn't this almost exactly the beginning of Greg Egan's novel Zendegi, at about approximately the same time?
In Zendegi, the money eventually went to SingInst!
Correction: 1% chance of influence is $14 million in expected funds.
And if the conclusion is to not accelerate, which seems likely given lukeprog's posts, might we then conclude that it would be worthwhile lobbying against the program?
I noticed that I went straight to the comments before reading too deeply about the specific options and then it occurred to me that it would be helpful for others if the real options (many of which bear on the singularity but only one of which is WBE from what I can tell) were in this thread where they were easier to find. I pulled text and links from the grant agency's page of options to put them into child comments that can be voted up or down independently, and can (hopefully) inform the rest of the thread while accumulating feedback in the form of votes. If you don't like the idea of me getting karma because one or another of the child options went up, there will be a karma sink right next door to the options :-)
The FuturICT Knowledge Accelerator and Crisis-Relief System: What if global scale computing facilities were available that could analyse most of the data available in the world? What insights could scientists gain about the way society functions? What new laws of nature would be revealed? Could society discover a more sustainable way of living? Developing planetary scale computing facilities that could deliver answers to such questions is the long term goal of FuturICT. This initiative is seeking to develop Information and Communications Technologies that will provide scientists, governmental officials and citizens with a planetary scale computer which is called a Living Earth Platform. The Living Earth Platform will provide the means of analysing data and managing complex events. It could for example provide a basis for predicting natural disasters, or managing and responding to man-made disasters that cross national borders or even continents. The intention is to undertake interdisciplinary research, involving domains such as complexity, computer and the social sciences, to address the scientific challenges associated with the realisation of this goal and the needed advances in Information and Communication Technologies. Prof. Steven Bishop
The IT Future of Medicine: Data-rich, individualised medicine poses unprecedented challenges for IT, in hardware, storage and communication. ITFoM proposes a data-driven, individualised medicine of the future, based on the molecular/physiological/anatomical data from individual patients. ITFoM shall make general models of human pathways, tissues, diseases and ultimately of the human as a whole. Patient individualised versions of the models will then be used to identify personalised prevention/therapy schedules and side effects of drugs. This is the first time that huge IT implications of worldwide individualized patient care will be addressed in combination with genomics and medical requirements. The project outcomes will enable calculation of health, disease, therapy and its effects for individual patients. These may revolutionize our health care with enormous (i) benefits for health (prevention, diagnosis and therapy), (ii) reduction in cost by individualising combinations of a limited number of drugs, and (iii) new commercial opportunities in IT, analytics and health care. This entails nothing less than the transformation of biomedical science from empirical and stochastic to fact based and knowledge driven i.e. based on an ICT paradigm. Prof. Hans Lehrach
The Human Brain Project: Understanding the way the human brain works could be key to enabling a whole range of brain related or inspired developments in Information and Communication Technologies, as well as having transformational implications for neuroscience and medicine. The long term goal of the Human Brain Project is to build the informatics, modelling, and supercomputing technologies that are needed to simulate and understand the human brain. Biologically detailed simulations of the brain will make it possible, for the first time, to identify the multi-level chain of interactions leading from genes to cognition and behaviour. Also to be researched, using supercomputer-based simulation technology, are new diagnostic tools and treatments for brain disease, new interfaces to the brain, new types of low-energy technologies with brain-like intelligence, and a new generation of brain-enabled robots. Prof. Henry Markram
Guardian Angels: Providing Information and Communication Technologies to assist people in all sorts of complex situations is the long term goal of the Guardian Angels Flagship Initiative. These Guardian Angels will be like personal assistants and are envisioned as intelligent (thinking), autonomous systems (or even systems-of-systems) featuring sensing, computation, and communication, and delivering features and characteristics that go well beyond human capabilities. It is intended that these will provide assistance from infancy right through to old age. A key feature of these Guardian Angels will be their zero power requirements as they will scavenge for energy. Foreseen are individual health support tools, local monitoring of ambient conditions for dangers, and emotional applications. Research will address scientific challenges such as energy-efficient computing and communication; low-power sensing, bio-inspired energy scavenging, and zero-power human-machine interfaces. Prof. Adrian Ionescu
Robot Companions for Citizens are soft skinned and sentient machines designed to deliver assistance to people. This assistance is defined in the broadest possible sense and covers all sorts of different settings. Based on multidisciplinary science and engineering, CA-RoboCom aims to develop a radically new approach towards machines and how these are deployed in society. Robot Companions for Citizens will be based on the novel solid articulated structures with flexible properties displaying soft behaviour. These companions will also have new levels of perceptual, cognitive and emotive capabilities. They will also be aware of their physical and social surroundings and respond accordingly. Such sentient characteristics will be achieved through understandings of the behaviour of sentient living creatures. In undertaking the research into the Information and Communication technologies that will need to be developed, the research will also validate understandings of the general design principles underlying biological bodies and brains, thus supporting a symbiotic relationship between science and engineering. Prof. Paolo Dario
Karma sink! If you voted up an option and don't want me to get reputation for that, vote this down. I'll have four children off of this in case you voted up five of the six. (You shouldn't vote up all six proposals, because then your net voting would cancel itself.)
Graphene Science and technology for ICT and beyond: Graphene, a new substance from the world of atomic and molecular scale manipulation of matter, could be the wonder material of the 21st century. Discovering just how important this material will be for Information and Communication Technologies is the long term focus of the Flagship Initiative, simply called, GRAPHENE. This aims to explore revolutionary potentials, in terms of both conventional as well as radically new fields of Information and Communication Technologies applications. Bringing together multiple disciplines and addressing research across a whole range of issues, from fundamental understandings of material properties to Graphene production, the Flagship will provide the platform for establishing European scientific and technological leadership in the application of Graphene to Information and Communication Technologies. The proposed research includes coverage of electronics, spintronics, photonics, plasmonics and mechanics, all based on Graphene. Prof. Jari Kinaret
Even if academic neuroscientists don't think much of the project in its current state, it seems likely that $1.4 billion would end up attracting a lot of talent to this problem, and get us the first upload significantly sooner.
This isn't obvious to me. If it simply too early (due to lack of knowledge especially in relevant connected fields) then this sort of thing could have extreme diminishing marginal returns in investment. If that occurs and it takes talent away from other related areas, this sort of thing could actually slow down research overall. I suspect this isn't the case in this situation, but it seems probable enough that it should be considered.
If the project is well managed, funds might be invested in research in relevant connected fields.
re: threats: I don't see how exactly a brain based AI can be even seen as a 'threat' to survival of humanity. I instead see it as about the only way that permits humanity to survive at all. First of all, the simbrain is a part of humanity itself precisely in the way in which from-scratch AI isn't, second, even if the simbrains are a little bit charitable towards the original humans, that's your FAI.
This is the team responsible for simulating the rat cortical column.
http://www.nature.com/news/2011/110308/full/news.2011.143.htm
The team is one of 6 that is being considered for at least 2 "FET Flagship" positions, which comes with all that funding. Each of the six competing teams is proposing to work on some kind of futuristic technology:http://cordis.europa.eu/fp7/ict/programme/fet/flagship/6pilots_en.html
Of course, word on the street is that academic neuroscientists don't think much of the project:
Academic neuroscientists that I’ve ever spoken too, which is a fair number now, don’t think much of the Blue Brain project. They sometimes think it will be valuable in terms of collecting and cataloguing information about the neocortex, but they don’t think the project will manage to understand how the cortex works as there are too many unknowns in the model and even if, by chance, they got the model right it would be very hard to know that they had.
Almost all neuroscientists seem to think that working brain models will not exist by 2025, or even 2035 for that matter. What ever the date is, most consider it too far away to bother to think much about.
Such projects probably help to get more kids interested in the topic.
I think trying to influence the committee's decision potentially represents very low hanging fruit in politics as charity.
Even if academic neuroscientists don't think much of the project in its current state, it seems likely that $1.4 billion would end up attracting a lot of talent to this problem, and get us the first upload significantly sooner.
It's true that Less Wrong doesn't have a consensus position on whether to speed development of cell modeling and brain scanning technology or not. But I think if we have a discussion and a vote, we're significantly more likely than the committee to come up with the right decision for humanity. As far as I can tell, the committee will essentially be choosing at random. It shouldn't be hard for us to beat that.
Edit: But that's not to say that our estimate should be quick and dirty. In the spirit of holding off on proposing solutions, I discourage anyone from taking a firm public position on this topic for now.
In terms of avenues for influence, here are a few ideas off the top of my head:
We, and the folks at the Future of Humanity Institute, SI, and other groups, seem to spend a lot of time thinking about what would happen in the ideal scenario in terms of the order in which technologies are developed and how they are deployed. I think there is a good case for also investing in the complementary good of trying to actually influence the world towards a more ideal scenario.