Request for proposals for Musk/FLI grants

[-][anonymous]11y50

I certainly hope that Kaj Sotala recent proposal is considered for funding as part of this programme. (Kaj, you around?)

http://intelligence.org/files/ConceptLearning.pdf

[-]Kaj_Sotala11y60

Thanks, Mark. I'm definitely thinking about applying, but my current problem is that I have too many potential proposals that I could write:

There's the concept learning research program that you mentioned.
I also have an interest in figuring just what exactly human values and preferences are: this seems like a topic where there is likely to be plenty of low-hanging fruit, given that a number of fields touch upon the topic: it relates to concept learning, also in AI there's preference learning, philosophers have tried to ask the question of what value is, there's a bunch of stuff in neuroscience about motivation and preference, a bunch of stuff that I've read about emotion research lately is relevant, there's the economic definition of preferences and their research into the topic, there's intercultural comparisons about values in sociology and anthropology... It seems like if someone went through all of this stuff while keeping in mind the question of "okay, so exactly what part would we want an AI to extrapolate and how and why", one should be able to make considerable progress.
I was just recently talking to some friends about ontology identification and ontological crises, and based on some preliminary discussion, it seemed to us like one could make progress on it using an approach inspired by conceptual metaphors. Briefly, a conceptual metaphor is a mapping from one domain to another, "that allows us to use the inferential structure of one conceptual domain (say, geometry) to reason about another (say, arithmetic)" (Lakoff & Núñez, p. 6). Intuitively, the physicists who discovered quantum mechanics knew that their discovery didn't make everything they knew of classical mechanics obsolete - all the observations supporting CM were still there, so there had to exist some mapping from CM concepts to QM concepts in a way that allowed us to preserve most of what we knew of CM. That would suggest that an AI that discovered that its current understanding of the world was lacking could attempt to take its old world-model, and identify the things that it valued in the new model by looking for the inferential rules of the old model that could still be mapped into the new one, and using that to identify the corresponding mappings to entities. One of the friends I was talking with indicated that there's research on the deep learning side that might be able to do something like this.
At this rate, tomorrow I'll probably come up with a fourth thing that seems like a promising research direction.

I guess I could just write proposals for each of these and let FLI decide which one they find the most promising - their FAQ says that one can submit several proposals but they'll only invite one Full Proposal from a single PI.

[-]danieldewey11y50

I would encourage you to apply, these ideas seem reasonable!

As far as choosing, I would advise you to choose the idea for which you can make the case most strongly that it is Topical and Impactful, as defined here.

[-][anonymous]11y30

The concept learning proposal is basically finished right? Submit it now. (OK, I'm biased -- I really like this proposal.) Start writing up the others in the order you think is most important / most likely to be funded and submit those as well.

As some feedback, I don't think "what are human values?" is likely to get much funding from FLI, although obviously a representative of that organization should correct me if I'm wrong. It seems they have a preference to projects more directly connected to code.

Regarding your third idea, I'm pretty sure there is already some published work in this area. I certainly recall some discussion in the OpenCog community about the nature of creativity and concept formation via conceptual metaphors. I'm pretty sure that was in response to some published academic papers, but I'll have to dig those up...

[-]Kaj_Sotala11y60

The concept learning proposal is basically finished right? Submit it now.

Good point, that makes sense.

I guess "can't choose the right one" wasn't actually my true rejection, rather I'm hesitating because I'm not sure whether this field is actually where my comparative advantage lies, and whether this is the kind of thing that I'd want to be doing. I do fine when it comes to vague philosophizing at the level of my original concept learning paper, but I'm much less certain of my ability to do actual rigorous technical work. Meanwhile I seem to be getting promising feedback of doing well on some other (non-technical) high-impact projects I've been pursuing.

Though I guess I could apply for the first stage of the grants anyway and decide later, since it doesn't commit me to anything yet...

[-][anonymous]11y10

I'm not sure whether this field is actually where my comparative advantage lies.

What else are you considering?

I would advise that's only half the the equation though. You should also weight by how unique that contribution would be. We simply don't have enough people doing AGI work like concept formation. Not to place too much pressure, but if you don't work on this then it's not clear who would. It's an underfunded area academically (hence these grants are a great opportunity), and too long term to be a part of industrial research efforts...

[-]Kaj_Sotala11y00

What else are you considering?

Rationality training and community-building, basically.

But I just submitted my FLI grant application for the concept learning project anyway. :-)

[-][anonymous]11y00

Rationality training by itself is worse than useless. Apply things in practice or you risk building free-floating castles detached from any practical application. A basic rule of thumb: if you spend more than 10-15% of your time on meta improvements, you are probably accomplishing less in your life than you could be. That means 85% to 90% of your time should be spent doing actual work.

As for community building, if that floats your boat, sure why not. I'm hoping you choose the FLI grant instead however :)

[-]Kaj_Sotala11y00

Oh yeah, forgot to say that my initial grant application on concept learning was accepted to the second round of proposals.

Working on the full-length proposal now.

[-][anonymous]11y00

Let me know if you need a review.

[-]Kaj_Sotala11y00

Rationality training by itself is worse than useless. Apply things in practice or you risk building free-floating castles detached from any practical application. A basic rule of thumb: if you spend more than 10-15% of your time on meta improvements, you are probably accomplishing less in your life than you could be. That means 85% to 90% of your time should be spent doing actual work.

Yeah, CFAR-style rationality training is the goal: carried out by actually troubleshooting and solving one's real-life problems, while also building a community of like-minded people to remind you to actually think about your problems instead of doing whatever default thing comes to mind.

LESSWRONG
LW

LESSWRONG
LW

35

Request for proposals for Musk/FLI grants

35

35