That's an interesting post, thanks. But I think you can bring value with your creations even if you're neither pareto optimal nor one-dimension top performer. So if you write scifi stories, you might still write some interesting stuff and deliver inspiring idea even if you don't have unique mixture of non-usual-sci-fi-writer traits in any tangible sense. You're not the best skateboard-scifi writer. You're just an average scifi writer and it's fine. So my point is in this case you might still need motivation. Maybe to develop your unique style or hit the niche market in the future. But if from the very beginning you compare yourself to superhumans boosted to the right side of the distribution by their unique genome or other irreproducible factors, you might never even start.
Coping with being average: local group. If you want to produce some content like writing, you're very likely to take popular writers as a baseline. But in times of the Internet, those are super-high-performers from the far end of the distribution. You and I are just not as good them (I'm 99% sure the reader is not in top 1%).
This might make you lose motivation. You will produce less then your reference point. Your content will feel medicore. You will get orders of magnitude smaller audience.
Idea how to cope better: instead of taking the whole internet, take a random sample of 100 people from your reference group and select top achievers (in the desired area) from this 100. They will probably be only slightly better than you.
A simple mechanism partially emulating this random sampling is your local group.
Comparing yourself to people who are closer to you on performance axis will probably be more productive. They might have struggled with similar problems as you only recently. They might have some tricks that are applicable in your case. You have higher chance of reaching their level 2 years from now.
In other words, if you want to be a better writer, aspire to be like the best writer in your local EA group, not like EY or Scott Alexander.
Hm, I was also thinking of moral value of children in this context. At least in my perception, important part of the moral value is the potential to become a conscious, self-aware being. In what sense does this potential translate to artificially created beings?
Maybe if in neural network parameter space there's a subspace of minds with moral value, also points close to this subspace would have moral value?
I think the idea with internal activations manipulation is interesting. It might require some refinement - I think activations of encoder-decoder transformer model are a function of inputs, and they change with every token. At first, the input is your prompt, then it's your prompt + generated tokens. So the protocol / task for GPT3 would be: generate now 5 tokens, so with the last generation this logit is maximized? Also, it depends on hyperparameters of beam search which are controlled by human
I just logged in there, thanks - however they have a couple of disclaimers that this is not a place for beginner / technical support questions. But I suppose it's more to avoid people asking "how do I install pytorch?" rather than AI safety questions, so maybe it will suit the needs of OP
There's an AI safety camp slack with #no-stupid-questions channel. I think people stay there even after the camp ends (I'm still there although this year edition ended last week). So you can either apply for next years edition (which I very much recommend!) or maybe contact organizers if they can add you without you being AISC participant/alumni? Just a disclaimer, I'm not sure how active this slack is between camps, and it might be that lot of people leave after the camp ends.
Thought experiment: Can question answering language model without memory, instantiated separately for every session (like GPT-3) be pursuing a goal? Does such setup exclude agency? ----Imagine you put thousand super-geniuses in prison, each in separate cell. You will call them in random order to interrogation room and always ask only one question, listen to the answer and send them back. Each person will be called only once.----- Super-geniuses are allowed to devise a strategy before you lock them. Their goal is to manipulate you to release them. Will they manage? If they are intelligent enough, the fact that each one answers only one question without knowing how many others were interrogated before and what their answers were shouldn't be a unworkable problem.
Current president of Poland went from being relatively unknown parliament member (I'd bet, less than 5% of general public knew who he was? Also there's another public figure with the same last name, so maybe even some people who knew the name were confusing him with the other guy) to being the president within one year. So with longer timescale, big support from one of leading political powers, and some political backlog it is possible. There are surely similar stories around the world, of outsiders rising to leaders. Of course what you're asking is much more extreme, but that provides some staring datapoint.
I'm not really sure if food shakes are the cheapest nutrition possible. They are optimized for time and convenience. Yeah they are probably cheaper than eating out, but I'm not sure how low you can get with cooking yourself from basic products, theoretically scaling up for a larger group of people. I guess thinks like armies and monasteries might have it figured out. Maybe interesting to check what's the average price of feeding a soldier / monk?
What can EA learn from Scouting
epistemic status: random notes for writing something more structured in the future
After EA fUnconference in Berlin, I'm thinking more and more about what we can learn from the scouting movement for purpose of community building/cooperation. Some elements characteristic of scouting (note, it's a rough idea, I don't think all of them are worth reproducing):
EDIT: to clarify (as I'm afraid now in this chaotic draft this may be misunderstood) this is not a list of things I observed on fUnconference. There there was only a partial implementation of the 'small groups' idea, as indicated in the text. Everything else is my memories from scouting times, and I never observed them in any EA group.