James Stephen Brown

Utilitarian* atheist, artist, coder, documentarian and polymath (jokes.. but I don't believe to be a jack of all trades necessitates one being a master of none, rather that the synergy of many fields can lead to novel insights—and I also just want to know everything!).

I write about moral philosophy, artificial intelligence and game theory—in particular non-zero-sum games and their importance in solving the world's problems. Most of my writing originates on my personal website nonzerosum.games.

I have admitted I am wrong at least 10 times on the internet.

* I don't really class Utilitarianism as an ethical framework in competition with other ethical frameworks, I see it more as a calculus that most people, when it comes down to it, use to determine or assess more the generalised virtues, principles and laws that they live by (well, at least I do).

Wiki Contributions


I think the idea is really interesting. As someone who spent 5 years creating student video resources, I appreciate the impact they can have, and I have at times tried to convince my father—a life-long maths teacher to collaborate with me on replicating his course... but the fool didn't take me up on the offer.

the cost of showing it to every student in the country is approximately zero

I feel like the cost-effectiveness argument is valid but might run into issues. To begin with, as you have in one of your comments pointed out, video resources with a teacher who can respond dynamically, adds much more than a video alone. So, this means there is no cost saving in terms of teachers time—which I think is a good thing (I'll put a pin in that for later) and then video production on top of that is not at all cheap. One thing that was consistent, in my experience creating educational resources, was the need to constantly update the resources (there was a team of us working full time to just maintain one course).

So, while the cycle of feedback and constant improvement of the resources is a vital part of the process, it makes what seems like a one-off expense into a perpetual expense.

Furthermore teachers are already underpaid, relative to other professions requiring similar skills, so the additional funding for these new resources would need to result from an unprecedented increase in education funding (which could have gone to teachers) or would have to be taken from the budget at the expense of teachers.

Unless of course you leave it to the private sector in which case you have to worry about advertising, special interests and competition leading optimisation for what is appealing to students rather than what is necessarily effective—Hollywood, after all only has the mandate to entertain, they don't have to also educate.

To get back to that pin: If we did manage to create a resource perhaps incorporating generative AI that can present ideas in an engaging way and provide dynamic feedback, making teachers unnecessary we run into another issue. There's something to be said for having well-rounded educators in society, learning in a non-specialised way is enriching for people in general. One negative side of chat GPT is this way drastic drop-off in activity on forums like Stack Overflow, because people don't need other people any more.

There's something about the person to person trading of ideas that I think contributes to a robust community, in the same way that international trade helps to curb international conflicts—we might find that making human interaction unnecessary to education whether in schools or on forums might lead to a fragmentation of the social fabric. Personally I really like the idea of lots of amateurs sharing ideas—like on LessWrong and other forums, there's something uniquely human about learning from sharing, with benefits for the teacher also (à la the Feynman Technique).

But, I think you make a good case. Thanks for sharing the idea.

Hey, again good points.

But I have recognized sparks of true understanding in one-shot AI works.

I absolutely agree here, this is what I was referring to when I wrote...

I think we can appreciate the beauty of connecting with humanity as a whole, knowing that it is the big data of humanity that has informed AI art - I suspect this is what we find so magical about it.

I suspect that AI has an appeal not just because of its fantastic rendering capacity but also the fact that it is synthesising works not just from a prompt but from a vast library of shared human experience.

you're looking at hours of thought and effort

Regarding the arduous* process of iteratively prompting and selecting AI art, I think the analogy with photography works in terms of evoking emotions. Photographers approach their works in a similar way, shooting multiple angles and subjects and selecting those that resonate with them (and presumably others) for exhibition or publication. I think there is something special about connecting with what a human artist recognised in a piece whether it came from a camera or an algorithm. I acknowledge this is a form of connection that is still present in AI art, just as it is in photography.

* I caveat "arduous" because, while it might take hours of wrangling the AI to express something approximating what we intend, the skill that takes artists years to master—that of actually creating the work, is largely performed, in the case of AI art, by the non-sentient algorithm. It is not the hours of work that goes into one painting that impresses the viewer generally, it's the unseen years of toil and graft that allowed the artist to make something magic within those hours. The vast majority of the magic in AI art is provided by the algorithm.

This is why I see it as analogous to photography. Still a valid art form, but not one that need make actual painting obsolete.

Okay, so I think I get you now, in the imbalanced game, if the payoff is 100 or 1 as in "Zero Sum" is a misnomer, a rational player will still make the same decision, regardless of the imbalance with the other player, given the resulting preference ordering.

However while this imbalance makes no difference to the players' decisions, it does make a difference to the total payoff, making it non-zero-sum. I'm having difficulty understanding why values such as happiness or resources cannot be substituted for utility—surely at some point this must happen if game theory is to have any application in the real world. Personally I'm interested in real world applications, but I fully acknowledge my ignorance on this particular aspect.

I find a practical way to look at a zero-sum game is to imagine that each of two players must contribute half of the total payoff in order to play. This takes a game that is constant-sum, and makes it zero-sum, and does so in a way that doesn't break the constant-sumness. In the case of the imbalanced game, because it is not constant-sum it doesn't reduce to a zero-sum game in this way, remaining a non-zero-sum game with terrible odds for one player, meaning that a rational player won't opt in if given the option.

If I'm not mistaken, this is generally what is meant when someone refers to a zero-sum game. In chess for instance you enter a competition with your rating (essentially your bet) and the outcome of the game has either a negative or positive (or no) impact on your rating and an opposite impact on your opponent's rating.(I'm not exactly sure if chess ratings are calculated as exactly zero-sum, but you get the idea). So, the game is zero-sum. Of course there are outside factors that make it beneficial to both players; enjoyment, brain-exercise, socialising etc which may have positive utility on another level, but the game itself and the resulting rating changes are essentially zero-sum.

This is the sense in which I am using the term "zero-sum", in the most basic sense for someone to win (relative to their starting point, bet or rating) another must lose by an equal amount.

There is probably a more mathematically succinct way of expressing this, but I don't have those tools at my disposal at present. Again, thanks for your comments. Please don't feel the need to continue your labours educating me on this topic, I understand that you clearly have a better understanding of game theory than I do, so I appreciate your time. I should probably continue reading further to level up my understanding. Of course if you feel like continuing the floor is yours. 

I’m not sure how the game is the same when you add a constant. The game as proposed in the example is clearly different. I can see that multiplication makes no difference, and as such doesn’t make the sum non-constant. I don’t see how asymmetrically changing the parameters is a “mere change in notation”.

By the way, I’m sure you’re entirely correct about this, I just simply don’t see how there is a problem with using the concept of zero-sum understood as constant-sum.

Hi Vladimir, thanks for your input, it has been fascinating going down the rabbit hole of nuance regarding the term "zero-sum".

I agree that the term is more accurately denoting "constant-sum", I think this is generally implied by most people using it. There was the interesting "zero-sum" example in the linked article that veered away from "constant-sum" with asymmetrical payoffs, 100,0 or 0,1, meaning that depending on the outcome of the game the total sum would be different. This, to me disqualifies it from being called a zero-sum game, given the common understanding that zero-sum denotes constant-sum. The example seemed to solve the problem by conflating zero-sum and constant-sum and then proceeded to stick to a strict definition of zero-sum, which was confusing. But perhaps I just need to sit with it longer.

To your point about Kaldor-Hicks, yes I guess many positive-sum situations could be described in these terms but I'm really referring to something more general—any situation where the total sum payoff increases regardless of Pareto improvements or promised reimbursement by other means to any party left worse off. For instance if a left-wing government were to increase taxes on the wealth, not offering them any reimbursement, but rather doing this based on the mandate that comes with being democratically elected, then this policy might be positive-sum due to the fact that dollar-for-dollar money makes a bigger difference to a poor person than a rich person, due to diminishing returns on happiness.

I really appreciate your comments, and intend to continue exploring the nuances you've raised. I think for a primer on non-zero-sum games, particularly with a site that is focused on practical solutions in the real world rather than pure theory, the more accessible (perhaps less nuanced) definitions I've used are probably appropriate.

Hi Vladimir,

Thanks for your comment, please excuse the delay in getting back, I'm actually busily digesting your response and the various branches of dependencies that comprise it (in terms of links to other concepts). I intend to get back to you with a considered answer, but am enjoying taking my time exploring the ideas you've linked to.

Hi bideup, thanks for your comment. The graph is simplified from one in the Pew Report with the left bar representing the lower quintile and the right representing the upper. I see what you mean, but the intention of pointing to the 20% mark is to show where it should be given 100% social mobility. Perhaps the omission of the central quintiles didn’t help.

There's a tiny possibility he may have influenced my thinking. I did spend 6 months editing him, among others for a documentary.

That looks interesting, will read :) Thanks.

What an insightful post!

I have difficulty judging how likely that is, but the odds will improve if semi-wise humans keep getting input from their increasingly wise AGIs.

I think we're on the same page here, positing that AGI could actually help to improve alignment—if we give it that task. I really like one of your fundamental instructions being to ask about potential issues with alignment.

And on the topic of dishing out tasks, I agree that pushing the industry toward Instruction Following is an ideal path, and I think there will be a great deal of consumer demand for this sort of product. A friend of mine has mentioned this as the no-brainer approach to AI safety and even a reason what AI safety isn't actually that big a deal... I realise you're not making this claim in the same way.

My concern regarding this is that the industry is ultimately going to follow demand and as AI becomes more multi-faceted and capable, the market for digital companions, assistants and creative partners will incentivise the production of more human, more self-motivated agents (sovereign AGI) that generate ideas, art and conversation autonomously, even spontaneously.

Some will want a two-way partnership, rather than master-slave. This market will incentivise more self-training, self-play, even an analogue to dreaming / day-dreaming (all without a HITL). Whatever company enables this process for AI will gain market share in these areas. So, while Instruction Following AI will be safe, it won't necessarily satisfy consumer demand in the way that a more self-motivated and therefore less-corrigible AI would.

But I agree with you that moving forward in a piecemeal fashion with the control of an IF and DWIMAC approach gives us the best opportunity to learn and adapt. The concern about sovereign AGI probably needs to be addressed through governance (enforcing HITL, enforcing a controlled pace of development, and being vigilant about the run-away potential of self-motivated agents) but it does also bring Value Alignment back into the picture. I think you do a great job of outlining how ideal an IF development path is, which should make everyone suspicious if development starts moving in a different direction. 

Do you think it will be possible to create an AGI that is fundamentally Instruction Following that could satisfy the market for the human-like interaction some of the market will demand?

I apologise if you've, in some way I've not recognised, already addressed this question, there were a lot of very interesting links in your post, not all of which I could be entirely sure I grokked adequately.

Thanks for your comments, I look forward to reading more of your work.

Load More