Disambiguation: What I’m talking about here is essentially the same thing that John Wentworth works on and writes about, including in Public Static: What is Abstraction?, and I stole some of the things he’s written (by quoting him directly) to save myself time. But I think this post has a different aim from anything I’ve read from him or anyone else (which isn’t to say no one has written a post with this aim; someone probably has).
For a very quick TL;DR, I'd suggest reading the Definitions section and the Abstract Answer subsection.
My aim is to get readers (with approximately any background?) to understand the concept of abstraction; see that the concept is ubiquitous in the world and even in their lives; appreciate the usefulness of the concept; hone the skill of recognizing, generating, and utilizing abstractions; internalize the concept deeply enough to install the habit of employing this skill in the wild; and, ideally, learn to love the concept.
Simply skimming this post will hopefully partially serve all of these purposes, but I think doing the exercises included below is probably also an important component.
I expect this to be less valuable if you are already familiar with the concept of abstraction, but perhaps skimming this post will still lead you to encounter some valuable new perspectives on it.
Abstraction is a conceptual process in which general rules and concepts are derived from specific examples by identifying shared high-level features and ignoring low-level differences. ‘An abstraction’ is the outcome of this process: a general idea that unifies a collection of specific objects, especially in a way that provides non-trivial insight into the nature or relationships of those objects.
John Wentworth said:
Anyway, (my reading of) Jaynes' answer to the main question: abstraction is mainly about throwing away or ignoring information, in such a way that we can still make strong predictions about some aspects of the underlying concrete system.
These are closely related definitions of abstraction; having multiple framings available to you may be helpful.
This section contains a bunch of examples of abstractions - I urge you to read as many as you need to feel like you have a handle on the basic concept, then skip to the next section. That's because in this section, for the sake of simplicity and clarity, the abstractions are fairly mundane; I don't expect many people to find them very exciting. Later in this post, I'll try to provide more exciting examples, and provide exercises that allow you to generate exciting examples for yourself.
We have a gas consisting of some huge number of particles. We throw away information about the particles themselves, instead keeping just a few summary statistics: average energy, number of particles, etc. We can then make highly precise predictions about things like e.g. pressure just based on the reduced information we've kept, without having to think about each individual particle. That reduced information is the "abstract layer" - the gas and its properties.We have a bunch of transistors and wires on a chip. We arrange them to perform some logical operation, like maybe a NAND gate. Then, we throw away information about the underlying details, and just treat it as an abstract logical NAND gate. Using just the abstract layer, we can make predictions about what outputs will result from what inputs. Note that there’s some fuzziness - 0.01 V and 0.02 V are both treated as logical zero, and in rare cases there will be enough noise in the wires to get an incorrect output.I tell my friend that I'm going to play tennis. I have ignored a huge amount of information about the details of the activity - where, when, what racket, what ball, with whom, all the distributions of every microscopic particle involved - yet my friend can still make some reliable predictions based on the abstract information I've provided.When we abstract formulas like "1+1=2*1" and "2+2=2*2" into "n+n=2*n", we're obviously throwing out information about the value of n, while still making whatever predictions we can given the information we kept. This is what abstraction is all about in math and programming: throw out as much information as you can, while still maintaining the core "prediction" - i.e. the theorem or algorithm.I have a street map of New York City. The map throws out lots of info about the physical streets: street width, potholes, power lines and water mains, building facades, signs and stoplights, etc. But for many questions about distance or reachability on the physical city streets, I can translate the question into a query on the map. My query on the map will return reliable predictions about the physical streets, even though the map has thrown out lots of info.
Given an abstraction, there are two main uses (that I know of): incorporation and generation of new examples into the framework defined by the abstraction.
Abstraction in action shapes the world, and will likely do so even more radically in the future. Multilayered abstractions are critically useful, and it intuitively seems to me that the extent to which we can use them is the main difference between humans and other animals that allows us to shape the world to our desires as much as we do. Everyone relies on tools that other people provide without having the first clue how to produce them themselves (e.g. houses, cars, electronic devices, the internet, software packages, etc.), but because using these tools only requires knowledge of how to directly interface with them rather than low-level details of how to create them, we can build on each other’s work very effectively. Further, the creation of many of these tools also utilizes other deeper abstractions, which themselves are often built on abstractions, and so on for many levels. The state of human civilization today is the result of millennia of progress, much of which can be attributed to the development and application of new abstractions. This progress may have brought us to The Precipice, and the way we build on the abstractions we currently have in the near future may have a dramatic influence on the long-term future.
Frankly, abstraction may not be the most important idea to be consciously aware of, since people use it by default without paying too much direct attention to it. For example, here are a couple of abstraction exercises that I tried to do:
If these exercises generate useful ideas for you, that's great! But they didn't for me. While at first I expected these to lead to useful insights, I realized after a few minutes that most of the things that came to mind were already familiar, and I just hadn't explicitly thought about them from the lens of abstraction before. They included things along the lines of "Read more about AI alignment (to figure out what relevant abstractions I'm not aware of yet)," "Get good at ML engineering," "Study more math," "Try to generate meaningful theoretical results for AI alignment", etc. The answers didn't really feel very abstraction-related, because that wasn't the most natural language in which to express them.
After that, my view on the value of being consciously aware of the concept of abstraction changed. I still do think that being deeply familiar with the concept of abstraction (in the ways described in the Aim section above) can help a lot. However, this value doesn't come from just thinking about abstraction and getting a major shortcut to large important ideas. Instead, it makes you less likely to miss opportunities to learn general lessons. It makes you more likely to notice when cool things you come across share properties with other interesting things you've previously seen, and that kicks off a mental process that allows you to synthesize generalizable takeaways.
By this point, I have on numerous occasions come across something, realized it is an example of an abstraction, and thought "Woah, that's super cool! I totally feel like I've seen this general idea in a bunch of interesting places." Then, I have often had one or more of the following thoughts:
I would then try to answer those questions, and a reasonable proportion of the time I had enough success to find it fruitful. Each time, this activity made a new abstraction more familiar to me, and I became more likely to incorporate new examples of it that I came across into an existing framework in my mind.
In the hopes that I can recreate this experience for you, this section will consist of:
If you are comfortable sharing, please put your responses to any or all of the exercises in the comments!
The notion of abstraction is itself an extraordinarily deep and powerful abstraction. This whole post is dedicated to analyzing it through its examples and uses! It is the one that I most want you to appreciate, but I think you come to appreciate it by getting excited about several other examples of abstraction, and then registering that the thing that all of those have in common is that they are abstractions.
This may take some time, since the process is highly recursive: to appreciate the notion of abstraction, you first need to appreciate many abstractions, but to appreciate abstractions, you need to appreciate their specific examples, and sometimes those are also abstractions, and so on. The good news is that I think this process gives rise to deeply intellectually satisfying and enlightening trains of thought, and is extremely fun. The next few subsections are all aimed at walking you through this process, so let's get into it!
One quick note before we do: Keep in mind that different prompts and examples will resonate for different people. It's fine if some of them don't work for you - hopefully, at least some of them will.
Many things that have precise definitions are abstractions. The circle example above fits this category.
Make a list of abstractions that have precise definitions. Write down some examples and some of the key insights that these abstractions provide or allow for (e.g. theorems related to them). Continue until you feel like you have a good grasp on the richness of this class of abstractions (or until you think it's unlikely that will be attainable in a reasonable amount of time).
Suggested starting point: Look at the high-level topics, core results, and major theorems covered in STEM subjects you have previously studied (ideally ones where you know the material particularly well), and try to identify abstractions within them. You may want to look at syllabi, review guides, cheat sheets, etc. which give concise overviews of important high-level topics. (I think this is probably easiest for math and theoretical computer science courses.)
This is essentially my (relatively brief) response to the exercise above. Looking at some of my examples may help you come up with ideas, but it may also use up some of the things you could have come up with on your own. I don't go into detailed descriptions of these abstractions or insights, and I only thought of examples in my head rather than writing them down, so this may not be very legible if you aren't already familiar with the subjects I draw from. My list may be unsatisfying to you, but your list should not be, since it should contain abstractions and insights that you are familiar with.
Indented bullets indicate nested abstractions; if an object is an example of the sub-bullet, that implies it is also an example of the parent bullet. 
Computer Science Theory
The layers of abstraction indicated above encode several key results themselves.
There are a lot of other abstractions and theorems here - it's called abstract algebra for a reason!
There are many problem solving techniques that are effective for large classes of problems. For example, doing a sanity check when you arrive at a solution to a problem often helps catch major flaws.
Make a list of problem solving technique abstractions. Write down some examples and some of the key insights that these abstractions provide or allow for. Continue until you feel like you have a good grasp on the richness of this class of abstractions (or until you think it's unlikely that will be attainable in a reasonable amount of time).
There are countless themes, ideas, principles, strengths, and shortcomings that are shared across stories of all kinds. Numerous stories are variants of the Hero's Journey: "the common template of stories that involve a hero who goes on an adventure, is victorious in a decisive crisis, and comes home changed or transformed." Many stories suffer from deus ex machina, in which "a seemingly unsolvable problem in a story is suddenly and abruptly resolved by an unexpected and unlikely occurrence" (though sometimes this can be intentional and serve a purpose).
Come up with as many storytelling abstractions as you can. Also, come up with as many examples of each storytelling abstraction as you can.
This definition combines components from the Wikipedia page on abstraction, John Wentworth's writing on abstraction, and my own interpretation.
That is, every quadratic equation with coefficients in the complex numbers has two solutions in the complex numbers.
Here, a = 1, b = 0, and c = -1. Plugging these values into the quadratic formula yields x = 1 and x = -1.
Some of John's examples show that you need not care about all of the specific objects that an abstraction unifies - in fact, sometimes you only care about one of the objects. For instance, in the map of NYC example, the degrees of freedom that arise from moving from the actual streets to their representations in the map are things like "street width, potholes, power lines and water mains, building facades, signs and stoplights, etc." So the abstraction unifies all the possible versions of NYC streets with different widths, potholes, power lines, etc. However, we don't care about all these different versions - we only care about the real version. In these cases, the value of the abstraction doesn't come from its unification of many specific objects, but rather from its compression of the description of the single object we care about into a form that is easier to work with.
Of course, our use of abstractions to shape the world isn't always positive. Most people regularly use products that are sourced via factory farming and burning fossil fuels, and the fact that we can use these products without seeing their production processes makes it very easy to indirectly contribute harm to animal welfare and the climate. These are just two examples, and there are countless others.
It isn't really a matter of needing to appreciate examples before starting to appreciate an abstraction, but I think in many cases, you slowly gain an appreciation for the abstraction as you see and come up with more and more examples. It may actually take quite a few examples before you stop seeing new interesting instantiations of the abstraction.
In some cases, like the sets-groups-rings-fields example under abstract algebra, this is because the sub-bullet is a variant of the parent bullet abstraction with additional structure. See the discussion of structure in the Miscellaneous section.
This is kind of just a list of some of my favorite stories, but thinking about the specific ways they instantiate this abstraction helps me remember many of the specific things their creators did to make the stories so great.
If you've completed these stories and want to know what specifically I'm referring to, feel free to ask in a comment. I had a bit of trouble with spoiler formatting so I tried to avoid spoilers, but I know that may have made it hard to know what I was referring to in some cases.
One of the best episodes of TV I've ever seen.
Sanderson's First Law is closely related to the "Have answers to the question 'Why?' as often as possible" abstraction, and his Third Law is closely related to the "Make story components as interconnected as possible" abstraction. I read about Sanderson's Three Laws before crystallizing those abstractions in my head, so they probably helped me do that.
To me abstraction = lossy compression. I don't think there is anything else there. An example of abstraction is mapping a specific human face to the word "face". A really high compression ratio.