Rather than counting objects/distances, one way I like to think about the definition of space is by translation symmetry. You do get into symmetry in your post but it's mixed together with a bunch of other themes.
Like, you are in your cave and drop a ball. You then walk out of the cave and look back in. The ball is still there, but it looks smaller and you can't touch it anymore. You walk in, pick up the ball, and walk out again, and then drop the ball outside. The ball falls down the same way outside the cave as it does inside.
If you think of what you observe from a single position as being a first-person perspective, then you can conceive of transformation that take one first-person perspective to a different one, but for such a transformation to make sense, objects need to have positions in space so they can be transformed.
Notably, you don't need a collection of symmetric objects, or a volume with limited capacity for containing things, in order for space to make sense (and you can make up alternate mathematical rules that have limited capacity and similar objects but have no space). On the other hand, if you don't have something like translational symmetry, it feels like you're working with something that's not "space" in a conventional sense? Like it might still be derived from space, but it means you can't talk about "what if stuff was elsewhere?" within the model, which seems like the basic thing space does.
(I guess one could further distinguish global translation symmetry vs local translation symmetry, with the former being the assertion that ~you have a location, and the latter being the assertion that ~everything has a location. Or, well, obviously the latter is an insanely exaggerated version of locality which asserts that Nothing Ever Interacts, but I feel like this is where the physics-as-the-study-of-exceptions stuff goes.)
I also like to think that something similar applies to other symmetries, e.g. symmetry to boosts are basically asserting velocity is a sensible concept (and quantum mechanics provides a reductionistic explanation of how they function).
Be careful. Physics seems to be translation invariant, but space is not. You can drop the ball in and out of the cave and its displacement over time will be the same, but you can definitely tell whether it is in the cave or out of the cave. You can set your zero point anywhere, but that doesn’t mean that objects in space move when you change your zero point. Space is isotropic. There’s no discernible difference between upward, sideways, or diagonal, but if you measure the sideways distance between two houses to be 40 meters, a person who called your “sideways” their “up” will measure the distance between the houses to be 40 meters up and down. You can do everything here as you can do there, but here is not there. In the absence of any reference point, no point in space is different from any other point, but in the absence of any reference point there’s no need for physics, because if there was anything to describe with physics, you could use it as a reference point.
I suppose you could try to define space as the thing you can move around in without changing your physics, but the usual strategy is to define physics and derive conservation of momentum from the fact that your physics is translation invariant.
Formally, I mean that translation commutes with time-evolution. (Maybe "translation-equivariant" would be a better term? Idk, am not a physicist.)
I guess my story could have been better written to emphasize the commutativity aspect.
This is an attempt to describe space and time without much physics baggage. Then I can refer back to this post when I want to add more physics baggage.
It is surprisingly difficult to explain what I mean by “time” and “space” colloquially. The assumptions of time and space permeate language. The most defensible thing I can say is that locations in space are measured by comparing position to measuring tapes, and space is the thing in which you stretch out measuring tapes to measure distance. Time is the thing measured by the number that shows up on a stopwatch that I start when I see some first event and stop when I see a second event. Is it tautological to refer to clocks or measuring tapes? Am I cheating if I use the word “when” to describe events? Am I cheating if I use the word “interval” to describe the space or time in which things happen? As I often answer questions of definition, I will vaguely wave at concepts and hope that the thing in concept space that I’m referring to is well-defined enough that you can understand what I mean.
You live in a house with no inner walls inside a cave for some reason. You can tell when something is inside the house or outside of the house. You decide to fill your house with bricks. You pick a particular type of brick with flat faces that all meet at right angles so you can fit them right next to each other and they looks the same no matter what side you put them down on. You start on one side of the inside of your house and put one brick near the wall. Then you put another brick next to that brick and keep going until you have a line of bricks from one side of the house to the other. Then you could set an identical line of bricks right next to that one and keep doing that until you have a flat layer of bricks all the way across your floor. Then you could stack another layer of bricks on top of that and keep doing that until your house is full and then there would be no room for more bricks. You notice that you had to fill your house in three different directions. First you filled from one side to the other with bricks, then from one side to the other with lines of bricks, then from bottom to top with layers of bricks. But once you filled one layer of bricks, you couldn’t tell anymore what direction you started with. There are lines of bricks running from side to side and from front to back. In fact, there are also lines running from bottom to top!
You pull out a ribbon out and stretch it along one of your lines of bricks from front to back of your house and make a mark on the ribbon at all of the places where it covers up a line between two bricks. Then you stretch it from side to side of your house and find that you can move the ribbon sideways until the lines on your ribbon match the lines between the bricks. On a hunch, you stretch it from bottom to top, and you can do the same thing in that direction. You stretch the ribbon diagonally from top right to bottom left of the front of your house and find that the lines don’t line up with anything much. Your house is now full of bricks, so you build an identical second house right next to your first one. With a feeling of inevitability, you keep building houses until you have a line of houses stretching from one side of the cave to the other. You stop there because you see where this is going.
You realize that you can hold your ribbon up against stuff and count the number of lines between the ends of the object, and that will tell you the number of bricks you could line up end to end next to the object such that the ends of the line of bricks line up with the ends of the object. You call this count length. All of the houses you’ve built have the same length of 140 bricks. You fit 5 houses side by side in your cave. The cave is 700 bricks wide. You realize that you could just as easily have used houses as your base measurement, and then bricks would be (1/140) houses long. But you could fill a cave with houses just like you can fill a house with bricks. You call the thing “space” which can be filled with bricks or houses or caves or anything really. Things that take up space have lengths along any direction, and those lengths may or may not be similar. A brick has the same length along any edge, but a larger length diagonally across a face. The space in your house has a “volume” which can be measured by the number of bricks that fit in it, and it goes like some fraction of the house’s length to the third power. Your neighbor lives in a triangular-based pyramid, and the number of tetrahedrons you need to fill the volume of the pyramid is a different fraction of the third power of the number of tetrahedrons you can line up along one edge, but the volume still goes like the cube of the length.
You want a way to know where in your house you are, so you imagine you have filled your house with bricks. You realize you can uniquely define a brick-sized space in your house by listing out three numbers: the number of bricks you are away from the back wall, the number of lines of bricks you are from the right-facing inside wall, and the number of layers of bricks above the floor you are. You call this set of numbers the position of a brick. You realize you could just as easily started with the number of bricks away from the ceiling, then the number of bricks left of the right wall, then the number of bricks from the back. In fact, you could count bricks starting from the left wall instead, or even start with bricks laid diagonally across your floor, and then fill the floor with lines parallel to that get shorter as you approach the corner and then count coordinates using this diagonal set of lines of bricks. Space doesn't seem to care what direction you measure your lengths to define your position, but it demands at least three different directions to cover a volume. For convenience of bricklaying, you try to use directions which are all mutually perpendicular to each other, but you don't have to.
You do things between when the sun is first visible in the sky and when it disappears on the other side of the sky. You call it a day when you do stuff between the sun being visible and the sun disappearing and notice that from one day to the next you get roughly the same amount of things done per day. Interesting. You notice that every 28 days or so, the moon goes in a complete cycle of not really being visible to the light part growing to circular and then shrinking to nothing again. That sure seems regular. The days get shorter as they get colder and then start getting warmer and longer again. This cycle repeats, and over your life you notice that the cycle predictably lasts a little over 12 moon cycles. You hook a battery up to a u-shaped quartz crystal and notice that it vibrates the same number of times every day. It vibrates that many times times 28 per moon cycle. There is some sort of abstract thing which contains things like a day or a moon cycle or a single quartz vibration. You call it time. The more time you have, the more events you can squeeze into that time.
You measure time by choosing some arbitrary point to start counting quartz vibrations, and then when an event happens you write down how many quartz vibrations you have counted since then. You say that things that happened at a higher number of quartz vibrations happened after things which happened at a lower number. If two things happen while you write a number, you say they happened at the same point in time. You can imagine that before that arbitrary time when you started counting, you could have started measuring, so you can say that negative numbers of quartz vibrations happened to at least in theory account for things that happened before you started counting. You can choose any event you want to be your zero point, you just have to add or subtract some number to or from all of the previous numbers you wrote down for each event and you’ll still be able to tell which events happened before which other events. You call whatever is happening while you write a number the present. You suspect you will count more vibrations and see more events or do more things when you write bigger numbers and you expect to believe that it will also be the present when in time it happens, and you call the segment of time in which more things will happen the future. You don’t know for sure what will happen there until it becomes the present. The stuff that you already know happened is the part of time called the past.
Space and time are very different. You have an obvious order to put events in time and an observer has no control over the time they see on a clock, but you can freely move around wherever you want next to a measuring tape and orient the tape in whatever direction you want in space. Maybe time and length in one direction have some things in common though. If you have a line of bricks, you can specify a direction and say which of any two bricks came before the other. Sure you could have chosen the other direction, but once you choose one you can put yourself in the middle of a line of bricks and say one brick is “here” and other bricks are split into two groups based on what side of you they are on. This is perhaps reminiscent of the “past, present, future” situation in time.
There are physics theories without space and time, but most that I know have some connection to space and time, and space and time work effectively as above. You stick three meter sticks together at right angles to each other and start a stopwatch at the origin and say where things are by writing down the coordinates on the meter sticks which correspond to the locations of the things and the time on the stopwatch when they are there. In field theories like electrodynamics or the standard model, you define the field values at all points in space and time, but those points are still defined by a coordinate system and a clock at rest.
The differences between physics theories come when we consider multiple coordinate systems. Newtonian physics assumes all clocks run at the same rate and objects always have the same length regardless of whether your meter stick is moving differently than the objects. Special relativity says moving clocks run slower than yours and moving objects are shorter than they would be at rest. As a consequence, observers moving relative to each other won’t even agree on whether two events occurred at the same time, much less the distance between the events or what time they occurred. General relativity says clocks close to a lot of mass-energy run slower relative to clocks in free fall far from any mass and also gravitational waves can change the distances between objects slightly as they pass through space. Quantum mechanics says that electrons in a given energy state about a nucleus do not have a definite position about the nucleus anymore. I could write a whole post about each of those sentences.