It's possible that "teaching to the test" tends to refer to something a bit more specific.  Here is John Holt in "How Children Fail", which some upstanding citizen has put onto the internet in easily googleable form:

This past year I had some terrible students. I failed more kids, mostly in French and Algebra, than did all the rest of the teachers in the school together. I did my best to get them through, good-ness knows. Before every test we had a big cram session of practice work, politely known as "review." When they failed the exam, we had post mortems, then more review, then a makeup test (always easier than the first), which they almost always failed again.

Much later:

We teachers, from primary school through graduate school, all seem to be hard at work at the business of making it look as if our students know more than they really do. Our standing among other teachers, or of our school among other schools, depends on how much our students seem to know; not on how much they really know, or how effectively they can use what they know, or even whether they can use it at all. The more material we can appear to "cover" in our course, or syllabus, or curriculum, the better we look; and the more easily we can show that when they left our class our students knew what they were "supposed" to know, the more easily can we escape blame if and when it later appears (and it usually does) that much of that material they do not know at all.

When I was in my last year at school, we seniors stayed around an extra week to cram for college boards. Our ancient-history teacher told us, on the basis of long experience, that we would do well to prepare ourselves to write for twenty minutes on each of a list of fifteen topics that he gave us. We studied his list. We knew the wisdom of taking that kind of advice; if we had not, we would not have been at that school. When the boards came, we found that his list comfortably covered every one of the eight questions we were asked. So we got credit for knowing a great deal about ancient history, which we did not, he got credit for being a good teacher, which he was not, and the school got credit for being, as it was, a good place to go if you wanted to be sure of getting into a prestige college. The fact was that I knew very little about ancient history; that much of what I thought I knew was misleading or false; that then, and for many years afterwards, I disliked history and thought it pointless and a waste of time; and that two months later I could not have come close to passing the history college boards, or even a much easier test, but who cared?

I have played the game myself. When I began teaching I thought, naively, that the purpose of a test was to test, to find out what the students knew about the course. It didn't take me long to find out that if I gave my students surprise tests, covering the whole material of the course to date, almost everyone flunked. This made me look bad, and posed problems for the school. I learned that the only way to get a respectable percentage of decent or even passing grades was to announce tests well in advance, tell in some detail what material they would cover, and hold plenty of advance practice in the kind of questions that would be asked, which is called review. I later learned that teachers do this everywhere. We know that what we are doing is not really honest, but we dare not be the first to stop, and we try to justify or excuse ourselves by saying that, after all, it does no particular harm. But we are wrong; it does great harm.

It does harm, first of all, because it is dishonest and the students know it. My friends and I, breezing through the ancient-history boards, knew very well that a trick was being played on someone, we were not quite sure on whom. Our success on the boards was due, not to our knowledge of ancient history, which was scanty, but to our teacher's skill as a predictor, which was great. Even children much younger than we were learn that what most teachers want and reward are not knowledge and understanding but the appearance of them. The smart and able ones, at least, come to look on school as something of a racket, which it is their job to learn how to beat. And learn they do; they become experts at smelling out the unspoken and often unconscious preferences and prejudices of their teachers, and at taking full advantage of them. My first English teacher at prep school gave us Macaulay's essay on Lord Clive to read, and from his pleasure in reading it aloud I saw that he was a sucker for the periodic sentence, a long complex sentence with the main verb at the end. Thereafter I took care to construct at least one such sentence in every paper I wrote for him, and thus assured myself a good mark in the course.

Not only does the examination racket do harm by making students feel that a search for honest understanding is beside the point; it does further harm by discouraging those few students who go on making that search in spite of everything. The student who will not be satisfied merely to know "right answers" or recipes for getting them will not have an easy time in school, particularly since facts and recipes may be all that his teachers know. They tend to be impatient or even angry with the student who wants to know, not just what happened, but why it happened as it did and not some other way. They rarely have the knowledge to answer such questions, and even more rarely have the time; there is all that material to cover.

In short, our "Tell-'em-and-test-'em" way of teaching leaves most students increasingly confused, aware that their academic success rests on shaky foundations, and convinced that school is mainly a place where you follow meaningless procedures to get meaningless answers to meaningless questions.

And also:

It begins to look as if the test-examination-marks business is a gigantic racket, the purpose of which is to enable students, teachers, and schools to take part in a joint pretense that the students know everything they are supposed to know, when in fact they know only a small part of it--if any at all. Why do we always announce exams in advance, if not to give students a chance to cram for them? Why do teachers, even in graduate schools, always say quite specifically what the exam will be about, even telling the type of questions that will be given? Because otherwise too many students would flunk. What would happen at Harvard or Yale if a prof gave a surprise test in March on work covered in October? Everyone knows what would happen; that's why they don't do it.

It's hard to write good tests. It's even harder to write good standardized tests that have to be administered to huge numbers of people, within a reasonable time, in purely written form, and graded consistently, again within reasonable time^[1].

... and there are a ton of practical and political pressures that make it even harder.

People don't actually agree on what they're trying to teach, why it's important to teach it, what constitutes good performance, what's essential versus what's "extra", what it's fair to test, what forms even of practically accessible testing are fair to use, etc. That list of critical competencies in the state curriculum wasn't handed down by God. It was negotiated, and it's at best a poor approximation of what anybody wants. And there was still more negotiation involved in making a test out of it.

Here's an example of a practical and maybe political limitation: If you have a diagnosed learning disability, certified by somebody with the right piece of paper on their wall, you can (often) get extra time on tests. But you still have to take the tests, because the bottom line is that, disability or no, you still have to learn the material. You don't get excused from actually getting the required understanding, even if it's harder for you. You're just given the opportunity to show that you can do it anyway.

The Good and the Great have agreed among themselves that, after all, what they care about is your understanding of, say, this or that kind of math, which they believe is best demonstrated by your ability to solve whatever problems are on the test. But they don't really care about speed. It's not a footrace, and nobody's training (or hiring) people for an assembly line. If anybody in 2025 cared about actually solving the problems fast, or indeed accurately, they'd use a machine to do it anyway^[2].

So we've established that speed doesn't matter, at least not enough to withhold grades or credentials over it. It's not a critical part of the "X" you're trying to test.

OK, then why was the test timed in the first place? You don't care about speed, remember? So why doesn't everybody taking the test get all the time they feel they need? Your test is measuring the wrong thing!

The main real answer is probably that it'd be costly and administratively inconvenient to reserve a big room for, say, twice as long, and costly and personally inconvenient for somebody to sit there and proctor for that long. The second real answer is probably that timed tests are traditional, dammit.

So if it's even possible to do what you're suggesting, it's going to be a lot of work. It's going to have a cost. You'll have to give something up to do it. Before you do that, you should know what value you're getting in exchange.

So what value is this fancy test actually supposed to produce, once you have it?

A more abstract reason why this irritates me is that it is an example of a more general pattern that goes like this:

• Try a half-assed implementation of X.
• It fails predictably.
• Conclude: "X was experimentally disproved."
• If anyone suggests to do X properly, say: "true socialism has never been tried, comrades, right?"

Sometimes I wonder whether each popular fallacy has its opposite which is also a popular fallacy.

This got me thinking, I really wish school had looked more like unlocking a tech tree in a game like Civilization than completing ordered stages in a game like Super Mario Brothers (with occasional warps/shortcuts for the clever).

Maybe with AI tutors it can. Study a subject. Take a test. If you pass, (and passing here means demonstrating mastering, not getting some score that's "good enough" to meet some standard for much you have to know to be allowed to move on to the next grade with all your peers even though you clearly don't have full mastery of the topics) you move on, and if you don't, you keep studying until you can master the test. No need to learn subjects in lockstep with same-sage peers, because everyone can be studying what they need to learn with their own, independent AI teacher.