Purpose

I want to make better discoveries about a research-project topic to use for the comprehensive selection admissions process of the School of Computing at Institute of Science Tokyo (a Japanese university).

Background

Considering the distribution of Less Worng users, there do not seem to be very many Japanese users, so in order to explain the background, I need to give at least a minimal explanation of the Japanese university admissions system.

First, in order to apply to a Japanese national university, students first take something called the Common Test in January. Then, in February, they take an individual university examination (English, mathematics, chemistry, and physics). Some people may wonder why there are two examinations. The Common Test is administered by the national government, whereas the individual examinations are created by each university, and their difficulty and tendencies differ greatly between universities. Each university converts the Common Test score according to a certain weighting, and admission is decided based on the sum of that score and the score from the university’s individual examination.

However, each university also has distinctive admissions methods separate from this. For example, in the admissions method I am considering, the first-stage selection is based only on the Common Test and the evaluation of a research project conducted before the admissions process, and the second-stage selection is based only on an interview of about 30 minutes.

The advantage of this method is obvious. Compared with the former method, the required academic level is relatively lower. Under the ordinary method, approximately 500,000 people take the Common Test every year, and in order to be admitted to a university at the academic level of Institute of Science Tokyo, one would, by a simple calculation, need to be in roughly the top 2–3%. In addition, many students in Japan invest a considerable amount of time into university entrance examinations, so being in the top 2–3% is by no means a low level. To use an analogy, it is like trying to pick a fruit at the top of a Sequoia sempervirens. I want to pick a fruit from a lower tree.

On the other hand, in the admissions method that places emphasis on research projects, although it depends on the school to which one applies, about 40 people apply, around 12 pass the first-stage selection, and finally about 6 are admitted. The level of the research projects of successful applicants is very high for high-school students, but it is by no means an unreachable level, and I am exploring research-project topics that could reach a level capable of being admitted through this route. For that purpose, I want to organize my current thinking and hear opinions from outside. For what I mean by “level” in this post, please refer to this.

As a premise, although this is a selection process for the School of Computing, the range of topics that can be submitted as research projects is quite broad. One may write about mathematical investigations. One may write about what one learned through a programming class that one planned oneself. One may write about what one learned from competitions one participated in, such as competitive programming or the Chemistry Olympiad. It is also acceptable to write about how one’s experience studying abroad led one to think about something connected to the field of computing.

Also, if one passes the first-stage selection, there is an interview, where questions and answers are based on the contents one wrote. Therefore, even though one can use the Internet and AI during the research stage, if one writes something too far removed from one’s own actual ability, it will quickly be exposed. I also expect that such superficial thinking would be quickly seen through and would not even pass the first-stage selection.

There are also restrictions on the submission format of the research project: four pages in a Word file, font size 10 points or larger, with a free format. The research project may be carried out individually by the applicant, individually while receiving guidance from a teacher or others, or collaboratively.

At first, based on the theory of change and original seeing, I conducted observations and analysis based on successful applicants’ accounts and examples published by the university. (LW readers are probably likely to know them, but I feel that those two essays are extremely valuable for producing results, so if you have not read them, I recommend doing so.)

What I learned from the observations and analysis

• First, when the contents of the research projects submitted by successful applicants were reduced to a very rough scale that would be quite crude for rationalists but understandable to anyone, I found that they ranged from early undergraduate to senior undergraduate level.
• I found that a certain degree of novelty in the research content is valued.
• I found that the entire process of discovering a problem in the field oneself, thinking of a solution oneself, and verifying it is valued.
• I found that the attitude of pursuing something one personally wondered about and actually experimenting with it is valued.
• I found that it is valued when what one learned through the research is connected to what one wants to study at university in the future.
• The research content itself does not necessarily have to be advanced. What is highly valued is an attempt to solve an unresolved problem that is practically problematic in that field, and the inclusion of experiences and insights specific to that research.
• I found that it is highly valued when the contents of the report make it possible for the university to judge that the applicant fits the admissions policy issued by the university.

Finding a research project that satisfies many of these conditions is certainly difficult, but there seems to be a certain level that valued research reaches. Conversely, I thought that if I could meet that level, it would help differentiate me from other applicants and increase the probability of passing the first-stage selection.

However, there are also many problems.

• First, the number of research projects on which the observations and analysis above are based is small. This admissions system itself is relatively new when viewed in the history of Japanese university entrance examinations, only about six people are admitted each year, and very few people record their experience for later applicants. In addition, because there is no data in my investigation about what kinds of projects were submitted by people who failed the first-stage selection, the overall level of applicants is unknown, and I expect that there is selection bias among successful applicants in terms of which people choose to publish their projects.
• The rise of AI is also a problem. Applicants are expected to have at least a certain level of interest and ability in information-related fields, and I expect that they are using models at the 5.6 Sol level in some form, although the frequency of use may differ. Applicants can also be expected to have at least some skill in how to use them, so the level of research-project content may be raised substantially. In other words, I expect that rather than how advanced the content is, things that are difficult for AI to substitute for, such as experiments and statistics, will become differentiating factors. Therefore, the difficulty level of research examples produced by the university last year or several years ago may not be very useful for estimating the current competitive level.
• Among the activities I have carried out so far, there is almost nothing that seems likely to reach a sufficient level as a research project. I think that I have accumulated enough insights about crystallized intelligence and fluid intelligence that I might be able to write a fairly interesting article in that area, but this has little primary-research element and has a large literature-review and synthesis aspect, which is exactly the kind of direction in which differentiation is difficult under evaluation standards in the AI era.
• I have nobody I can consult with. In successful applicants’ accounts, they received feedback in some form from university students or researchers through social media such as X (Twitter). However, I do not use X in the first place, and I judged that it would be quite difficult in my current situation to meet someone I could consult about a topic like this, so I decided to post on LW.
• I expect that AI will improve the quality and level of all applicants’ projects, but I do not intend to use AI for anything other than understanding knowledge and theory and proofreading writing. (See this famous article.) In my view, AI is the tool I know whose output usefulness depends most strongly on the context and amount of knowledge possessed by the user. For example, even if the same question is entered, I think readers of this Quick Take will intuitively understand that the usefulness of the answer will differ considerably between the context of someone who has read a large amount of discussion in the rationalist community and the context of someone who usually only uses AI for things like relationship advice. To put it further: if you do not know an answer, you can ask AI and find out. If you know what you need to investigate but do not know the terminology, you can ask AI and find out. But if you do not know that the domain or concept itself exists, you cannot even formulate the question to ask AI. In my own case, for example, it took me quite a long time to learn that the kind of discussions I was looking for were grouped under the name “rationality.” Especially from the Japanese-language sphere, there are very few paths that lead there. That is why I want to study systematically at university.

Research topics I am currently considering

• Could fluid intelligence be substituted for to a considerable extent by crystallized intelligence? Fluid intelligence certainly seems to exist. However, from the perspective of producing results toward some terminal goal, could scaling meta-strategies and external tools eventually allow people to produce roughly the same level of results regardless of differences in individual fluid intelligence?
• This may become an article separate from the research project because it seems likely to be useful to people, but: concrete methods for using spaced learning with Anki to efficiently increase crystallized intelligence in a specific context one wants, and thereby make judgments and think in a way that is pseudo-expert-like or ultimately close to that of an expert. (At least within what I searched, I could not find an LW article that directly dealt with this specific practical method.) This is based on the hypothesis that a considerable part of thinking is supported by words that label concepts. For those who are not very familiar with spaced learning, this is also a well-known article, and I recommend referring to it if you have not read it.
• A thought experiment about creating an AGI specialized for the development of science. TsviBT writes the following in an article: “Current AI got capabilities through a path other than accumulating crystallized intelligence generated by its own fluid intelligence. That is, it got capabilities by, in a broad sense, ‘copying’ human crystallized intelligence that was originally generated by human fluid intelligence. This explanation seems able to account for most of the rapid increase in capability of current AI without invoking the idea that current AI has large fluid intelligence. In addition, given the apparent gap in which humans still remain far ahead in general fluid intelligence, I do not know how we could be very strongly confident that we already possess most or all of the ideas needed for AGI.” I think that human fluid intelligence and crystallized intelligence may ultimately have been formed through observations of the real world. Also, human working memory is limited, and the amount of time available to one human is limited, and there are few people who have simultaneously mastered distant fields such as mathematics and archaeology. Therefore, there may be unexplored transfers of methods between fields, such as meta-strategies that are common in archaeology but almost never used in mathematics. If AI were given an environment in which it could directly observe and experiment on the real world, might it therefore be possible to make new discoveries through cross-field combinations that humans have not sufficiently tried before?
• AI’s output changes greatly for the same question depending on the context it is given, but can variables other than the prompt cause the model side to introduce contexts that the questioner themself does not know? If that is possible, then even if a user is not proficient in the field being asked about, could they obtain answers that are more useful for their objective?

Conclusion

These are still immature ideas, so there may be parts that I have not organized well. I would like people to point out the rough parts.

Anyone who has read this is welcome to point out or advise me about any part of it.

At present, I do not deeply understand each individual field, so if there is a problem currently being worked on in a field that a reader is knowledgeable about and that seems like it could make an interesting research project for a high-school student, I would like as many ideas as possible right now regardless of the field or level, so please tell me in the comments. (I am prepared to study for it.)

At least at the stage of generating research-topic ideas, I have not used AI. This is because in fields where I still possess only shallow context, I can only draw shallow answers from AI, and because if most applicants are using it in some form, there is a possibility that ideas will become homogenized in some way. What kinds of research topics would current AI be unlikely to suggest from a general prompt? Also, what kinds of research activities are difficult to substitute for even with AI? (For example, real-world experiments or surveys involving people.)

Is there a way to turn my current candidates, which lean toward thought experiments and literature reviews, into something closer to primary research?

Is there anyone who is interested in one of the research-topic candidates I am currently considering and would be willing to provide even a little consultation or advice?

In translating from Japanese into English, I used this prompt:

“Translate this Japanese directly into English without changing the content, theory, or body text. Do not add or remove information, and do not improve it through paraphrasing.”