This is a special post for quick takes by Karthik Tadepalli. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
If China is doing espionage against American labs, why don't we see personnel flow from US labs to Chinese labs?
People seem to take it as a fact that there are Chinese spies in US labs. But historically, the main vector for corporate espionage is the flow of personnel. This is because it's the only arguably legal form of corporate espionage, a handy tool to have alongside all the illegal ones. There's no law against recruiting researchers from US labs, and it would be nearly impossible to prove they were leaking secrets. Yet I've seen zero announcements of US lab researchers leaving to work at Chinese labs. Given the high scrutiny on AI researcher movements, I expect any serious flow here would be news. So why is there no such movement?
Obviously the most valuable thing a spy can do is stick around and keep sharing information. But there is likely a contingent of researchers at US labs who would be willing to taking a job at a Chinese lab but would not be willing to act as ongoing assets. I see no reason that China would be unwilling to take advantage of that contingent by recruiting them.
This missing flow seems like evidence against China having significant espionage efforts against US labs.
Maybe one doesn’t announce it prominently when one is leaving a US lab to a Chinese lab for espionage reasons? (This is just a guess, I would be curious to see data on this)
True enough, I just imagined that it would be hard to hide from the prying eyes of AI twitter. But also open to seeing data
I just tested frontier models on a riddle from the podcast Lateral with Tom Scott: "a woman microwaves a chocolate bar once a year. What is her job and what is this procedure for?"
I recall seeing someone do a pretty systematic evaluation of how models did on Lateral questions and other game/connections shows, but with the major drawback that those are retrospective and thus in the training data. The episode with this riddle came out less than a week ago, so I assume not in training data. I also didn't give any context, other than that this was a riddle.
I'm interested in seeing more lateral thinking/creative reasoning tests of LLMs, since I anticipate that's what will determine their ability to make new science breakthroughs. I don't know if there are any out there.
This is a neat question, but it's also a pretty straightforward recall test because descriptions of the experiment for teachers are available online.
Imagine the question was instead inverted to "a physics teacher wants to do an experiment demonstrating the speed of light to children. What would the experiment be?" Now this is straightforward recall because the model can look up what physics teachers do. But in the form that it is, what is the actual lookup that would help solve this question trivially?
The metric is not whether the information to answer this question is available in a model's corpus, but whether the model can make the connection between the question and the information in its corpus. Cases in which it isn't straightforward to make that connection are riddles. But that's also a description of a large class of research breakthroughs – figuring out that X solution from one domain can help answer Y question from another domain. Even though both X and Y are known, connecting them was the trick. That's the ability I wanted to test.
I tried googling to find the answer. First I tried "melting chocolate in microwave" and "melting chocolate bar in microwave", but those just brought up recipes. Then I tried "melting chocolate bar in microwave test", and the experiment came up. So I had to guess it involved testing something, but from there it was easy to solve. (Of course, I might've tried other things first if I didn't know the answer already.)