All of Jader Martins's Comments + Replies

Is GPT-3 already sample-efficient?

The sample efficiency is not a formal claim, like, RL algorithms are claimed to be sample inefficient as only takes 10 games of Pacman to a human get good at it, but we can't isolate this knowledge in human brain. The point a human learns to play Pacman it already learned many things, like GPT-3, and we don't know what things contribute to playing Pacman, is it motor skills? spacial skills? Knowing all the skills that enable human to play Pacman in only ten games and passing this as a pre-training for the RL algorithm then training it to play Pacman would be a fair comparison of how sample efficient it is. The same applies for the names example, could we really measure how many times a human heard a name or maybe a similar name?

Money isn't real. When you donate money to a charity, how does it actually help?

I don't like the term "money is a shared illusion" (as it misguides the real concept), money is the materialization of credit. The history of money is pretty long to this comment, but to summarize: money is the idea that someone owns you some amount of resources (work/material/etc), and as the society agreed with this common denominator (money), so the society also owns you some amounts of resources.

Price formation (or price system) consider a lot of things, geographical resources, comparative advantages, technology and many more, all this k... (read more)

-1aleph_four2y
woah, a marvelous inversion
What types of compute/processing could we distinguish?

"Why can I have little information but still have to search a huge state space, why can't I go straight to the conclusion/action?"


There's no answer for this question, if you find one, pick your prize of 1million dollars:

https://en.wikipedia.org/wiki/P_versus_NP_problem


Here is a video explaining it better: https://www.youtube.com/watch?v=YX40hbAHx3s

1MoritzG2y
Thank you, I should have thought of it in that (Time complexity) context. Time complexity is not just about how long it takes but also about the nature of the problem. Chess is neither P nor NP, but the question of complexity is certainly related. Maybe my question is: Why can there be a Heuristic that does fairly well and is nowhere near exponential? Even a count of the pieces left on the board usually says something that only a full search can prove.
How do you do hyperparameter searches in ML?

Usually the RandomSearchCV, I've tryied the bayesian optimization from skopt, but empirically I did not see advantages in this over random search, not sure if I used it wrong, someone with good results?

2lsusr2y
I think it depends on your problem. If you have lots of compute power, high dimensionality and powerful higher-order emergent behavior from hyperparameters then Bayesian optimization makes sense. And vice-versa.