How much video data is there? It seems there is plenty:
This https://www.brandwatch.com/blog/youtube-stats/ says 500 hours of video are uploaded to youtube every minute. This https://lumen5.com/learn/youtube-video-dimension-and-size/ says standard definition for youtube video is 854x480 = 409920 pixels. At 48fps, that’s 3.5e13 pixels of data every minute. Over the course of a whole year, that’s +5 OOMs, it comes out to 1.8e19 pixels of data every year. So yeah, even if we use some encoding that crunches pixels down to 10x10 vokens or whatever, we’ll have 10^17 data points of video. That seems like plenty given that according to the scaling laws and the OpenPhil brain estimate we only need 10^15 data points to train an AI the size of a human brain. (And also according to Lanrian's GPT-N extrapolations, human-level performance and test loss will be reached by 10^15 or so.)
But maybe I've done my math wrong or something?
We can discuss more, I think I know how we will "get there from here" in broad strokes. I don't think it will be done by someone writing a relatively simple algorithm and getting a sudden breakthrough that allows for sentience, I think it will be done by using well defined narrow domain agents that each do something extremely well - and by building higher level agents on top of this foundation in a series of layers, over years to decades, until you reach the level of abstraction of "modify thy own code to be more productive"... (read more)