All of lennart's Comments + Replies

What is Compute? - Transformative AI and Compute [1/4]


  • I'm working with a colleague on the trends of the three components (compute, memory, and interconnect) over time of compute systems and then comparing it to our best estimates for the human brain (or other biological anchors). However, this will still take some time but I hope we will be able to share it in the future (≈ till the end of the year).
2Gunnar_Zarncke2moCool. Looking forward to it.
What is Compute? - Transformative AI and Compute [1/4]

Thanks for the correction and references. I just followed my "common sense" from lectures and other pieces.

What do you think made AlexNet stand out? Is it the depth and use of GPUs?

1teradimich2moI do not know the opinions of experts on this issue. And I lack competence for such conclusions, sorry.
What is Compute? - Transformative AI and Compute [1/4]

Thanks for the feedback, Gunnar. You're right - it's more of a recap and introduction. I think the "newest" insight is probably the updates in Section 2.3.

I also would be curious to know in which aspects and questions you're most interested in.

2Gunnar_Zarncke2moThe update in 2.3 was a valuable update. Based on the title (and my interests) I was hoping for * some integration of the limits for compute, memory, and interconnect. Like you say they limit each other but it is not very clear how the limits interrelate and scale with each other. Empirically, it would be interesting to see the relative sizes of these parts over time. * some comparison of the relative sizes of the human brain responsible for processing where we do have algorithms that are comparable to what the brain does, e.g. image processing and object and scene detection in the visual cortex.