10+ years in Machine Learning Infrastructure Engineering - My perspective:
Intelligence in systems (human, AI) can be conceptualized as the resolution and throughput at which a system can process and affect Shannon information. This perspective emphasizes not just the quantity of information processed (throughput), but also the depth and detail with which it is handled (resolution), constrained to thermodynamic limitations.
Aside from improving the thermodynamics of the system here are the 7 levers we can use to improve its intelligence:
Physical Capacity: A system's intelligence increases as it expands its physical limits. This encompasses augmenting processing units (akin to neurons in humans or parameters in AI), improving thermal regulation, and maximizing energy throughput. Such enhancements enable a system to process information at a higher resolution and throughput.
Cooperation: When entities collaborate, the collective intelligence of the system increases. This is due to the improved resolution at which information can be processed and influenced, a principle manifest in ensemble methods in AI where multiple models aggregate their insights.
Conflict: The presence of conflict within or between systems can lead to an increase in intelligence. The necessity to adapt for survival and resolve conflicts escalates energy expenditure, which in turn refines the system's ability to process and affect information at a greater resolution and throughput.
Attention: Enhancing the range, depth, and sampling rate of information a system can process boosts its intelligence. This increase is achieved by allowing the system to operate with a wider context and more frequent assessments of information, thereby processing it at a higher resolution.
Actuation: Increasing the scope and precision of a system's actions directly impacts its intelligence. More diverse and precise actuation improves the system's capacity to affect information at a finer resolution.
Memory (Past): Building and utilizing shared physical memory elevates a system's intelligence by enabling it to process information over time at a higher resolution, fostering a more nuanced understanding of historical data.
Predictors (Future): The intelligence of a system can be significantly increased by evolving its core predictive model architecture. This includes developing new AI-designed architectures or employing techniques like neuroevolution, where algorithms evolve and optimize neural networks. Such advancements not only enhance the system's predictive capabilities but also improve the resolution and throughput at which it can process and affect future outcomes. By continually refining the architecture, the system becomes adept at anticipating and influencing future scenarios with greater accuracy and efficiency.
Specific Measures to Enhance GPT's Capabilities:
Increase Memory: Pretty obvious but nuanced approaches
Expand Actuation: More actions. Not plugins, but an open API marketplace not for humans but AI.
Enhance Cooperation: More ensembling, Open ensembling protocol, 'undisclosed secret sauce'.
Boost Physical Capacity: Pretty obvious, more Closed-source parameters, or Giant P2P networks (Petals.ML alternative)
Intensify Conflict: Employing more discriminators, ‘undisclosed secret sauce' .
Augment Attention: This is challenging due to energy limits
Better Predictors: Increasing the length, scope and sophistication of predictive models (Neuroevolution)
10+ years in Machine Learning Infrastructure Engineering - My perspective:
Intelligence in systems (human, AI) can be conceptualized as the resolution and throughput at which a system can process and affect Shannon information. This perspective emphasizes not just the quantity of information processed (throughput), but also the depth and detail with which it is handled (resolution), constrained to thermodynamic limitations.
Aside from improving the thermodynamics of the system here are the 7 levers we can use to improve its intelligence:
Physical Capacity: A system's intelligence increases as it expands its physical limits. This encompasses augmenting processing units (akin to neurons in humans or parameters in AI), improving thermal regulation, and maximizing energy throughput. Such enhancements enable a system to process information at a higher resolution and throughput.
Cooperation: When entities collaborate, the collective intelligence of the system increases. This is due to the improved resolution at which information can be processed and influenced, a principle manifest in ensemble methods in AI where multiple models aggregate their insights.
Conflict: The presence of conflict within or between systems can lead to an increase in intelligence. The necessity to adapt for survival and resolve conflicts escalates energy expenditure, which in turn refines the system's ability to process and affect information at a greater resolution and throughput.
Attention: Enhancing the range, depth, and sampling rate of information a system can process boosts its intelligence. This increase is achieved by allowing the system to operate with a wider context and more frequent assessments of information, thereby processing it at a higher resolution.
Actuation: Increasing the scope and precision of a system's actions directly impacts its intelligence. More diverse and precise actuation improves the system's capacity to affect information at a finer resolution.
Memory (Past): Building and utilizing shared physical memory elevates a system's intelligence by enabling it to process information over time at a higher resolution, fostering a more nuanced understanding of historical data.
Predictors (Future): The intelligence of a system can be significantly increased by evolving its core predictive model architecture. This includes developing new AI-designed architectures or employing techniques like neuroevolution, where algorithms evolve and optimize neural networks. Such advancements not only enhance the system's predictive capabilities but also improve the resolution and throughput at which it can process and affect future outcomes. By continually refining the architecture, the system becomes adept at anticipating and influencing future scenarios with greater accuracy and efficiency.
Specific Measures to Enhance GPT's Capabilities:
Increase Memory: Pretty obvious but nuanced approaches
Expand Actuation: More actions. Not plugins, but an open API marketplace not for humans but AI.
Enhance Cooperation: More ensembling, Open ensembling protocol, 'undisclosed secret sauce'.
Boost Physical Capacity: Pretty obvious, more Closed-source parameters, or Giant P2P networks (Petals.ML alternative)
Intensify Conflict: Employing more discriminators, ‘undisclosed secret sauce' .
Augment Attention: This is challenging due to energy limits
Better Predictors: Increasing the length, scope and sophistication of predictive models (Neuroevolution)