📄 Full paper on arXiv: A Coherence-Based Measure of AGI

True general intelligence should reflect coherent sufficiency: balanced competence across all essential domains. To capture this, we propose a coherence-aware measure of AGI based on integrating generalized means across compensability regimes.
This “area-under-the-curve” (AUC) approach penalizes imbalance and tracks robustness under varying assumptions.

Applied to published CHC-based domain scores (Hendrycks et al. (2025)) for GPT-4 and GPT-5, the coherence-adjusted AUC reveals that both remain far from general competence (e.g., GPT-5 ≈ 24% despite an arithmetic score of 58%).

1. Motivation

Hendrycks et al. (2025) formalized AGI as the arithmetic mean of performance across ten cognitive domains derived from the Cattell–Horn–Carroll (CHC) model of human cognition — reasoning, memory, perception, speed, etc.

While elegant, this assumes that exceptional ability in some domains can hide total failure in others. An AI with 100% reasoning but 0% memory could still appear “general.” But, that’s not how human intelligence or robust systems work.

In complex systems the overall function is limited by weak components. That is the case of human intelligence, which depends on coherence among faculties. So general intelligence should measure balance and interdependence, not isolated peaks.

2. The Idea: From Arithmetic to Generalized Means

Let $s_i$ be normalized proficiency in domain i.

Hendrycks et al. use:

$A G I_{1} = \frac{1}{n} \sum i s_{i}$

We generalize this into the continuous family:

$A G I_{p} = {(\frac{1}{n} \sum i s_{i}^{p})}^{1 / p}$

p=1: arithmetic mean (fully compensatory)
p=0: geometric mean (moderate coupling)
p=−1: harmonic mean (non-compensatory)
p→−∞: strict bottleneck (min score)

As p decreases, compensation becomes less allowed; weakness in one domain matters more.

Thus, to summarize robustness across regimes, we integrate over $p∈ \in [-1,1]$:

$A G I_{A U C} = \int_{- 1}^{1} A G I_{p} d p$

This yields a single scalar capturing coherence under varying compensability assumptions.

3. Results (GPT-4, GPT-5)

Figure 1: Model performance across aggregation exponents $p$. The shaded area under each curve (AUC) represents performance and coherence under varying compensability assumptions.

Even GPT-5, which looks impressive by the arithmetic metric ($p=1$), collapses under coherence-based evaluation. Weaknesses in long-term memory and adaptive reasoning dominate. The AUC reve...

Posts

Wikitag Contributions

Comments