15

LESSWRONG
LW

14
Probability & Statistics
Personal Blog

41

Covariance in your sample vs covariance in the general population

by RomeoStevens
16th May 2012
1 min read
3

41

Probability & Statistics
Personal Blog

41

Covariance in your sample vs covariance in the general population
8othercriteria
7Randaly
0jsalvatier
New Comment
3 comments, sorted by
top scoring
Click to highlight new comments since: Today at 1:43 PM
[-]othercriteria13y80

Sampling effects like this can be really pernicious for network data (and I imagine similarly for other dependent data). It can be difficult to tell if a network is scale-free from observing a subnetwork [1] or impossible to learn an ERGM (basically, a maximum entropy distribution with graph properties as its statistics) from a subnetwork [2].

[1] M. P. H. Stumpf, C. Wiuf, and R. M. May, “Subnets of scale-free networks are not scale-free: sampling properties of networks,” Proceedings of the National Academy of Sciences of the United States of America, vol. 102, no. 12, p. 4221, 2005.
[2] C. Shalizi, “Consistency under Sampling of Exponential Random Graph Models,” arXiv.org. 2011.

Reply
[-]Randaly13y70

Incidentally, Pearl's original explanation in Chapter 1 of Causality is here; the whole first edition of the book is available online here.

Reply
[-]jsalvatier13y00

That was quite good.

Reply
Moderation Log
More from RomeoStevens
View more
Curated and popular this week
3Comments

A popular-media take on a subtle problem in sampling.  I found the graph quite illustrative.

http://www.theatlantic.com/business/archive/2012/05/when-correlation-is-not-causation-but-something-much-more-screwy/256918/