Disallowing a symbol for "all events" breaks the definition of a probability space. It's probably easier to allow extended reals and break some field axioms than figure out do rigorous probability without a sigma-algebra.
Gray Area -- good question, thanks for bring my attention to reservoir sampling.
I found a compact description of it in Devroye's "Non-Uniform ..." and for sampling just 1 integer x it looks as follows
sum_i 1/i diverges, this means x will never stop growing
pdf23ds -- I think you can use reservoir sampling for sampling from infinite streams, an interesting question is when it works. For instance, consider an infinite stream of IID bits, 1-element reservoir sampling converges after 1 ste... (read more)
Interesting post, thanks for the Jaynes link. Related book which is a great read is Szekely's Paradoxes in Probability Theory and Statistics
I think the most intriguing paradoxes are the ones that experts can not agree how to resolve. For instance, take the two envelope paradox: you are presented with two envelopes, one has twice as much money as the other. You are told that first envelope contains x dollars, which envelope should you choose? From expected value calculations, the other envelope has $1.25x which is larger regardless of x. Turns out that the ... (read more)