Consider an agent with a model of the world W. How does W relate to the real world. W might contain a chair. In order for W to be useful it needs to map to reality, i.e. there is a function f with W_chair ↦ R_chair.
The pointers problem is about figuring out f.
In John's words (who introduced the concept here):...