Akhil Rasheed

I think I understand where you're coming from now, but I'm still a little confused by the wording that the target network has access to more information.

My understanding is that the replay buffer contains the full transition , so the algorithm already has access to both and . During training, the online network is evaluated on , while the target network is evaluated on . To me, this feels like the two networks are simply evaluating different inputs, rather than one network possessing information that the other lacks. In fact, if we replaced the target net... (read more)

I am still confused by this wording, it seems to heavily imply that the target network has access to some privileged information, or has seen more data.