Under what circumstances are frontier AIs currently connected to the Internet during training and/or fine-tuning? Do any of the major players building "agents" like Operator do this? I'm aware that offline web hosting is a thing, just not sure if everyone's using it or if some just hook up to a browser and let fly. Specific examples would help tremendously.

Relatedly, when during training do models start writing and getting feedback on code? I'm guessing this is part of standard fine-tuning now for models like Llama, Gemini, Claude, and GPT? Can anyone point to some go-to papers describing the process?