I found your idea fascinating.  You have a good company too. Percy Liang's group just published a paper along this line of thought and showed transformer's effectiveness in learning "ML trainers":

What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

https://arxiv.org/abs/2208.01066