some personal beliefs I've updated on recently

• building model organisms is hard
  • things often don't work for non-obvious reasons
    • good OOD generalization often depends on having a good dataset
    • finetuning on small datasets often has lots of side effects
  • it's valuable to iterate on bigger models where possible
• evaluating model organisms is hard
  • the main bottleneck seems to be knowing 'what' to eval for
    • it helps to just have a ton of pre-existing evals ready to go
    • brainstorming with friends / mentors is also invaluable