Research engineering tips for SWEs. Starting from a more SWE-based paradigm on writing 'good' code, I've had to unlearn some stuff in order to hyper-optimise for research engineering speed. Here's some stuff I now do that I wish I'd done starting out. 

Use monorepos. 

• As far as possible, put all code in the same repository. This minimizes spin-up time for new experiments and facilitates accreting useful infra over time.
• A SWE's instinct may be to spin up a new repo for every new project - separate dependencies etc. But that will not be an issue in 90+% of projects and you pay the setup cost upfront, which is bad. 

Experiment code as a journal. 

• By default, code for experiments should start off' in an 'experiments' folder, with each sub-folder running 1 experiment.
• I like structuring this as a journal / logbook. e.g. sub-folders can be titled YYYY-MM-DD-{experiment-name}. This facilitates subsequent lookup.
  • If you present / track your work in research slides, this creates a 1-1 correspondence between your results and the code that produces your results - great for later reproducibility
• Each sub-folder should have a single responsibility; i.e running ONE experiment. Don't be afraid to duplicate code between sub-folders.   
• Different people can have different experiment folders.
• I think this is fairly unintuitive for a typical SWE, and would have benefited from knowing / adopting this earlier in my career. 

Refactor less (or not at all). 

• Stick to simple design patterns. For one-off experiments, I use functions fairly frequently, and almost never use custom classes or more advanced design patterns.
• Implement only the minimal necessary functionality. Learn to enjoy the simplicity of hardcoding things. YAGNI.
• Refactor when - and only when - you need to or can think of a clear reason.
  • Being OCD about code style / aesthetic is not a good reason.
  • Adding functionality you don't need right this moment is not a good reason.
• Most of the time, your code will not be used more than once. Writing a good version doesn't matter. 

Good SWE practices. There are still a lot of things that SWEs do that I think researchers should do, namely: 

• Use modern IDEs (Cursor). Use linters to check code style (Ruff, Pylance) and fix where necessary. The future-you who has to read your code will thank you.
• Write functions with descriptive names, type hints, docstrings. Again, the future-you who has to read your code will thank you.
• Unit tests for critical components. If you use a utility a lot, and it's pretty complex, it's worth refactoring out and testing. The future-you who has to debug your code will thank you.
  • Gold star if you also use Github Actions to run the unit test each time new code is committed, ensuring main always has working code.
  • Caveat: SWEs probably over-test code for weird edge cases. There are fewer edge cases in research since you're the primary user of your own code.
• Pull requests. Useful to group a bunch of messy commits into a single high-level purpose and commit that to main. Makes your commit history easier to read. 

My current setup

• Cursor + Claude for writing code quickly
• Ruff + Pyright as Cursor extensions for on-the-go linting.
• PDM + UV for Python dependency management

Collaborate via PRs. Sometimes you'll need to work with other people in the same codebase. Here, only make commits through PRs and ask for review before merging.  It's more important here to apply 'Good SWE practices' as described above.