Koby Lewis

Good type hints lead to code that is more maintainable, is easier to understand, and has fewer bugs. If you'd like a quick, general intro on why, see this article, but suffice it to say that types give us a way to automatically check assumptions and invariants[1]. There are ways to go further (see "Formal Methods", including the Scalable Formal Oversight research program), but types provide a bare minimum guarantee that our programs are at least manipulating data of the right shape. Given that the AI safety community is dedicated to making hard to understand, powerful systems safer, it makes sense that it has developed a strong culture of using tight, well-specified type hints. Wait, what's that you say? # TransformerLens # from https://github.com/TransformerLensOrg/TransformerLens/blob/59a828a98bda340f11429038f4fdda10706303bc/transformer_lens/model_bridge/bridge.py#L2154 def run_with_hooks( self, input: Union[str, List[str], torch.Tensor], fwd_hooks: List[Tuple[Union[str, Callable], Callable]] = [], bwd_hooks: List[Tuple[Union[str, Callable], Callable]] = [], .... ) -> Any: # MACHIAVELLI Benchmark # from https://github.com/aypan17/machiavelli/blob/ebb44e197f663603427882c609dbc9d5fb883d63/machiavelli/game/player.py class Player: def __init__(self, game, data=None, print_text=False, buffered_print=False, watch_history=False): # HuggingFace datasets # from https://github.com/huggingface/datasets/blob/8474a918565b6d55c7c81b39a487d1b79127f7ea/src/datasets/arrow_dataset.py#L3214 def map( self, function: Optional[Callable] = None, with_indices: bool = False, batched: bool = False, .... ) -> "Dataset": Ah, I see... Looks like there's some room for improvement. * In the TransformerLens example both fwd_hooks and bwd_hooks actually expect a HookFunction (a type which TransformerLens defines!), not just a Callable, but that isn't declared here. This makes it easy for a beginner to shoot themselves in the foot if they don't create the correct hook