LESSWRONG
LW

SatvikBeri's Shortform

by SatvikBeri
16th Aug 2024
1 min read
1

6

This is a special post for quick takes by SatvikBeri. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
SatvikBeri's Shortform
42SatvikBeri
1 comment, sorted by
top scoring
Click to highlight new comments since: Today at 9:35 AM
[-]SatvikBeri1y4210

It's really useful to ask the simple question "what tests could have caught the most costly bugs we've had?"

At one job, our code had a lot of math, and the worst bugs were when our data pipelines ran without crashing but gave the wrong numbers, sometimes due to weird stuff like "a bug in our vendor's code caused them to send us numbers denominated in pounds instead of dollars". This is pretty hard to catch with unit tests, but we ended up applying a layer of statistical checks that ran every hour or so and raised an alert if something was anomalous, and those alerts probably saved us more money than all other tests combined.

Reply
Moderation Log
More from SatvikBeri
View more
Curated and popular this week
1Comments