Great question. I am relatively new to the conversation on AI, but have developed several critical physical systems requiring high reliability (in human spaceflight). Could you help me understand why the focus on the spec which I presume would be used in post-training alignment? My instinct is to focus on the pre-training data set instead as the foundation for making these models as useful as possible for critical system work. Using the example you offered, could a focus on Jeff Bezos' training set, (both technical and leadership training, including experi...
Twenty years in aerospace have taught me the importance of understanding the limitations of our engineering models. We build systems where a single failure mode such as combustion instability in a rocket engine or resonance in a wing can be catastrophic. The extreme environment aerospace systems operate in demands rigor, but this same environment also requires taking smart risks to develop new and improved designs. We are unable to test every scenario including combined structural, thermal, acoustic and aerodynamic loading before first flight. This means the success of our first flights depends on strong engineering models. A good model must be technically sound, but since all models are approximations, they must also declare their limits, so we know when we're taking appropriate risks versus simply gambling. As...
This is a fascinating follow up to this important research. Two things stand out to me.
Wow, just discovered this site. So many interesting articles! In regards to this question I sort of wonder if AI will be capable of flexibly verifying mechanical systems without real world interactions in robotic form, "experiencing" gravity, friction, impacts, viscosity etc. This would allow simple real world "tests" like dropping a ball, dragging a backpack, breaking a brick, pouring syrup - kinda the way kids learn about the real world! Without this "experience" I suspect it will take detailed human curated/tailored verification instructions for each sc...
Very interesting work and exciting results. This resonates with something I was thinking about after reading @RogerDearnaley's section in his post about aspects of safety pretraining matching how we raise children. We wouldn't show young children lots of movies about them being super villains, and we probably shouldn't do that with our AI pretraining either (terminator, HAL, etc). At risk of going too far down the parenting analogy here… it is a notable parallel that showing both examples of good and bad is effective (as @RogerDearnaley points out)… and fo... (read more)