If “realistically simulating human impacts in silico” means something like high-fidelity models of how humans respond psychologically to AI interaction, that seems like one of the higher-risk cases of safety research also being capabilities research. A good sim for "how AI affects humans psychologically" is a good sim for "how to affect humans psychologically, using AI".
The hardest part of CogSec evals is figuring out an eval meaningful enough to detect dangerous effects, while not being meaningful enough to hill climb on. Maybe eval is too benchmark code...
From my research, the current meta is the Luggable https://www.cleanairkits.com/products/luggables. Basic insight is that by going from a 99.97% removal HEPA filter to a 93% MERV-13, you can run your filters at a higher speed, quieter, with much better power efficiency, and cycle air through the filter much more quickly due to the higher fan speed. Most HEPA grade filtration misses the mark because they're far too loud running at full blast, so a quieter, faster, less efficient filter actually does a better job. The LuggableXL is about as loud
The one missi...
Downstream unit testing
Instead of autogenerating the unit tests to fill any gaps, we can see if software that depends on it still works.
Simon Willison discusses an early version of this employed by the StrongDM team. For context, they build Digital Twins of all the software their system depends on, and have agents run QA testing continually against those digital twins. To verify correctness of the digital twins, one technique they use is verifying client libraries that make use of their dependencies still work. Direct quote from the article:
...I did have an i
The social/logistical aspects of cybersecurity vulnerabilities will accelerate greatly due to AI. I'd expect the response from tech-savvy organizations will be to increase the pace of software delivery - a long standing trend for other reasons. Continuous deployment, forced autoupdates, focused research on fraud and suspicious activity detection.
The main risks are around organizations that structurally cannot increase their pace. Think banks, aviation, medical systems, drug manufacturing, areas where because the risks of vulnerabilities/defects has histori...
I'd love to see anonymized version of one of these project documents. For example, with your plan to victory, do you assign actual dates to each concrete step? Or just assign different sections of the plan into milestones that have actual dates? What form factor works? Is a checklist okay, or should it be a spreadsheet?
Regardless, fantastic article, this type of tacit knowledge is usually only made explicit for those within the walls of major companies, so it's nice to have as a reference.
Even ignoring AI, current expectations are that world population will peak in ~60 years at a population of ~10 billion. This seems totally workable with relatively incremental improvements in technology for resource management, energy extraction, and habitat preservation. This is only bolstered by much of our future energy supply growth coming from non-extractive solar rather than fossil fuels, which we continue to find new sources of regardless.
When you add on the fact that almost all population growth in the next 50 years is due to occur in sub-Saharan ... (read more)