Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well. TL;DR Documented communications of risks (especially by employees) make companies much more likely to be held liable in court when bad things happen. The resulting Duty to Due Diligence from Discoverable Documentation of Dangers (the 6D effect) can make companies much more cautious if even a single email is sent to them communicating a risk. Companies tend to avoid talking about risk through documented media. Companies often intentionally avoid discussing the risks of what they are doing through permanent media such as email. For example, this article gives some very shady advice on how companies can avoid liability by using “safe communication” practices to avoid the creation of incriminating “bad documents”. > Often the drafters of these documents tend to believe that they are providing the company with some value to the business. For example, an engineer notices a potential liability in a design so he informs his supervisor through an email. However, the engineer’s lack of legal knowledge and misuse of legal vocabulary in the communication may later implicate the company with notice of the problem when a lawsuit arises. I personally enjoyed the use of “when” and not “if” in the excerpt. This is a perverse consequence of how it is relatively hard for companies to be held liable for risks when it cannot be proven they knew about them, even if they did. When an incident happens and a company is sued, evidence about its role in the problem is gathered during what is known as the “discovery” phase of a lawsuit (emails are usually discoverable). When records showing that a company had knowledge of the problem are found in discovery, they are much more likely to be found liable. One em