If you think that the net costs of using ML techniques when improving our rationalist/EA tools are not worth it, then there can be some sort of argument there.
Many Guesstimate models are now about making estimates about AI safety.
I'm really not a fan of the "Our community must not use ML capabilities in any form", not sure where others here might draw the line.
My comment was 99% a joke. Though if you used Squiggle to perform an existential risk-reward analysis of whether to use Squiggle, who knows what would happen. :-)
Squiggle Hub is a platform for the creation and sharing of code written inSquiggle. As with Squiggle, Squiggle Hub is free andopen-source.
As a refresher, Squiggle is a simple programming language for probabilistic estimation that runs on Javascript. It begins with the syntax of Guesstimate, but generally adds a lot more functionality. See its launch post here for more information, or the website for the full documentation.
Squiggle Hub is a lot like a more powerful, but less visual, version ofGuesstimate. We hope that it will eventually be much more valuable than Guesstimate is now.
If you can use Guesstimate, you can basically use Squiggle. If you already use Guesstimate, try using the same syntax in Squiggle. It should mostly work.
All models on Squiggle Hub are public. We've produced several small ones so far, and a few friends have written some as well. We're looking forward to seeing what others make!
Write functions that accept and return probability distributions. Squiggle generates automatic plots for these.
You can provide explicit ranges for functions. This helps with the visualization, and ensures they won't be called outside that range. Like, appleStockPrice(t:[2023, 2060]).
Custom plots for distributions and functions. Scales include linear, log, and symlog (like log, but with support for negative values). Symlog scales accept a parameter constant that you can use for adjusting the scale.
Make custom tables of any data and functions, with Table.make({data, columns}).
Automatic conversion of Monte Carlo samples to distribution plots, using KDE. In the cases where the distribution is heavily skewed, Squiggle does this with a log transformation. The result of this is often more accurate than using histograms. Combined with custom scales, Squiggle much better supports highly skewed distributions (i.e. “5 to 5M”) than Guesstimate does.
Squiggle supports most of JSON. You can copy & paste JSON data and begin using it in Squiggle.
Squiggle runs on Javascript. You can simply take Squiggle code and run it in your website,Observable, Obsidian, and more.[1] If you are making an application that uses probability distributions, you can use the Squiggle components directly.
Lots of performance enhancements, library additions, and bug fixes, since Squiggle's initial release.
All the docs and grammar are consolidated here. You can use this to feed into Claude, for some Squiggle generation and assistance.
Squiggle Editor (The window on the left)
With "Autorun", the output will update as you type. Turn this off if you want to run it manually.
Adjust the number of Monte Carlo samples. It defaults to 1000, but you can change this in the settings. Also, there are a bunch of other toggles there to play with.
A built-in Squiggle code formatter. Useful for lengthy files and for storing data.
Function and variable autocomplete.
Squiggle Viewer (The window on the right)
Click the arrows to open and close visualizations.
Select any variable in the right view to "zoom in" on it. Very handy when working on particular estimates of a file!
On hover, there's a "show in editor" button, that finds the source code to the variable in question.
If your Squiggle file ends with a variable, this will be shown as the "Result." You'll still be able to see all of the other defined variables, but these will be collapsed when you open the model. Useful for selecting a few key findings for display purposes.
You can adjust the view settings for specific distributions.
Squiggle strings are rendered in Markdown.
If you write variable-level comments, with /** ... *, these will be displayed by their respective variables. Like with regular strings, these support Markdown.
Squiggle Hub is somewhat straightforward, but very general-purpose. Our main hope is that it will be used for effective altruist use cases, but we imagine there are many ways it could help do so. It will likely take time to see what’s best.
Some examples of how we envision Squiggle Hub being used:
Ovidia writes a Squiggle function that expresses her estimates of the value of technical AI alignment, AI policy, and biosafety, in the next 10 years.
Eskil likes Ovidia’s estimates. He imports them in a model that prioritizes specific donation opportunities. Olivia’s estimates change over time, and Eskil’s correspondingly update.
Karrie expresses her beliefs about the future of AI progress in a set of functions that predict several key stock prices and company valuations, over the next 10 years. Very reasonable assumptions lead to surprising implications for short-term stock purchases.
Katia has a small priorities research organization. They produce a function, pIntervention(human, intervention), that takes in details of a human health intervention for a person, and outputs the expected value of that intervention.
Akmad makes a big list of estimates of the value/disvalue of things he owns and activities he’s considering. He uses those estimates to inform his decisions.
Morten is an ML researcher. He experiments with ML agents that can predict Squiggle functions at scale, helping with human decision-making in a fairly understandable way.
Galla organizes a bunch of data about EA interventions and organizations, and posts this data as JSON objects on Squiggle Hub. This can then be used as a mini-API, and used to update other Squiggle models.
The forecasting platform ForecastsNow supports functional forecasts using Squiggle. People submit squiggle code for things like, “Each stock price at each point in time, for the next 30 years”, and this is formally tracked and scored as a forecast.
Organization ComputeForecastingResearch makes extrapolations of computation over time. They export these estimates as a Squiggle function that returns probability densities. This is later used in other models and formally updated and scored.
Kayla makes a big estimation function of how overconfident all the estimates mentioned above are. Kayla has a trustworthy reputation. Others chose to use these estimates to automatically adjust the others.
Yonah works to make an estimate of the value of all of the estimates mentioned above. The people above adjust their work accordingly.
Future Work
We are planning to add numerous features. This includes:
Models can import code from other models
Note: Even though you can’t formally import code now, you can still copy & paste code from other files.
Multi-file models
Private models
Simple support for years and dates
Integration with apps such as Google Sheets, Guesstimate, Github, and more
Better support for presenting / describing models
We've listed some of our planned features inthis model.
For input on these or any other features, please connect with us onDiscord or through theGithub Discussion.
Relative Values
Squiggle Hub currently supports experimentalrelative values, with future improvements planned. These are currently partially broken and undocumented, we are in the process of improving it.
Theory: A Middle Path between Spreadsheets and Code Editors
Squiggle and Squiggle Hub are an interesting intersection between spreadsheets and programming environments.
Spreadsheets typically are made for:
Broad user bases, much of which doesn’t program.
Large lists of numbers and some numeric calculations.
Editing experiences that feature rapid visual feedback.
User-interactive math models and visualizations.
Meanwhile, programming environments typically are made for:
Large codebases of fairly arbitrary executions.
Programmers, with a fairly high skill floor.
Lots of interactions of libraries and external services.
If there are GUIs, these are generally heavily customized and polished.
In comparison, Squiggle/Squiggle Hub are meant as a “middle path” between these two extremes.
Medium-sized lists/arrays of numbers and numeric calculations.
Editing experiences that feature rapid visual feedback.
Support for small to medium sized codebases. (Maybe <50k lines, including imports?)
Limitations that promote consistency over flexibility. (As with spreadsheets, there are a fixed set of charts you can natively use - but the ones provided work fairly well).
We think that this middle road is both neglected and promising. However, because it’s neglected, it’s taking time to design, and we expect that it will take more time for users to figure out how to best utilize it.
Notebooks, like Jupyter, are one distinct option of a middle path. However, notebooks have significant limitations; they’re nice for editing and demos, but not great for being part of a larger codebase. We might add something like notebooks to Squiggle Hub later on, but wanted to begin with something else first.
Team & Funding
Donations
You can support the development of Squiggle and Squiggle Hub by donating to QURIhere. We are currently actively fundraising for the next year of development. Get in touch if you might be interested!
Squiggle has very much been a collaborative effort. You can see a list of contributorshere. Squiggle Hub has been developed bySlava Matyuhin and Ozzie Gooen.
[1] Right now, the online Observable and Obsidian support is for an old version of Squiggle. We plan on updating this eventually. If this is important for you, let us know.
2