By Lia Zheng, Karina Chung, Natalia Siwek, and Bright Liu

This work was done as an experiment for Boaz Barak’s “CS 2881r: AI Safety and Alignment” at Harvard. The lecture where this work was presented can be viewed on YouTube here,

For CS 2881r at Harvard, we wanted to explore both sides of this question of economic implications:

How financial markets have reacted to major AI-related announcements, and
How large language models (LLMs) perform and self-assess on tasks of varying “messiness.”

1. Market Reactions to AI Announcements

Motivation

We wanted to evaluate the relation between new model releases and the stock market reactions. We were motivated by assessing to what extent the reactions on social media/academia/general public are related to changes in the market - in particular tech-focused ETFs and the treasury yields.

Process

We compiled a list of 19 major AI-related announcements from November 2022 (ChatGPT) through June 2025 (Gemini 2.5 Pro). For each event, we measured changes in 10-year and 30-year Treasury yields within a ±5 trading day event window. Using data from the FRED API, we compared actual yield movements to a constant model of expected changes, computing cumulative abnormal changes (CACs) to detect unusual movements around announcement dates.

Findings

Phase 1 (2022–2023): The AI Shock.
Announcements like ChatGPT and GPT-4 triggered significant yield drops (as large as 51 basis points) which could be reflecting investor excitement and a belief in AI-driven productivity gains.
Phase 2 (Early 2024): Diminishing Returns.
Yields still moved, but the changes were much less sharp.. Markets seemed to be adjusting their expectations, showing smaller and mixed reactions.
Phase 3 (Mid-2024–2025): Saturation.
By the time of later announcements like Gemini 2.5, yield changes were statistically insignificant. Investors appeared to have fully priced in AI’s long-run productivity story.

Our interpretation is that perhaps markets have just adjusted to AI and the new releases were within the expectations of stakeholders. At the same time, it could be noted that the early major AI announcements coincided with other world events: like FED announcements and the SVB crisis.

2. How LLMs Handle Messy, Real-World Tasks?

Motivation

Secondly we were wondering how we can evaluate the LLM performance on daily human tasks, individuals could often encounter at work. We wanted to acknowl...

Posts

Wikitag Contributions

Comments