All of ada's Comments + Replies

Gotcha, and thank you so much for writing this post!

Ah! Were you the one who decorated the Rose Garden Inn? I'm really curious how you made the lighting that looks like the sun coming through a cracked door / coming through the cracks between bricks.

Picture below -- I took a ton of pictures when I was there to steal your interior decoration ideas.

I have not worked on the Inn, but the search term for that kind of light is 'recessed channel lighting' :)  (I guess I should mention in the post that my recommendations were rental-space-oriented; Lightcone was able to install this recessed lighting because they own the space and can do whatever they want to it, but it's too invasive for a rental.)

Aside which the original author may be interested in -- there has been some work done to reduce the scaling of the context window below O(n^2) -- e.g. I also think of OpenAI's jukebox which uses a hierarchical strategy in addition to factorized self-attention for generating tokens to effectively increase the context window (