samhealy

(Your response and arguments are good, so take the below in a friendly and non-dogmatic spirit)

Good enough for what?

Good enough for time-pressed people (and lazy and corrupt people, but they're in a different category) to have a black-box system do things for them that they might, in the absence of the black-box system, have invested effort to do themselves, and as an intended or unintended result, increased their understanding, opening up new avenues of doing and understanding.

We're still in the "wow, an AI made this" stage.

I'm pretty sure we're currently exiting the "wow, an AI made this" stage, in the sense that 'fakes' in some domains are approaching undetectability.

We find that people

... (read more)

Replying toPlease Understand

I don't think I'm selling what you're not buying, but correct me if I misrepresent your argument:

The post seems to assume a future version of generative AI that no longer has the limitations of the current paradigm which obligate humans to check, understand, and often in some way finely control and intervene in the output...

Depending on your quality expectations, even existing GenAI can make good-enough content that would otherwise have required nontrivial amounts of human cognitive effort.

but where that tech is somehow not reliable and independent enough to be applied to ending the world...

Ending the world? Where does that come in?

and somehow we get this long period where we get to feel

... (read more)

Replying toPlease Understand

For what it's worth, I think even current, primitive-compared-to-what-will-come LLMs sometimes do a good job of (choosing words carefully here) compiling information packages that a human might find useful in increasing their understanding. It's very scattershot and always at risk of unsolicited hallucination, but in certain domains that are well and diversely represented in the training set, and for questions that have more or less objective answers, AI can genuinely aid insight.

The problem is the gulf between can and does. For reasons elaborated in the post, most people are disinclined to invest in deep understanding if a shallower version will get the near-term job done. (This is in no way unique to... (read more)

In which a case is made for worrying about the AI Prompt Box.

Preamble

Technology serves to abstract away nonessential aspects of creative activities, giving us more direct access to their conceptual cores. Few audio engineers pine for the days of flaky reel-to-reel tape machines that unspool at the worst moments; few graphic designers long to swap their Macbooks for bulky old photostat rigs; few mathematicians grieve for the sliderule or the log table.

Yet domain understanding survived those leaps to digital abstraction. Music producers working entirely 'in the box' still know and/or intuit dynamics, frequency equalisation, melody and harmony. Photoshop natives still know and/or intuit colour theory, visual communication, the rules of composition. Recent... (read 1583 more words →)

Replying toAIOS

samhealy2y*

To clarify: I didn't just pick the figures entirely at random. They were based on the below real-world data points and handwavy guesses.

ChatGPT took 3.23 x 10^23 FPOPs to train
ChatGPT has a context window of 8K tokens
Each token is roughly equivalent to four 8-bit characters = 4 bytes, so the context window is roughly equivalent to 4 x 8192 = 32KB
The corresponding 'context window' for AIOS would need to be its entire 400MB+ input, a linear scaling factor of 1.25 x 10^4 from 32KB, but the increase in complexity is likely to be much faster than linear, say quadratic
AIOS needs to output as many of the 2 ^ (200 x 8 x

... (read more)

Replying toAIOS

Agreed, largely.

To clarify, I'm not arguing that AI can't surpass humanity, only that there are certain tasks for which DNNs are the wrong tool and a non-AI approach is and possibly always will be preferred.

An AI can do such calculations the normal way if it really needs to carry them out

This is a recapitulation of my key claim: that any future asymptotically powerful A(G)I (and even some current ChatGPT + agent services) will have non-AI subsystems for tasks where precision or scalability is more easily obtained by non-AI means, and that there will probably always be some such tasks.

Replying toAIOS

Plucked from thin air, to represent the (I think?) reasonably defensible claim that a neural net intended to predict/synthesise the next state (or short time series of states) of an operating system would need to be vastly larger and require vastly more training than even the most sophisticated LLM or diffusion model.

-8

Summary

A reductio ad absurdum of the position that deep neural networks, given enough time, can do literally anything. In playful Socratic dialogue form.

Welcome to AIOS!

Snay/Coyle Systems is proud to announce AIOS.

Its notional 'clock speed' is 60Hz, representing a typical screen's refresh rate and a typical human interface device's polling rate. But it's no slouch!

Its outputs are just over 1.67 billion bits, representing up to 200MB of arbitrary data to be sent across a 100Gbps Ethernet connection per tick, a 4K video frame buffer/monitor output, and 0.0166 seconds of multi-channel high-definition audio.

Its inputs are its own previous output state and a 0.0166-second buffer of data from the Ethernet connection and standard human interface... (read 1513 more words →)

-3

Is a photographer "not an artist" because the photos are actually created by the camera?

This can be dispensed with via Chalmers' and Clarke's Extended Mind thesis. Just as a violinist's violin becomes the distal end of their extended mind, so with brush and painter, and so with camera and photographer.

As long as AI remains a tool and does not start to generate art on its own, there will be a difference between someone who spends a lot of time carefully crafting prompts and a random bozo who just types "draw me a masterpiece"

I'm not as optimistic as you about that difference being eternal. The pre-to-post-AI leap is fundamentally more disruptive, more kind-not-degree,... (read more)

Agreed. However,

Do you think those IRL art forms will always be immune to generative capture and the threshold?
Even if the analogue original (a sculpture, a particular live dance performance, a particular live theatre performance) remains immune, most people will consume it through digital reproduction (photographs, VR, AR, video, audio) for which the threshold does apply.

Oh I see! So that's just the default slideshow range, padded front or back with zeroes, and you can enter much longer locations manually?

I like this.

It feels related to the assertion that DNNs can only interpolate between training data points, never extrapolate beyond them. (Technically they can extrapolate, but the results are hilarious/nonsensical/bad in proportion to how far beyond their training distribution they try to go.)

Here's how I see your argument 'formalised' in terms of the two spaces (total combinatorial phase space and a post-threshold GenAI's latent space over the same output length), please correct anything you think I've got wrong:

A model can only be trained on what already exists (though it can augment its dataset to reduce overfitting etc.), and what already exists approximately overlaps with what humans consider to be good, or at... (read more)

A Kindness, or The Inevitable Consequence of Perfect Inference (a short story)

tl;dr

When (not if) generative models get so good at faking human artefacts that no human can distinguish between human and synthetic artefacts, Strange Things will happen to the way humans consume and think about and make art.

Claim 1: Models get better

Generative AI is well on the way to patching its 'tells'. Hallucination in LLMs, garbled text and polydactyly in image generators, outlying misclassifications in classifiers, etc., are being fixed in new versions. Even allowing for higher-fidelity models provoking more rigorous criticism and therefore discovery of new, subtler tells, GenAI will (perhaps quite soon) reach a point where no naked-eyed human can do better than guess at whether a given artefact is human-made... (read 1073 more words →)