Note: details have been obfuscated and changed to protect – not the innocent, who are invariably vindicated by the end of act three – but the guilty, for whom the narrative provides no such guarantees.
My buddy Mike is one of a very small number of people in my life I would unhesitatingly describe as having a real job[1]. He wakes up, takes the train into work, and directly helps specific people accomplish worthwhile things. Every shift, he leaves the world a little brighter than it was when he clocked in. And for this, the Economy awards him slightly less than double the local minimum wage.
(Meanwhile, I spend about the same amount of time moving some numbers around, and am awarded approximately triple what he makes. They’re good numbers, and I move them with uncommon skill and deftness; still, it’s hard to look at the two of us and not feel like Graeber was onto something.)
Among his many mundane duties is collecting, collating and reformatting The Information. The Information is freely available on The Website, but not in the format his bosses find most palatable. So every month, when The Information is updated, he spends a full day copying it line-by-line into The Spreadsheet.
Or, rather, he spent a full day. When he told me about this, I immediately informed him that was ridiculous, and that I – with help from a then-current Frontier Model – could very easily construct a HTML/JS file he could paste The Website’s source code into, and get The Information back out in perfectly Spreadsheet-friendly format with a click of a button. An hour after I brought this up, we had a functioning proof-of-concept; two hours after that, we had a full solution.
And here our troubles began.
Mike: My employer won't let me use it yet
Me: ?!
Mike: Infosec paranoia
Mike: As I suspected
Me: What information is even being secured?! It's publicly available information!
Mike: I have a way forward tho
Me: I'd caution you against using explosives but in this case I think it's warranted
Mike: I basically just need to talk to [REDACTED]'s colleagues / boss instead of my IT department
Mike: I actually got a light telling off from my boss for not telling him about this before we started working on it.
Me: It's <200 lines of [VARIOUS OBSCENITIES] cookie-cutter HTML/js! The IT team can just, like, look at it!
Mike: See, the trouble with that is that I'm not allowed to download / send it to my work laptop
Me: Are you allowed to take your nonwork laptop to work?
Me: Or are they worried you'll use it plus your l33t hacker skillz to download all their RAM?
Mike: I'm not allowed to do work on my personal laptop, but I can probably get away with bringing it and showing them the code we already set up
Me: Would it help if I actually hosted these websites on a website?
Me: So they can just navigate to it in browser?
Mike: I asked my boss about that and he didn't like it
Me: . . .
Me: Did you get a chance to demo [THE SOLUTION] yet?
Mike: No . . . they need to get approval from [REDACTED][2] first, and I've received no updates . . .
Mike: See, this is why I need to learn HTML. Then I could persuade my workplace to let me write it on my work computer.
Me: Have you considered having your regular monitor open and just painstakingly copying every line onto your work computer by hand?
Mike: Tempted. But I'm sure I would need permission to do that, too
Me: (I'm joking, but the joke is that I don't understand what they think the problem even is.)
Mike: I don't think they understand what the problem is either, as none of the people I've actually talked to are tech savvy at all
Mike: But they're who I need to go through before I get to actually talk to an IT person
Me: "Do you have a license for that? And do you have a license-showing license?"
Mike: One of the few things I dislike about my job is that in terms of the social hierarchy, I am essentially a legume.
Mike: People are nice about it thankfully, but it does mean that I don't have power to do stuff that doesn't normally get done
Mike: Without asking permission first
Me: Can't you just call the IT dept? Or show up there?
Mike: I would have to ask [REDACTED] first
Mike: And it would have to be a day where we're not too busy, and we're actually in the same building
Me: . . . you know, I actually really like insane bureaucracies, under the right conditions.
Me: When the arbitrary constraints are just unconstraining enough that you end up with a fascinatingly-novel problem . . . and you're given enough time & leeway to fix it . . . that's pretty fun.
Me: Doesn't seem like either of us are in those circumstances atm tho.
Mike: No lmao
Mike: [REDACTED] just told me that we probably can't go through with automating this process as the website will probably not give permission for the process to be automated and will probably charge for access to the source code
Mike: So this was all for nothing
Mike: But that's wrong, isn't it? We already have the source code!
Mike: And it's a public website?
Mike: WHY IS EVERYONE SO ADVERSE TO MAKING IT EASIER TO DO A VERY BORING PART OF MY JOB
Me: That sucks.
Me: I'm not sure why you need the site's permission, they expose the info you use to the world every time someone clicks Inspect Element, or hell every time you access the site at all.
Me: At a guess I think they might be worried about you Scraping them, like, telling a robot to repeatedly automatically access their site?
Me: And they think that might chew up their bandwidth?
Me: But if anything this means you'd have to put LESS load on their servers . . .
Me: Lowkey I think the problem might have been that you asked permission.
Mike: I KNOW
Me: If you’d just gone ahead and done it no-one would care.
Me: And if you’d presented it as a fait accompli, bureaucratic logic would have grandfathered you in.
Mike: The thing is I wanted everybody else who does a task like this to ALSO be able to do it the Right Way
Mike: Can’t make that happen quietly
Me: fwiw I think this should be the heart of your argument: "We're not Scraping you so you won't end up DDOS'd, and also it's not automatic because I still have to press the big button each time (& then look at the output to make sure it's actually right)."
Mike: You're a step ahead tho, we're not even at the stage of asking permission from [The Website]. We're asking IT’s permission to ask [The Website]’s permission.
Mike: No even that's a step too far. We're asking [REDACTED]’s bosses for permission to ask IT for permission to ask for [The Website]’s permission
After that last conversation, I got frustrated enough to check The Website’s official Terms Of Use, hoping to find a sword with which to cut this knot. What I discovered instead . . . was that Mike’s task, regardless of the means by which he accomplished it, had always been illegal.
Before we continue, a few clarifications:
The way it was written made it clear that The Website was trying to exclude scrapers, AI labs, and people mirroring it to leech clicks. Prohibiting Mike’s work was obviously an oversight.
The Website was completely in favor of almost anyone using and accessing The Information. If Mike’s bosses wanted to check it themselves instead of relying on Mike’s summaries, that was fine. If they wanted to make money doing so, that was fine. And if a nonprofit or an individual wanted to do exactly what Mike was doing, that was also fine. It was specifically a profit-seeking enterprise using The Information to seek profit while making a copy of it which was forbidden.[3]
There’s illegal and then there’s illegal. Like, there’s murdering toddlers because you think they might have witnessed you selling state secrets, and then there’s torrenting and jaywalking, and then there’s this.
To the very best of my knowledge and understanding, Mike’s (employer’s) use of The Information actively benefited The Website and its operators. Every time he did his thing, they got a little more money and clout in expectation.
I showed Mike the relevant boilerplate, and watched his face contort as he read it, then continue to contort as he processed it.
“This is . . .” he said, a then trailed off.
“This is,” he tried again, “like that one part in Black Dynamite where Black Dynamite announces he’s declaring war on everyone selling drugs in the community, and one guy replies ‘But Black Dynamite, I sell drugs in the community’. I’m the guy! I’m doing the thing they’re saying not to do! I’m daring to oppose Black Dynamite!”
I nodded, and smiled warmly. “You’re complicit in a criminal conspiracy!”
Pained laughter. “Why are you saying that like it’s a good thing?”
“Well, if it’s illegal anyway, then using the script shouldn’t change anything. In fact, it probably makes it less illegal, since improved efficiency means you visit their website a smaller number of times.”
He stared at me like I was twelve, and also insane. “I really don’t think my bosses will see it that way.”
That morning, Mike told the entire story to the most patient and accommodating of his superiors, a man wise in the ways of bureaucracy. His sensei pointed out that automation would inevitably cause the company to check it was following the Terms Of Use, which it wasn’t, and be obligated to get in touch with The Website. And while The Website’s owners actively benefited from the work of Mike et al, they only benefited a little, meaning it wasn’t out of the question they might refuse because they didn’t understand the request, or because they thought they might be able to squeeze money out of Mike’s org if they pretended to take issue, or because “well that seems innocuous but the fact you’re asking for permission makes us suspect you’re pulling a fast one”. And so the sage concluded by gently but firmly telling Mike to go back to doing things by hand.
And that’s where the story ended, when I first started drafting it. But I procrastinated long enough for further events to transpire. And this is the part I think LW will find most interesting:
Over the last few months, there’s been a push in Mike’s company to Get Everyone Using AI, because AI Is The Future (where by “AI” they specifically mean a particular, outdated version of Microsoft Copilot). And while Mike can’t use the solution we built, he’s actively encouraged to use AI to harvest the data[4]. The bot’s contributions are unreliable and need constant checking, but still make things faster than pure elbow grease; what took an entire workday by hand and five minutes with the HTML/JS solution now only takes three hours. Don’t you love progress?
Conclusions
Why did I write this?
Well, most immediately, I’m hoping commenters will suggest a way to avoid him having to spend those three hours each month. I already offered “what if you pretend the AI wrote the HTML/JS script when you told it about the problem” (“tempting, but my superiors might check the logs”) and “what if I ask The Website for permission on my own initiative, and only tell you if they say yes” (“more than my job’s worth if they say no and it’s somehow traced back to me”); if you have any other ideas, I want to hear them.
But I also think there are some meaningful things Lesswrongers could take away from this saga, especially from how it ends. Namely:
Efficiency gains from AI don’t all come from AI. I talk a lot of crap, but AI did make Mike’s job easier; compare the eight hours a month I failed to save him to the five hours Copilot successfully did. But it wasn’t because direct use of AI improved on the best available solution; it was because having a fad to play along with gave corporations an excuse to let workers do anything new. I think a decent fraction of the economic benefit from AI so far (15%, maybe?) would have come from companies getting similarly excited about abaci, or Agile Scrum Methodologies, or the Force.
Public sentiment is driven by AI use in corporate and government contexts, which generally sucks. If you're a typical techie – that is, if you’re a smart person with negotiating power and autistic tendencies working under conditions built by and for smart people with negotiating power and autistic tendencies – you’re probably underestimating how many people’s main exposure to AI is a lumbering bureaucracy pushing an out-of-date[5][6] LLM on them, in a context it isn’t particularly suited for. This, imo, goes a long way to explain why the average person continues to underrate AI’s promise and threat.
This is what having a real job is like. Everything where you directly help another person is considered too important not to over-regulate, making mundane innovation almost impossible. Ultimately, we may someday manage to ban everything meaningful or ambitious, except trying to destroy the world.
The fact that accessing The Website via browser created a copy of The Information on the user’s computer presumably didn’t count, because uhh it just didn’t.
I work for an anomalously-non-dysfunctional tech-adjacent corporation which talks a lot about being AI-First. I first got access to Claude from my work computer earlier this month. Most behemoths are, from the little I hear, further behind the curve.
What if Mike just gets the 200 lines of HTML onto his laptop without telling anyone about it, then uses it and pretends his monthly Spreadsheets are still handmade?
We thought of that: such a large deviation between fiction and fact is bound to be discovered sooner or later (What if someone looks over his shoulder in the office? What if he accidentally submits The Spreadsheet two hours faster than he 'should' be able to, one day?). Also, the guy really doesn't like actively lying to coworkers and disobeying direct orders. (Also also, we're still ideally hoping for an answer that lets everyone doing this kind of drudgery automate it.) (Sincere thanks for trying, though!)
“what if you pretend the AI wrote the HTML/JS script when you told it about the problem” (“tempting, but my superiors might check the logs”)
What if you actually get the AI to write the HTML/JS script? Tell it to make something structured the same way the good one is structured, the result will suck, then open up the good one on another monitor and go through it line by line coaching the AI on how to change its work ("it just had a few minor errors") until Copilot Menard has perfectly recreated the original. Then start using it without asking anyone for permission, because obviously you have permission to use the products of the AI they encouraged you to use, and tell coworkers about this cool script they can use too.
Note: details have been obfuscated and changed to protect – not the innocent, who are invariably vindicated by the end of act three – but the guilty, for whom the narrative provides no such guarantees.
My buddy Mike is one of a very small number of people in my life I would unhesitatingly describe as having a real job[1]. He wakes up, takes the train into work, and directly helps specific people accomplish worthwhile things. Every shift, he leaves the world a little brighter than it was when he clocked in. And for this, the Economy awards him slightly less than double the local minimum wage.
(Meanwhile, I spend about the same amount of time moving some numbers around, and am awarded approximately triple what he makes. They’re good numbers, and I move them with uncommon skill and deftness; still, it’s hard to look at the two of us and not feel like Graeber was onto something.)
Among his many mundane duties is collecting, collating and reformatting The Information. The Information is freely available on The Website, but not in the format his bosses find most palatable. So every month, when The Information is updated, he spends a full day copying it line-by-line into The Spreadsheet.
Or, rather, he spent a full day. When he told me about this, I immediately informed him that was ridiculous, and that I – with help from a then-current Frontier Model – could very easily construct a HTML/JS file he could paste The Website’s source code into, and get The Information back out in perfectly Spreadsheet-friendly format with a click of a button. An hour after I brought this up, we had a functioning proof-of-concept; two hours after that, we had a full solution.
And here our troubles began.
Mike: My employer won't let me use it yet
Me: ?!
Mike: Infosec paranoia
Mike: As I suspected
Me: What information is even being secured?! It's publicly available information!
Mike: I have a way forward tho
Me: I'd caution you against using explosives but in this case I think it's warranted
Mike: I basically just need to talk to [REDACTED]'s colleagues / boss instead of my IT department
Mike: I actually got a light telling off from my boss for not telling him about this before we started working on it.
Me: It's <200 lines of [VARIOUS OBSCENITIES] cookie-cutter HTML/js! The IT team can just, like, look at it!
Mike: See, the trouble with that is that I'm not allowed to download / send it to my work laptop
Me: Are you allowed to take your nonwork laptop to work?
Me: Or are they worried you'll use it plus your l33t hacker skillz to download all their RAM?
Mike: I'm not allowed to do work on my personal laptop, but I can probably get away with bringing it and showing them the code we already set up
Me: Would it help if I actually hosted these websites on a website?
Me: So they can just navigate to it in browser?
Mike: I asked my boss about that and he didn't like it
Me: . . .
Me: Did you get a chance to demo [THE SOLUTION] yet?
Mike: No . . . they need to get approval from [REDACTED][2] first, and I've received no updates . . .
Mike: See, this is why I need to learn HTML. Then I could persuade my workplace to let me write it on my work computer.
Me: Have you considered having your regular monitor open and just painstakingly copying every line onto your work computer by hand?
Mike: Tempted. But I'm sure I would need permission to do that, too
Me: (I'm joking, but the joke is that I don't understand what they think the problem even is.)
Mike: I don't think they understand what the problem is either, as none of the people I've actually talked to are tech savvy at all
Mike: But they're who I need to go through before I get to actually talk to an IT person
Me: "Do you have a license for that? And do you have a license-showing license?"
Mike: One of the few things I dislike about my job is that in terms of the social hierarchy, I am essentially a legume.
Mike: People are nice about it thankfully, but it does mean that I don't have power to do stuff that doesn't normally get done
Mike: Without asking permission first
Me: Can't you just call the IT dept? Or show up there?
Mike: I would have to ask [REDACTED] first
Mike: And it would have to be a day where we're not too busy, and we're actually in the same building
Me: . . . you know, I actually really like insane bureaucracies, under the right conditions.
Me: When the arbitrary constraints are just unconstraining enough that you end up with a fascinatingly-novel problem . . . and you're given enough time & leeway to fix it . . . that's pretty fun.
Me: Doesn't seem like either of us are in those circumstances atm tho.
Mike: No lmao
Mike: [REDACTED] just told me that we probably can't go through with automating this process as the website will probably not give permission for the process to be automated and will probably charge for access to the source code
Mike: So this was all for nothing
Mike: But that's wrong, isn't it? We already have the source code!
Mike: And it's a public website?
Mike: WHY IS EVERYONE SO ADVERSE TO MAKING IT EASIER TO DO A VERY BORING PART OF MY JOB
Me: That sucks.
Me: I'm not sure why you need the site's permission, they expose the info you use to the world every time someone clicks Inspect Element, or hell every time you access the site at all.
Me: At a guess I think they might be worried about you Scraping them, like, telling a robot to repeatedly automatically access their site?
Me: And they think that might chew up their bandwidth?
Me: But if anything this means you'd have to put LESS load on their servers . . .
Me: Lowkey I think the problem might have been that you asked permission.
Mike: I KNOW
Me: If you’d just gone ahead and done it no-one would care.
Me: And if you’d presented it as a fait accompli, bureaucratic logic would have grandfathered you in.
Mike: The thing is I wanted everybody else who does a task like this to ALSO be able to do it the Right Way
Mike: Can’t make that happen quietly
Me: fwiw I think this should be the heart of your argument: "We're not Scraping you so you won't end up DDOS'd, and also it's not automatic because I still have to press the big button each time (& then look at the output to make sure it's actually right)."
Mike: You're a step ahead tho, we're not even at the stage of asking permission from [The Website]. We're asking IT’s permission to ask [The Website]’s permission.
Mike: No even that's a step too far. We're asking [REDACTED]’s bosses for permission to ask IT for permission to ask for [The Website]’s permission
After that last conversation, I got frustrated enough to check The Website’s official Terms Of Use, hoping to find a sword with which to cut this knot. What I discovered instead . . . was that Mike’s task, regardless of the means by which he accomplished it, had always been illegal.
Before we continue, a few clarifications:
I showed Mike the relevant boilerplate, and watched his face contort as he read it, then continue to contort as he processed it.
“This is . . .” he said, a then trailed off.
“This is,” he tried again, “like that one part in Black Dynamite where Black Dynamite announces he’s declaring war on everyone selling drugs in the community, and one guy replies ‘But Black Dynamite, I sell drugs in the community’. I’m the guy! I’m doing the thing they’re saying not to do! I’m daring to oppose Black Dynamite!”
I nodded, and smiled warmly. “You’re complicit in a criminal conspiracy!”
Pained laughter. “Why are you saying that like it’s a good thing?”
“Well, if it’s illegal anyway, then using the script shouldn’t change anything. In fact, it probably makes it less illegal, since improved efficiency means you visit their website a smaller number of times.”
He stared at me like I was twelve, and also insane. “I really don’t think my bosses will see it that way.”
That morning, Mike told the entire story to the most patient and accommodating of his superiors, a man wise in the ways of bureaucracy. His sensei pointed out that automation would inevitably cause the company to check it was following the Terms Of Use, which it wasn’t, and be obligated to get in touch with The Website. And while The Website’s owners actively benefited from the work of Mike et al, they only benefited a little, meaning it wasn’t out of the question they might refuse because they didn’t understand the request, or because they thought they might be able to squeeze money out of Mike’s org if they pretended to take issue, or because “well that seems innocuous but the fact you’re asking for permission makes us suspect you’re pulling a fast one”. And so the sage concluded by gently but firmly telling Mike to go back to doing things by hand.
And that’s where the story ended, when I first started drafting it. But I procrastinated long enough for further events to transpire. And this is the part I think LW will find most interesting:
Over the last few months, there’s been a push in Mike’s company to Get Everyone Using AI, because AI Is The Future (where by “AI” they specifically mean a particular, outdated version of Microsoft Copilot). And while Mike can’t use the solution we built, he’s actively encouraged to use AI to harvest the data[4]. The bot’s contributions are unreliable and need constant checking, but still make things faster than pure elbow grease; what took an entire workday by hand and five minutes with the HTML/JS solution now only takes three hours. Don’t you love progress?
Conclusions
Why did I write this?
Well, most immediately, I’m hoping commenters will suggest a way to avoid him having to spend those three hours each month. I already offered “what if you pretend the AI wrote the HTML/JS script when you told it about the problem” (“tempting, but my superiors might check the logs”) and “what if I ask The Website for permission on my own initiative, and only tell you if they say yes” (“more than my job’s worth if they say no and it’s somehow traced back to me”); if you have any other ideas, I want to hear them.
But I also think there are some meaningful things Lesswrongers could take away from this saga, especially from how it ends. Namely:
. . . and no, I’m one of them.
NB: ~all these REDACTEDs are different people and/or departments.
The fact that accessing The Website via browser created a copy of The Information on the user’s computer presumably didn’t count, because uhh it just didn’t.
"Wait, isn’t this the exact ‘scraping’ situation The Website’s rules were written with the intent to prevent?" Yes! It is!
. . . because everything is out-of-date once it’s finished getting through procurement and compliance processes.
I work for an anomalously-non-dysfunctional tech-adjacent corporation which talks a lot about being AI-First. I first got access to Claude from my work computer earlier this month. Most behemoths are, from the little I hear, further behind the curve.