(Source: Todd Schneider / MTA)
Narrowly, then, the problem with using things like O*NET to try to analyse what a job is and how much it can be automated is that this tells you nothing about all the ways that the job shrinks and grow with automation, and the ways that the job itself might be changed by automation elsewhere, outside your analysis.
But I think there’s a more fundamental problem, too. Even if you set aside the question of change, I don’t think it’s possible, in principle, to create a usefully complete description of what the job is.
Reading O*NET descriptions of jobs reminds me a lot of the failure of expert systems, when people thought that you could use logical steps to build an AI system to do image recognition or language translation. Theoretically, you can describe a series of steps by which a machine can recognise a cat, and theoretically, you can write down exactly what an associate partner at a law firm does, but in reality, these things are just too complex or too subtle for us to be able to describe them like that. Sometimes, of course, the job really is just a task, that can be turned into a button, but that’s actually pretty rare. Generally, the job is a complex mesh of things that we lack the capability to explain explicitly (tangentially, this is also why most people seem to struggle to use chatbots). And, of course, once you dig into the detail these descriptions fall apart, just as logical systems did before machine learning: apparently administering a family trust and running a desk at a quant fund are comparable jobs, and they need fluency in Lotus 1-2-3, Oracle or Quickbooks but not Bloomberg.
Aaron Levie, CEO of Box, described this as a variant of ‘Gell-Mann Amnesia’. You have a pretty good sense of how complex your own field is, and how incomplete AI’s addressability of that might be, but in other fields you forget this – you see a Claude template for a Powerpoint or a legal draft and you think “wow, consultants and law firms are screwed!” When you hire Bain, BCG or McKinsey, they will give you some slides, but that’s not what you’re paying for, just as when you buy software, you’ll get some code, but that’s not the product.
The counter-argument to all of this, would be to say that, yes, well done, there are important exceptions, as there always are, but directionally and in aggregate, it is ‘surely’ correct to say that jobs that involve a lot of repetitive clerical work are most exposed, and this is how many jobs that is, and by how much. That sounds good, but you don’t know if the exceptions are bigger than the rule. Suppose we’d looked at the internet in 1995 and said that this would destroy the value of physical distribution for media – this was ‘directionally correct’, but in practice that meant totally different things for record companies, newspapers, TV companies and movie studios. On average, we’re all dead. Half of the jobs you’ve analysed might be entirely unaffected, and there might be other big pools of jobs to be transformed that you miss entirely. You don’t know.
A while ago, I noted someone had criticised my work by saying that I always end by saying ‘it depends’. But when you’re at such an early stage of a fundamentally new technology, any specific predictions about a particular field will only be correct by luck: it really does depend. As Yogi Berra said, “it’s tough to make predictions, especially about the future”. We can certainly point to framings and mental models for how this might work, and we can point to what happened the last half-dozen times we went through this kind of change. We can even say things that are probably directionally correct. But as soon as you try to quantify that, and model it out job by job and industry by industry, and make pretty radar charts, you’re fooling yourself, because you do not actually know what those jobs are today, and you do not know how they will change. At a minimum, you have to ask whether your model passes the newspaper test, the Uber test and the CPA test: would your approach have captured those effects? If not, how useful is it to the rest of us?