Grappling with AI progress

This post is just me trying to understand what the near-term future might mean in terms of AI progress. All of this is highly personal - I’m mostly writing this to reread it in 5 years and to see how it turned out.

I think AI becoming smart enough to replace most of today’s work in engineering and manufacturing is a possible scenario that might occur in the next decade. I have no idea what it will do to our society and life.

What happened so far

~2012..2014: I do a (short) deep dive on neurology, the brain and human modelling as part of my PhD in mechanical engineering. At that time, human modelling mostly relies on mechanistic control theory models and on statistical models. At the same time, deep learning emerges as a promising new technology. It gives us better voice models, Siri and tools like AI-generated art that will redraw a photo in various art styles. I get somewhat hyped and expect AI to become amazing over time.¹ That does not materialize, progress in voice assistants stalls after a few years. I still expect AI to be mostly tools, not intelligent agents on par with general human abilities.

~2019: I learn about predictive processing, a theory that emerged circa 2005 – 2015 as a potential unified theory of the brain. I’m still upset I wasn’t aware of this when investigating vehicle driver models – Andy Clark’s seminal 2013 paper got published just in the right timeframe for me. ²

2020: I read several posts by Scott Alexander on GPT-2 and learn about GPT-3 from this post here. I didn’t really remember the GPT-2 posts at all until I reread them and mostly put GPT-3 into the “it will finally generate nice literature including that on-demand biography for you” box. I expect GPT-X to eventually become a helpful ghostwriter for humanity and replace a few journalists along the way, with the same level of disruptiveness as “self-driving cars will put taxi drivers out of work, but almost everyone else will benefit”.

2021 .. 2022: I read “come, walk with me through latent space” illustrated by AI-drawn imagery. This is impressive enough for me to forward this to several friends. DALLE-2 gets published in 2022 and articles on my reading list start to feature images it generated. A few months later, stable diffusion gets its first public release and I play with it a little bit. For all of these, output still looks flawed: AI doesn’t get hands and faces right most of the time. GPT3.5 and Chat-GPT release, reaching a larger audience and I’m mostly “oooh, I already know that already”. It’s fun to play around with, sometimes useful, but still not on a human level.

2023: Several new tools release and I have my “oh shit”-moment:

• GPT-4 releases, claiming ~90ish-percentile performance compared to humans for a wide range of tasks.
• Microsoft promises a soon-to-arrive AI copilot for office, promising to automatically compose documents for you (as in: “summarize this 15-page report on one page”, “turn this presentation of findings into a written report”)
• AI timelines by various people get shortened a lot. Beren Millidge as one example: “while there are a few technical problems remaining — of significant difficulty — there is no reason why I think they cannot be addressed within the next few years. I think it is clear that we will have AGI (although not necessarily super intelligence) by 2030 and likely significantly sooner.”
• At roughly the same point I decide to check what progress AI-generated art has made and… wow. I didn’t see this coming that quick. See here for an example from that time (and here for a similar comparison as of 2024).

In 2024, things first stall a little bit and then pick up speed again with the release of Claude 3.5 iterations and GPT-o1 / o3 models.

A simple scenario

Instead of checking what AI can do right now, consider how fast it improves and extrapolate to the next years.

• GPT 2 is as smart as a 6-year-old
• GPT 3 like a highschooler
• GPT 4 like a college student
• GPT o1/o3 like a PhD student

This is of course a bad comparison in many ways, the models aren’t agents (yet) and it is really difficult to find good ways to work with them without doublechecking their writing or tolerating some amount of hallucinations. 

Still, progress is mindbogglingly fast. A crude extrapolation indicates that we should see superhuman behavior soon, or at least reach a state where lots of human labor gets replaced by ai models soon. Metaculus appears to agree and expects AGI some time soon, as a median circa 2030 (as in “5 years from now”).

The case for future improvements is that at some point we should see very smart models. Think of an army of smart PhDs you can throw at any mental problem you encounter. And while there are many things they won’t be able to do on their own, optimizing algorithms for tough problems like robotics won’t be on that list. 

So sooner or later we’ll also have machines that can see, work and perform standard tasks cheaper than a human worker. Think of robot arms but with enough smarts to sense and act on the world around them for manufacturing, and roomba robots with arms and actuators in your house smart enough to also do your laundry and cooking, or whatever.

If you’re a white-collar worker relying on your brain working well, this is bad news. If you’re a blue-collar worker relying on your hands working well, this is also bad news. It’s not that we should expect machines to replace our best people soon, it’s the average person we should be worries about.

I have no idea how this will turn out, but I’m very sure it won’t be normal. But the possibility of retiring and enjoying a relaxed life while AI fixes renewable energy and curing cancer sounds sort of OK, so let’s hope for that.

Appendix 1: Some room for scepticism 

The biggest caveat I have here is how little practical use I get from today’s AI models. I have access to GPT-4o for work and use Claude 3.5 in my spare time, but it’s hard to point to significant impact they’ve had on my life.

I already have more high-quality writing and art available than what I can consume. Just as it’s difficult to assign tasks to someone else (unless they already know what they’re doing) it’s just a lot of work to use text generator models for something useful.

This matches what I hear from many others, “yeah it’s kinda cool but look I tried some stuff and it failed in really dumb and stupid ways”.

I don’t think this makes the argument I lay out above any worse: What matters is the rate of improvement, not the current state. Also, if you’re interested in practical current impact, try asking a few programmers or graphic designers on what their experience is. I’d expect they have pretty strong opinions.

Appendix 2: Further reading 

If this freaks you out, you might want to read:

• AI: Practical Advice for the Worried - by Zvi Mowshowitz
• Raising children on the eve of AI • Otherwise

If you want more info ingeneral, I’d recommend the newsletter Don’t Worry About the Vase by Zvi Mowshowitz.