I am not an "AI" researcher

Why this post?

Confident and clueless people making physical threats. The public discourse on AI has reached a certain fervor with Jacob Coxon’s post, which, while expressing exceedingly standard viewpoints among some alignment researchers (note: NOT the typical ML researcher population), has been the only successful one in over 10 years to break containment. People seem to think that his post would have necessarily gone public, but alignment researchers have been trying to reach the public for ages trying all kinds of strategies. So weird that people are finally aware off of a fluke. While I have been frustrated with the quality of discourse in the past (see below), this incident has led to an especially large influx of especially confident and especially clueless people. Enough so that there have been multiple incidents of vandalism and a home break-in around Mila due to anti-AI sentiment, including a vandalism incident at Mila (which is completely the wrong target even if you hate frontier models and would do anything to stop it, but more on this below), AND that these actions are extremely widely condoned on the internet and more extreme actions have been condoned as well. Thus, in many ways, this post is targeted towards criticizers of AI in the non-technical public, which is probably most Gen Z and people on the internet now, because as it stands this is the population that seems to randomly want to hate crime me due to incorrect knowledge, as stupid as hate-criming Chinese-American people in WW2 because you think they are Japanese (when you shouldn’t even be hate criming Japanese people in the US). I can’t believe we have to say this but here we are. Below I will talk more on exactly what I think the problems are with this position (especially, concrete concerns about “AI” and conflation with frontier models vs. other types of ML).

Everyone is talking about AI too much for me. I have had a tiring few last weeks. With general discussion of LLMs, the release of GPT-6 Astra, and the Mila vandalism incident, I’ve spent an inordinate amount of time having conversations with people about “AI”. Sometimes this is as much as 15-20 hours in a week, and it is really too much to take. On one hand, my fault for choosing to engage. On the other hand, I feel an obligation to participate, having spent such a huge amount of time discussing and thinking about the issue, and obviously having a reasonable amount of adjacent experience. And, everyone is talking about it. If I took all the opportunities to discuss, I would literally have around 20-30 discussions in a week. For these reasons, as well as reasons below, I’ve decided to avoid conversations about “AI” (frontier models) if I can help it, because I think they are largely fruitless, and very frustrating/ragebait. But I feel obligated to make this post to clarify, and attempt to improve conversation discourse around me, or at least explain why I do not wish to discuss.

On “AI” discourse

Nuance in conversations, and understanding strengths of other positions and weaknesses in your own position. Despite my best efforts, it’s too much for me. I am big on nuance in conversations, and I think it is kind of the only way you can be remotely close to correct on controversial topics. It’s so critical to understand all the views out there, especially the ones that disagree with you. While the default is to understand your own position’s strengths and other positions’ weaknesses, this is a fundamentally stupid way to approach a discussion. Almost no one I see is looking carefully for their own position’s weaknesses and other positions’ strengths, but this is necessary to get value from a conversation and to try to make yourself correct. In my desire to understand better and be correct, I discuss with non-technical AI glazers, non-technical AI haters, technical LLM proponents, and technical LLM detractors, trying to understand and improve my own position, and I’ve learned a lot! But from all sides I see so little nuance in this way. It’s better among technical people, i.e. ML researchers or ML-research-adjacent people, who have more nuance/understanding, but unfortunately still largely suffer from not being sufficiently knowledgeable about opposing positions.

It is telling that I am significantly more skeptical of holes in my own position than most people I talk to (especially the general public), yet I have spent significantly more time discussing and analyzing the issue, along with this issue being adjacent to my expertise (being ML-related). If anything I should be significantly less skeptical of my own position than people I talk to, but you know how it is with this kind of discourse.

Bastardization of the term “AI”. By and large, the public has decided to use AI to refer to specific kinds of generative AI, mostly LLMs (especially frontier models) and some image generation models stapled alongside it. In other words, to the public, the word “AI” now mostly just means frontier models. Someone choosing to refer to LLMs specifically as LLMs immediately gives me a ton of signal that they are more nuanced than a lot of the public, even if overall I still don’t expect them to be very knowledgeable. Going forward, I will take the public up on the proposal to bastardize the term “AI” for exclusively referring to frontier models, especially because pretty much all of the complaints about AI are applicable exclusively to frontier models.

What’s actually bad and good about frontier models? There are a few camps that I’ve seen.

The biggest camp I want to address is haters of AI in the general public. Roughly, this camp has around 5 claims or so (alongside catastrophic risk, see below on safety/alignment people), 2 of which I consider to be mostly absolute (but still unclear on scale of impact), and 3 which I consider to be temporary, due to its newness as a technology.

The absolute ones are

  • Environmental concerns, hopefully possible to mitigate in the future and unclear on scale; directionally I believe this impact exists, small or large
  • Data rights/privacy, where people are generally not informed or compensated for their data being used

The other three which I consider more mixed/addressable with technology maturation/familiarity are

  • Slop/jagged capability frontier: you HAVE to test and understand where the LLM is strong and weak; it’s clearly strong in math and coding implementation, synthesizing search results at scale, and various specific tasks, less so if you arbitrarily pick stuff)
  • Unemployment/job churn: technologies have almost always resulted in lots of jobs being lost and new jobs being created, though I’m less sure about this point and would be happy to discuss this specifically, and if I’m wrong this could be a major point that needs to be addressed by the government
  • Mental offloading: people need to find better ways to work with LLM strengths/weaknesses and in particular use it to improve your knowledge rather than use it like Chegg on everything and learn nothing

Some people also consider concentration of power within frontier labs and already-wealthy people as a major problem. I am not here to litigate opinions for or against capitalism, nor have I thought sufficiently about the problem to have a strong position on this. From a glance, it seems resource concentration concerns are generally reasonable and that this type of resource concentration should be redistributed to a degree (cf. point about data rights which are relevant for the performance of these LLMs), though I think caution should be taken in how heavy-handed the approaches are re: incentives and amortized costs for R&D/startup creation, which I claim do have real benefits for society.

I consider many of these legitimate concerns, but they are exclusively for large and generalist models. 2010s-deep learning style task-specific “small” models (even something like AlphaStar, which was formerly considered insanely huge) have none of these concerns, and this type of smaller, task-specific models is the type of research I work on.

As far as other camps go:

  • Glazers of AI in the general public have the least ground to stand on, in my opinion. The biggest sin is not respecting the jagged capability frontier of frontier models, and seeing strong performance on individual benchmarks where models are known to have large amounts of training data, and thinking they’re good at everything even if task-specific weakness slaps them in the face.
  • Detractors in the technical community are primarily those worried about safety/alignment, fast improvements, and takeover. These people have spent the most time looking at the issue, so they tend to be the most nuanced, but I still think they have overly strong priors on the rate of improvement, generality of frontier model strengths (when it is really extremely jagged and task-specific), and likelihood that (in the short term) risks will turn out to be as they hypothesized, re: loss of control hypotheses.
  • Proponents in the technical community are mostly focused on research and efficiency benefits of frontier models and generally over-focus on their strengths, much like detractors in the technical community, but don’t really believe in fast RSI or loss of control hypotheses. My primary critique with this camp is that they tend to dismiss claims about loss of control as purely science fiction without understanding the strongest version of the argument, and don’t respect the potential for regime change. Again, not that they are wrong, but that there is so little engagement with opposing viewpoints.

I am not an “AI” researcher. In light of the usage of the word AI in public discourse, and more importantly, opinions about AI in public discourse pretty much exclusively being true for frontier models but deeply untrue for other kinds of ML, I think it is mostly fair to adopt this convention for AI = frontier models. In some ways the umbrella term of “AI” in the old sense (encompassing ML and more) no longer really makes sense as a term of communication, unless you use it to mean exactly frontier models. So in that way, I am not an “AI” researcher in any meaningful sense, I am a game-theoretic multi-agent RL researcher, or an algorithmic game theory researcher if you want to dodge bad associations between RL/agents and frontier models (but this is also stupid, as if now the term math is tainted by association with frontier models, this is really a field that exists entirely independently from frontier models but has been co-opted for use by frontier labs). Again, I hate that this needs to be said, but most ML researchers are not reasonable targets even if you are willing to do anything to mitigate “AI”/frontier model problems. Public support for Mila vandalism has made me pretty sick to my stomach because of how clueless but also counterproductive it is. I am not saying you should do anything to frontier labs with violence (what do you even think that’s going to achieve, be for real about making change) but if you were to pick a target then it’s frontier labs and not academic researchers almost exclusively NOT working on frontier model improvements.

Task-specific smaller models are still excellent. This is the old 2010s-era deep learning that I loved. Using data with lots of signal for good behaviors/performance, for a specific task, is a paradigm that drove so many huge advancements back in the 2010s and early 2020s. From a pure capabilities standpoint (if we were to ignore all the other problems with it), frontier models (large, generalist-ish models) still have their place, but I really wish that the public would understand this task-specific smaller-model paradigm. Using signal-rich task-specific data is just good, it’s not slop, it’s not environmentally harmful, it’s effective, and we get really nice insights from it. Field-specific impact can still cause job churn (cf. AlphaGo) but these consequences are usually much easier to deal with, and with developers of these kinds of models usually very interested in helping people within the impact radius understand how to adapt to and leverage technologies for their benefit, while frontier labs are generally not interested in or even capable of doing so given their huge blast radius.

Closing thoughts

  • AI (frontier model) discourse has become especially bad recently, and like with any other controversial topic, I urge people to understand opposing viewpoints better and understand weaknesses in their own position. This is all for the sake of having more accurate opinions, not that you need to agree with opposing views, or force yourself to take a moderate/fence-sitter stance when it is not warranted.
  • Huge distinctions should be drawn between LLMs/frontier models (“AI”) and the older 2010s-era deep learning paradigm, with smaller, task-specific models. While these misunderstandings do have personal impacts for me, I’m more just frustrated at the pointless and avoidable problems arising from conflating these topics. I’m not an AI researcher, with the way the term “AI” is being bastardized now. Neither are most people at Mila. Let’s learn something about what we’re hating on before we start the second Cultural Revolution in the US/Canada, this is ridiculous. Smaller, task-specific models are interesting and useful, and have none of the drawbacks of frontier models.
  • Concerns about frontier models are reasonable. Taking the time to discuss this is not bad at all (but count me out for now, I’ve had too much discussion), but more time discussed should result in a sharper picture about exactly where the problems are, exactly where the benefits are, and what we should do to maximize benefits and mitigate problems. I hope as the technology matures we’ll find better ways to handle it, and if catastrophic risk is a concern, that we can take reasonable measures to mitigate it.