It's a first

The tech industry is in a weird position. Or maybe I am in a weird position in my work + social tech bubble. You know what the issue is: LLMs. It’s so so tiring to see LLMs offered as a magic solution to every problem. LLMs are obviously a tool and, if you used them, you know that thare are some fields in which they are more successful (mostly code generation) and some in which they aren’t. In the latter, there are still many things that don’t work reasonably well and there is no evidence that things are going to change for the better.

Aside from the actual details, I find shocking to hear how many people believe that some things will just become magic very soon with no evidence that this will be the case. If we think about code generation, the potential was visible clearly from the first demos. Things have evolved and they did get better, but we are talking about something that can be tested, validated, that is partially not consumed by humans and that can always be proven to work (at last, in production). In other situations, that’s not the case: using an LLM to summarize data? If the data is hallucinated, you will be dealing with wrong data without realizing (because you wanted to summarize in the first place!). There are of course a lot of similar negative cases.

What’s shocking to me is that it is the first time for me in my career that I see unlimited trust in something that haven’t proven most of the things that have been reported. I know it’s because of capitalism and money and all the other things, but it’s a first for me, the first time in which I have to assess and evaluate technologies against a belief and not against facts. While I say this, I am also convinced that coding will be changed significantly in the next years and that the role of developers has shifted significantly (as I wrote here). Maybe there will be many more applications of LLMs, maybe not.

I am relatively convinced that for “AI” to be the game changer in lot of fields we will need much more than what we have seen today, different techniques, approaches and technologies. We should then not pretend that what we have today will do wonders if we technically know that it won’t. Improvements will come, new applications will keep showing up, but as technologists, we should assess things as they are and for what they can do. The fact that things aren’t going this way is making my life as a non skeptical open minded person significantly harder than it should be.