Slop right there ⏹️
If you haven't heard ai slop, if you're truly one of the fortunate people that hasn't, actually... you know what, wait a second, let's not rush here, if you haven't heard it, do yourself a favour and go out, I'm serious, take your things and leave, don't read this, don't you dare read this, I don't want to be responsible, bless your heart and go on your merry way 🪷.
...
You're still here? I see... curiosity? masochism? a bit of both? a bit of everything? you heard ai slop and it picked your interest? a colleague dropped it in a convo, or perhaps you overheard the IT guy mentioning it? or like me, you battle it every day? it's invading your dreams? it's the only thing you dream of recently if at all? Well come closer to the fire, you're in good company 🪵🔥 (you're so not).
For one reason or another the word slop lives rent free in my mind, it has been for a while, I think the initial fixation was that I've never heard it (as in the word itself), or perhaps I didn't pay attention when I did, you know, not in the context of software, and when I did hear it or read it in the context of software I thought well, this isn't a new idea; You know, back in my day we called slop: spaghetti 🍝.
And what made it stay with me was the sort of need to name this feeling for newcomers, I guess regulars in a way too, cause it's different than spaghetti since spaghetti code has a narrow application, many normies were being introduced to unorganised entangled results of statistically generated words applied to code and everything really.
It might be that the term stuck because it's a catchy word and because it was broadly applied and started to describe very well a sort of forming feeling, you know the big "ah anyone can do this" moment, and for people started vibing stuff, everything under the hood looked about the same, but for people who knew the difference between "smart", "clean" and "dumb" code, it was clear that ai was generating the former; in software smart code is the kind of code that will certainly will make you look smart (hence the name) but it will also come back to bite you in the 🍑, why well because for most people "smart" is a frugal state, sometimes you wake up with an anvil for head, denser than a neutron and you look at your masterpiece and get... nothing, you stare... still nothing, stare harder... almost... nah useless, now you lift both hands and pull away your cheeks and think:
I should've written the version for dummies.
Coming back to what it is and why is it called slop? why muddied water? well I can't speak for the person who initially called it that in the medium, was it Karpathy? or was he the father of the term vibe coding? I mean whoever it was, it lasted because it's a pretty good way of describing ai output, it certainly felt "muddied" it was like looking through a frosty glass, if you knew what you were looking for. I assume for most people that were being introduced to vibe coding, it was imperceptible, but as with all things internet, the moment they started showing off and people who knew what was going on, stepped in and felt a need to name it. Except they didn't? Slop was used, my sources tell me (thank you Yipitee and Claudio) that it was already attached to "rubbish tv" during the previous century. So in media it was already a thing; this was just an interpolation, mostly.
Slop in code quickly found its way into our hearts, even people who knew what they would be looking for, meaning that they had some kind of standard for code accepted that for some cases it was fine, throw away scripts or local stuff that won't be stress tested. It's clear by now that production code is subjected to many different circumstances that models aren't yet equipped to handle in one go. As many have suggested to replace a big player like github, (sorry gitlab I know you exist, and that you made too much sense calling them merge requests instead of pull requests but you still look like a 90s website and that somehow matters a lot) you'd have to recreate not only the git client and hosting of projects, but the social aspect of the platform, we're talking systems upon systems of data handling, payment processing, multi-tenancy, user roles management, api, cli, mcp, among many many other layers of data handling. Even if you were to somehow get a document that described the whole thing in detail, a software factory today, would still get it wrong.
And by the way slop doesn't mean inaccurate, which can happen with ai models, they hallucinate, slop is a qualitative description of the work, it's like done, but in a lazy way, in a way that shows no interest or expertise, it's like it does what it needs to but, sufficiently but not comprehensively.
Now slop remains interesting because it might be here to stay, notice that most of the big players are working hard (I swear I work hard at spaceXAi) to try and get rid of it. But it's to be determined if we ever will. I think that there's some irony in that I work with many of the prominent models and still know so little about how ai actually works, I know that linear algebra is involved, I tried training myself on the subject, I still can recall matrix transformations and will probably pick it up in the future, it somehow seemed urgent to do it in the middle of the ai rise, but by now it's unclear, at least to me, how important it is really, cause a couple of things are clear as I write this:
ai is here to stay
how is made is probably not as important currently as it is finding it an application
And notice I said, finding, I want to make a distinction here, cause ai is like an umbrella term, I mostly am referring to LLMs, cause for discrete models, that's easy, computer vision, classifiers and models that came to be with an objective in mind and even the name tells you what they are, they're still ai, conceptually, but they have clear use cases, LLM seemed like they had it too. Anything speech related, anything that involved writing in any way, right? which is almost anything but then, we tried and it isn't that clear anymore, customer service? management? drafting? video editing? all of those use cases exist and can be performed and lead by LLM's, but is that their optimal use case? is that where they excel? where their quality justifies the price? again, unclear.
And it is unclear not only because of the results, improving as we speak, but because there are plenty of technological, ethical, economic and social factors involved. Now another thing that might be the case for LLM's is automation, except that was already working at scale before them, and for most automation you need predictability, you need determinism, determinism if you're not familiar with the term is something that given the same input gives you the exact same output, which is not LLM's or how they work... don't believe me? go ask your favourite model what is an ice cream, 3 times in a row. Yeah the substance might be the same, but the answer in terms of determinism is not. 2 + 2 = 4, that's deterministic.
Now ice cream, might be a somewhat complex topic, try anything else, and let me know what you found.
I mention my work because I think that slop is a thing we need to look past, I was explained right when I started doing piping in ml workflows that Machine Learning has many layers to it, and just how you can write performant code for a web based application where you might not understand entirely or intimately how rendering on the browser works exactly, there's at least 2 layers in ML, one is the science behind it, what is usually loosely referred to as "The math" and then there's the application layer, the part where you import a model and train it on a certain number of parameters, will one help you with the other? absolutely, is a deep understanding of linear algebra needed to tune a model? Not necessarily, most of the parameters are isolated and exposed as concrete input, you can start in either end and work your way to the middle if you so desire.
And so in the same way, knowing that there's slop in anything generated by an LLM is crucial, you can't lower your guard, but also, I think we should also recognise when writing something by hand, when using your own criteria for something is needed. It's not that you or I are more intelligent than the sum of the almost all human information, expressed as virtual intellect, is that you and I, depending on the subject, might have more context and more intuition about it.
Which takes me to my practical approach, I must confess I still read the code, I still read most of the code and look for obvious mistakes in typing and out of place useEffects for frontend code, unused abstractions, etc. And I mostly refrain from reviewing anything I'm not deeply familiar with. I'm in disbelief for the time being in software factories, api pricing will mostly kill most of the initiatives, maybe not 🤞🏽, like with most things I can be a sceptic and still try it out, no harm in testing correct?
For coding I don't let it cook, I ask for very specific and narrow tasks, at least professionally, some side projects I let it go at it, but then I need to spend a significant amount of time reworking or rewording stuff so that it's maintainable for me at least, I don't take access to these machines for granted. And I mostly try to identify patterns, I know that most of it is probabilistic so for stuff I know it's repetitive I create a script and perhaps a skill, but mostly a script, skills, believe it or not are ignored quite often, which I see it as a readme for the script.
As for new processes that can be subjected to "aification", the following might be an over simplification; I believe that approaching any process and then capturing it in steps that are semantic and domain-driven, meaning that they reflect the inherent and endemic language of the task at hand (in a candy shop, you can call what is sold item or candy and I'm pretty sure candy aligns and conveys more meaning) and then you make a distinction between tools, processes, docs, business layer, application layer, actions, transactions, etc and you reduce those steps to the minimum expression and the you still have something that might need generation then you focus on isolating that part and perhaps if it's discrete enough you might use a narrow model, and if that doesn't cut it then you can use an LLM, and then, once you've decided that you need some sort of criteria you still have to think of providers, cost, scalability, etc.
So what do I use LLMs for these days other than for coding purposes, where I mostly spend my time writing narrow parameters and guiding them through the broad strokes:
And what about automation? only if simple automation is not good enough
Repetitive tasks? simple automation is usually cheaper
what about criteria based tasks where a simple value is not enough? and nuance takes place? well to a degree, if I can be the last reviewer the better, but I understand my time, like everyones is limited.
What about as thought sparring partner, limited, I've spent way too long talking to statistically generated text machines, I already know I'm right, they only confirm it... Yeah it generates an incredible bias that is hard to ignore
What about as a tutor? ah well now we're talking, I've had LLM's teach me stuff, I usually ask for a topic to be disclosed incrementally, without assuming any prior domain knowledge and then let it quiz me at the end or step the conversation so that I can mark that I understood a bit of info and then proceed, problem is I like to fork conversations jajaja and so sometimes it ends up in a royal mess, lost points and all, they don't quite work when steered too out of place, and the fact that they don't resist, well...
And I think that's where their true application relies, imnsho we shouldn't have called it artificial intelligence, we should've called them Assistive Intelligence, with the right setup you really can make it teach you almost anything, I know I'm a community taught (opposed to self-taught, no one does anything on their own) Software Engineer and I struggled back when I started, I couldn't make sense of anything, nowadays I ask it to clarify something, do it incrementally, verify my knowledge, and almost always tell it to help me build an intuition for it, get me to a state where I'm trained to identify patterns and act on them and then obviously field test it, with and without the assistance. It beats all else for me. Now obviously the cognitive load of always learning something is way higher than tagging along for the ride, set things on auto pilot 🤖🏎️... but it also means facing blank moments of embarrassment when being asked about something you're in charge of and your only answer is... well... 🤡.
I genuinely think that education is the future of AI, all other applications might prove too expensive, difficult to evolve, difficult to adapt, awkward, over engineered, etc... or not at all, they might prove useful and steady and scalable, but secondary to keep improving and training humans, that's the extent of my experience so far at least applied to coding.
And yeah I've seen image models do incredible things, ads, short form vide and stuff that was thought as impossible, but they almost always have tells, and for those that don't it's because they're usually close to reality... in which case, why not use the real thing? I guess scale and variation, there is something there for sure. But as with everything else best results remain in the domain of those who know what they want out of it, but an ai tutor can definitely at least point you in the right direction.
Where they excel however is at extracting a coherent path from rambling,
We also know that LLMs reward expertise, if you ask them about a topic you know nothing about how are you to judge your own grasping? You can't and then LLM's tend to agree with you, they tend to tell you that you're right, but watch a doctor or a lawyer or any professional, they get to answers you wouldn't, they know how to frame and evaluate
So in essence what I'm saying is slop is the norm for now, it might not be in the future and maybe we'll be able to forget about it, but today, today is a given, as for when asking a model to teach you anything, ask it for the elements needed for you to train in it, ask it how to develop an intuition for it and to give you pointers and references, that's less prone to slop in my experience, and allows you to have this face instead when asked about it: 🤠