I Vibe Coded an App and Learnt Exactly What I Thought I Would

If you read my blog On No / Low Code and AI for Automation, Testing, and Quality Engineering, then you probably already know how I feel about AI in software production and what the kids call “vibe coding”.  Feel free to read that first if you haven’t already.

I won’t keep you waiting.  I learnt nothing.  No, really.  Many things were confirmed to me, but nothing was new.  And that definitely isn’t because I’m a master programmer.  It’s because AI removes the need to learn anything.  Some may argue that’s the beauty of it, but I think it’s more of a danger.  We’re handing over more and more control, trust, reliance to AI, on the basis that AI really is intelligent; that it’s thinking and solving problems.  But it isn’t.

I think people are confused between artificial intelligence and synthetic intelligence.  What I mean by that is something which appears to have intelligence but does not, vs something which is intelligent because we manufactured it to be.

That’s kind of a confusing distinction, especially if you think about the use of “artificial” in “artificial sweeteners”, for example, and people better at English than me might say I’m still using the wrong words.  That’s fine.  But my point is that one thing is an illusion, and the other is simply man-made.  Think of synthetic fibres, for example, which aren’t naturally occurring, but fibres nonetheless.

What I’m getting at is that the thing we’re trusting, relying on, making ourselves vulnerable to, has no real capacity to comprehend any of that.  It’s not designed to think or reason.  It’s designed to tell you what you want to hear, and be extremely eloquent about it.  That’s what makes it so convincing.

Yes, it was very convenient to be able to spin up a fully fit-for-purpose app in two or three days, but I don’t dare publish it, because I simply don’t know what’s in it.  I provided very detailed instructions, pushed back on architectural design choices, and tried to code review everything, but…  It was just so easy not to.  To skim the top of the file; to jump to the interesting parts and skip the rest.  As a trained software testing professional who prides themselves on attention to detail, I did not pay nearly enough attention.  And I knew I wouldn’t.  What happens when something with literally no humanity is given lots of power and little to no oversight?  That’s the danger of AI.

Are you paying enough attention?