Of Looms and LLMs: A Cautionary Tale about GenAI

Nikolay Gekht
Sep 15, 2025
5 min read

by Someone Who Remembers Life Before the Hype Cycle

Let us begin with the obvious: everyone, everywhere, is talking about AI. Loudly. In all directions. Some believe it will save us; others think it will destroy us, and a few suspect it might merely rearrange our calendars with passive-aggressive flair.

As for me, well, I have no intention of joining either camp. I’m far too old, cynical, and frankly too busy sipping lukewarm tea to march behind yet another flag. So instead, let’s consider both sides: the good in the bad and the bad in the good.

The Jacquard Loom of Our Time

A good example of a historical cousin to GenAI is, arguably, the Jacquard loom. A fine tool from the early Industrial Revolution that enabled the mass production of intricate textiles. Beautiful results, at a fraction of the time, with the added benefit of employing less-skilled workers for less money (progress, as ever, knows how to count pennies.)

This is, in essence, what GenAI is doing today. It boosts productivity, opens once-guarded professional silos to the masses, and lets the apprentice do what once required a master.

The data, naturally, is already piling up like unread Jira tickets:

Brynjolfsson & Li (April 2024): +14% productivity boost in task resolution; +35% for novices in customer support roles [1].
Dillon et al. (May 2025): GenAI power users spend 31% less time on emails and wrap up documents faster [2].
Deloitte (2024): 74% of firms said GenAI met or exceeded ROI expectations [3].
Microsoft (April 2024): Every $1 spent on GenAI returned an average of $3.70 [4].

When even finance departments start calling something a good investment, you know it’s no longer just a passing trend. By late 2024, nearly 28% of U.S. workers used GenAI in their daily routine (CFO Dive) [5]. That’s not “early adoption” anymore. That’s Tuesday morning.

So yes, it’s here. Like the loom before it, GenAI is rapidly becoming a fixture of modern business. Ignore it, and you risk waking up in the discount bin of your industry.

But… There’s always a “but”…

The Return of the Loom’s Revenge

Just as the Jacquard loom transformed the industry, it also transformed craftsmanship into something else entirely. Artisanship faded. Mastery was replaced with button-pushing. Today, you can still find a master weaver, but they’re mainly on Instagram and possibly French (my oldest daughter is one of them, by the way).

And this is the uncomfortable parallel we need to talk about.

A recent MIT study (Nataliya Kosmyna et al., Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI [6]) hooked writers to EEGs and compared those using AI tools to those who weren’t. The result? The ChatGPT crowd consistently showed lower brain engagement across the board. Linguistically, behaviourally, and neurologically they underperformed.

In short: more output, less thinking.

This observation won’t surprise anyone who’s ever taught a teenager with access to Google Docs and a deadline. But it should concern those of us responsible for developing actual talents.

Because if we’re not careful, we may find ourselves with teams who can churn out impressive results, but don’t know why they work or what to do when they don’t.

Lessons from 36,000 Feet

To find a solution, I propose we look upward, literally. Commercial aviation offers a chilling case study of overreliance on automation.

Modern pilots spend less than 0.5% of their time actually flying the aircraft [7]. That’s not a typo. The result? When things go wrong, many can’t cope. Several high-profile incidents in the past two decades were caused (or worsened) by pilots who simply hadn’t had enough “manual time” to build instinctive, confident reactions.

A 2010 study [8] from Cranfield University showed that regular practice and real hands-on, eyes-on-the-horizon flying directly correlates with better emergency handling.

Airlines that care about safety ensure pilots practice flying, even when they don’t have to. We, too, should consider this when building, managing, and mentoring teams in an AI-augmented world.

What Now?

As my grandfather used to say, “If you can’t avoid it — lead it.”

So here’s what I’d suggest:

Adopt with intent. Use AI where it makes sense, not where it looks shiny.
Set clear goals. Ensure GenAI solves problems you actually have, not ones your vendor told you about.
Design for people. Let AI boost your people, not replace them. The goal is augmented intelligence, not abandoned intellect.
Manage the risk. Security, ethics, vendor lock-in: these aren’t side quests, they’re the price of admission.
Keep skills sharp. Bake “manual mode” into your teams’ operating rhythm. Challenge thinking. Review raw code. Write from scratch, sometimes just because.

Above all, build feedback loops. The second way of DevOps isn’t just a buzzword; it’s how you keep people and systems honest, connected, and improving.

In Closing: A Modest Plea

GenAI is neither saviour nor saboteur. It’s a tool. A powerful, slightly overeager, occasionally hallucinating tool.

Use it wisely and you can scale great heights. Lean on it blindly, and you may find your feet have left the ground entirely until they reconnect, suddenly and painfully, with the floor.

Let’s not burn the looms. But let’s not forget how to weave.

This is the article I wrote for the 2025 issue of Agile Bulletin.

We started the Bulletin as part of the Agile 2023 experience, with a small printed issue at the expo. Since then, it has become a yearly tradition: each year, we bring a fresh printed edition filled with voices that don’t always get through the noise of keynote stars and talking heads.

If you’re looking for straight, no-nonsense writing about Agile: without buzzwords, hype, or shiny slides, Agile Bulletin is the place to go. It’s written by people in the trenches, sharing what works (and sometimes what doesn’t) in real projects.

You can download past issues and subscribe to future ones at agilebulletin.org.

Of course, this piece was made with AI at my side.

The images? Obviously AI-made. I can’t draw a straight line with a ruler.
AI dug up stats and links so I didn’t have to spelunk through a labyrinth of links and PDFs alone.
It moonlighted as a ruthless editor: poking holes, spotting contradictions, and generally out-perfectionisting my inner perfectionist.
It also played coach, patiently nudging me toward that soft English humor I was aiming for. (English isn’t my mother tongue; more of a stern step-mom.)
And the title. I asked, “Okay, what did I miss?” AI suggested a better one. It was annoyingly right.