This isn’t entirely new. The industry has gone through waves like this before, from the rise of the internet and the advent of social media to mobile development and every new round of web technology in between. But the rapid progression of LLMs speeds up this cycle, which can leave teams to feel like they’re losing control.

Here’s what our study uncovered about the causes of AI fatigue and some practical ideas for successfully navigating these pain points.

73% feel pressure to evaluate every AI trend

Nearly three-quarters of survey respondents told us they feel significant or moderate pressure to evaluate every new AI trend despite limited bandwidth.

So, what’s a developer or product manager to do when the tenth email this month asks about another new AI tool? Start by looking behind the question. Ask executives what they’re hoping to achieve by adopting an AI trend. Are they looking for speed? Security? Once you know the objective, you can steer the conversation to what your team’s already doing to reach it.

It’s also 100% fine to admit you don’t know a given tool yet. In fact, change your goal from understanding every new tool to keeping a general eye on trends, so you can choose a promising tool to try out later.

59% struggle to evaluate AI technologies consistently

More than half of technology leaders told us they struggle significantly or moderately to evaluate new AI technologies consistently across governance, security, and operational requirements.

This seems like a messy problem to untangle, but the good news is you already know how to do much of this work. Start with the framework you already use to evaluate technology service providers. If you’re giving your whole development team access to Claude or ChatGPT, it means you’re putting data on someone else’s server. Set up and evaluate a data management agreement just as you would with any other Cloud provider.

You’ll also want to consider how strong a given tool’s adoption might be in the marketplace and how likely it is that the tool will be around later. Finally, evaluate whether the company uses trusted business practices and aligns with your industry’s priorities.

When you test out a new AI tool, make it as low risk as possible by setting up a developer sandbox and running short pilot studies that take weeks, not months. Then try basing your experiment around this simple framework: apply the tool to a concrete problem, build a working baseline, and improve iteratively. Stay away from mission-critical systems and use mock or anonymized data to keep personally identifiable information (PII) and protected health information (PHI) out.

Pilots show how a tool behaves, what it costs, and where the trade-offs live. This information all informs later governance decisions. Once you decide to move into production, you’ll want to follow your organization’s usual controls for new technology tools and collaborate with key partners in product and compliance.

62% say engineering fatigue is growing

Nearly two-thirds of survey respondents said engineering fatigue is growing significantly or moderately as teams manage AI-generated code alongside legacy systems.

Reviewing AI-generated code happens along a continuum of approach and time investment:

• Pairing with an LLM: It’s a powerful partner but checking every step can be slow.
• Reviewing afterward: Going over every piece of AI code can bury developers in work.
• Letting AI review AI: This works for catching errors but only up to a point

At the far end of this effort continuum is the largely hands-off dark factory approach. You don’t watch what happens inside the AI code factory. Instead, you define tests and controls that measure success and limit risk.

Take a financial app that recommends trades. Tests might require expected returns between 5% and 20% and block the AI from executing trades. The AI can write code overnight, but it’s never allowed to change the tests. It stops only when it can’t meet the criteria or reaches a judgment call.

Our team sometimes takes what we call a dim factory approach in our agentic coding. Most of the lights are off (we’re not hands-on with the AI coding process), but we can still walk through the factory and see what’s generally happening without reviewing every detail. It reduces the burden on developers while still maintaining additional oversight.

The right approach to AI-generated code depends on your culture, your risk tolerance, and whether you’re prioritizing feature speed or maintainability. Legacy systems also come with legacy cultures, and those need navigating, too.

Emotions loom large

What all these stats hint at but don’t outright state is that there’s a gigantic emotional component to rapid AI change, too. Many developers take pride in crafting code, so naturally, there can be significant grief in turning over this hard-won skill to AI.

One way to re-frame the situation: Focus on the new craft you’re developing as you direct an LLM. As with any emerging technology, you’ll make mistakes, but you’ll also work through them and learn. Focus on personal skill growth vs. letting imposter syndrome creep in.

Another big source of mixed emotions about AI: Decision fatigue. Constantly evaluating output from a dozen agents leaves no room to breathe, especially as AI pushes productivity expectations higher. This volume can leave people feeling burned out. In a recent workplace survey from Lenny’s Letter, 55.7% of tech workers reported significant burnout, up from 44.7% in last year’s survey. There’s no easy answer, but as you’re trying to navigate all the change, focus on the decisions that matter most, set a manageable pace, and accept that everyone has a cognitive load limit.

There’s no one-size-fits-all fix

How your engineering team approaches AI depends on business context and whether you’re defining the team’s overall relationship with AI or simply matching a tool to a problem. Plus, all this context and technology keep changing at a rapid pace.

Want help navigating it? Our AI and machine learning expertise can guide you through governance, focused pilots, and strategies for managing AI-generated code. Let’s talk.

ReadySetGo is now open for anyone to try (fine print: within a daily new user limit).

ReadySetGo helps you turn a rough idea into a build-ready plan by walking you through a product sprint. It guides you through our team’s structured process to pressure-test your idea, challenge your assumptions, and make some of the important decisions that should happen before you start building.

For a limited time during open beta, it’s free to use.

A product expert that doesn’t just agree with you

There are already a lot of powerful AI tools that can help you build software faster. But building faster isn’t an advantage if you’re building the wrong thing.

We’ve spent more than 23 years helping founders and product teams turn back-of-napkin ideas into products worth investing in.

ReadySetGo is an experiment in making more of that expertise available to anyone. We’re baking in the product discovery and shaping practices our teams use every day and have iterated on for years. In the process, our team has been working against one of the biggest blockers for general AI tools to be good product partners: their tendency to agree with you.

ReadySetGo is a strategic sparring partner for you. It asks questions, challenges assumptions, brings in research, and helps you get increasingly specific about the problem, audience, opportunity, and first version of your product.

One of our early alpha testers described it as:

The first AI tool that actually understood my business. It’s a thought partner for product design.

You leave the ReadySetGo sprint with a validated build plan you can hand off to an AI tool like Bubble or a development partner like thoughtbot.

We’ve been building it in public

If you’ve been watching our weekly livestream, AI in Focus, you may have already seen pieces of ReadySetGo come together.

Our founder and CEO, Chad Pytel, along with cohost and Development Team Lead Louis Antonopoulos have been leading the live development of the product alongside many from our team across disciplines. You can continue to tune in to see them work on updates throughout the beta period on our YouTube, Linkedin, and Twitch.

Help us test it

ReadySetGo is now in open beta, and it’s free to use for a limited time.

Bring a rough product idea and try it out. The tool will guide you through 10 steps that you can go through at your own pace. You’ll pressure-test the idea with a process used by product experts and leave with a plan you can build from.

We’re still learning. We want to understand where ReadySetGo is genuinely useful, where it falls short, and whether we’ve succeeded in creating something meaningfully different from opening up a general-purpose AI tool and starting a conversation.

Start a free ReadySetGo sprint

Rob Whittaker is Director of Software Development at thoughtbot. He has over two decades of professional web development experience and has worked with Ruby and Rails since 2012. (He’s also the co-organizer of NWRUG!)

We’ll be at the venue for pizza from 6:30pm with the talk starting at 7pm. Afterwards we’ll retire to The Crown & Kettle for a drink. The venue, pizza, and drinks are kindly provided by NWRUG’s sponsors Fatsoma and thoughtbot.

Register here if you would like to attend. Full details on this (and all NWRUG events) over on NWRUG’s website.

The topics are different, but they both come back to the same bigger question: what does the Product function need to become next?

Has Product become the bottleneck?

The roundtable will explore what it actually means to become an AI-native product organisation, beyond simply adding AI tools to an existing process.

How should the role of Product change when teams can move faster? Which parts of the traditional product process still help, and which ones might need to be rethought?

There probably isn’t one clean answer, which is exactly why it should make for a good conversation.

Should Product own more of the commercial outcome?

The second roundtable focuses on another shift we’re seeing in product leadership: moving closer to revenue, growth, P&L, and commercial accountability.

So what changes when Product is recognised more explicitly as a revenue-generating function?

That opens up questions around what Product should own, which metrics matter, how product leaders work with commercial teams, and whether greater accountability changes the way priorities get set.

Join the conversation in London

If you’re attending the London Chief Product Officer Conference on 29 October, you can join Bethan for either of the two discussions.

We’ll see you there.

Us AI skeptics have good reasons for our fear/hatred/disgust even as we use LLMs daily:

• adverse effects on the environment and the economy
• data misuse
• training on works of unwilling authors
• bias, hallucinations and lack of accountability
• stealing our jobs
• and more!

So please build your AI products to let me opt in.

Opting in progressively

The idea is to cater to everyone on the AI love/hate spectrum. You do that by giving me granular control over just how much AI is used, and just which personal data is shoveled into the firebox. I want to

1. see how my data will be used by AI
2. see what benefit I get
3. grant permission

And I don’t want to do that by reading through your Terms of Use and Privacy Policy. It needs to be in context because I will decide on a case-by-case basis.

Just-in-time consent

I want to grant zero permission up front. I want you to ask me for permission when you need it. No AI until I say so.

In mobile apps you already use this pattern to request extra permissions like location sharing and push notifications. You know that if you pop a notification as soon as the app opens, I’m going to say no. So you wait until you can demonstrate usefulness. We call this pattern a “soft ask” or “pre-prompt” or “pretty please”.

It’s the same way with AI: if your registration screen says “hey BTW we’re going to send all your data to LLMs and force you to chat with our AI agent!” I will have misgivings. But if you wait a bit and say “hey, we see that you’re stuck on this form - want to enable our agent to help?” - I will probably say yes, and I will probably say sure you can consume my data for it. You have shown that you respect my choice and you have demonstrated value.

Once I’ve given the OK to use AI, that doesn’t mean I’m OK with you shipping it every single byte of data you’ve compiled about me. Sure, it’ll make your AI more productive and accurate if it can access all my PII, credit card statements, and health information. I don’t care. Again - you need to prove value.

I haven’t granted wholesale permission with the click of one checkbox and submit button. You need to ask for permission progressively, and then let me retract my permission later.

How your agent can ask for permission progressively

It’s all about the context and prompting. Here are some techniques I’ve used or observed:

Conditionally build the list of agent tools or skills based on what the user has given access to.

In your LLM system prompt, give a list of the data items that the user has and has not consented to sharing.

Prompt your agent to ask for permission. Something like “This user (has/has not) given you access to their financial accounts. You may ask for permission to access the accounts - direct the user to their Settings screen to enable access.” or maybe “This user has granted read-only access to their document. If you need to edit it, please ask for write access, and explain exactly what you need it for.” Make sure the agent knows that “NO!” is a valid answer. It shouldn’t guess or hallucinate if it doesn’t have the real stuff.

The important parts are to let the LLM know

• what does it currently have access to?
• what could it get access to?
• how can it use that data to benefit the user?
• how can the user grant access?

Pre-approval

If you can’t do just-in-time consent, then you’ll need to get consent ahead of time. And remember - keep it granular and tell me how I benefit. If I’m linking health data to your app, you need more than just a single “yes please deliver my entire disease history to Anthropic” “Allow additional information sharing with our business partners" checkbox. Instead consider something like

• [ ] allow our AI agent to book appointments on your behalf
• [ ] share anonymized appointment notes with LLM partners so we can give you a summary
• [ ] review X-rays and other radiology imaging with AI to flag abnormalities

Ask for permission to use data with AI at the place where you’re collecting that data. If I’m filling out a registration form - that’s where I should grant permission to share the data in that form. If I’m connecting my calendar, that’s where I should grant permission to let an AI manipulate my calendar.

Clawback

If I can give your LLM permission to slurp up my PII, I better be able to revoke that permission! So give me a user interface for that. Show me what the AI is consuming today and what it’s used for and what the effects will be if I uncheck the checkbox.

If I change my mind and want to remove AI access to my data, that means really removing it. Clear your prompt caching. Sanitize agent conversation histories. Delete your logs. Remove AI access to those tool calls.

Do not tempt me! I dare not take it, not even to keep it safe, unused. The wish to wield it would be too great for my strength. –Gandalf

With chat-style agentic AI, you have the new ability to ask for lots and lots of freeform sensitive data. It’s a free text input field and I might accidentally reveal more than I wanted: API keys, passwords, library card numbers. DON’T TAKE IT. You don’t want the responsibility of safeguarding my PII or secrets. You don’t need to risk Claude going on a spending spree.

How can your agent reject PII and secrets?

Here are some techniques I’ve used:

Prompt the agent to avoid asking for secrets. When I worked on an agent that could configure API calls, we added a system prompt instructing the LLM to never ask for API keys, tokens, or passwords. The LLM should instead teach the user how to provide this info in a secure manner. We also asked the system prompt never to repeat private, secret data given by the user. Just because it hit our logs once doesn’t mean it’s fair game for the agent to use as context.

Detect PII and secrets in the browser, before they even get to the agent. I used regular expressions to detect API keys. You could also use a lightweight ML model to detect frequently shared PII. Then warn the user “hey, looks like you dropped this - are you sure you want to share it with us?”. Or censor it out. thoughtbot’s top_secret Ruby gem does this same thing on the server-side.

What if he says no?

So what do you do if I say “no thanks, no AI please?” The same thing you’d do if I said “no I won’t share my location data”. You give me the fallback option. The manual one. Maybe my experience is not so great or maybe I have to do more work. But my principles remain intact.

A tale as old as the internet

Guess what - all this advice isn’t new for LLMs. It’s a spin on the demands coming from the privacy movements of the early 2000s and before, back when we learned how our data was being taken by governments and sold to spammers. All I ask is to let me control my exposure to AI, and give me a good reason to hand over my data to the robots.