AI & IntegrationEngineering Series: How to use the Explore/Exploit framework to stay ahead

The conversation surrounding AI often gravitates toward two extremes. One camp views AI as an existential threat to the craft (software engineering); the other sees it as a panacea that will eventually automate away every architectural hurdle.

For those in the trenches, the reality is far more nuanced. Over the last six months, the pace of integration has accelerated, creating what Shon Urbas, Pandium CTO,  describes as a "tight" environment where workflows are shifting almost monthly. Recently, Shon and Lizzy, Pandium Engineer, sat down to discuss how these tools are actually impacting the day-to-day lives of engineers, and why the most vital skill in 2026 isn't prompting, but skepticism.

The Framework: Explore vs. Exploit

To frame the current state of engineering, Lizzy introduced a concept that serves as the foundation for this episode, borrowed from the book Algorithms to Live By: the Explore/Exploit Tradeoff. This is a mathematical problem found in computer science and probability, but Lizzy applies it to the emotional and technical friction engineers are feeling right now.

The basic idea is a choice between two states:

• Exploitation: You have found a "good thing" (a workflow, a language, or a tool), and you sit tight to maximize the value you get from it.
• Exploration: You step away from your "known good" to see if there is something better out there, knowing that searching has a cost.

Lizzy: "Under what circumstances do you decide I’m gonna go explore and find something better versus I’m gonna sit tight and exploit this good thing I’ve already found? ... If you haven’t done much exploring, you probably should, because you don’t know what’s out there."

Why 2026 is an "Explore" Year

Lizzy argues that while engineers naturally prefer the "Exploit" phase (where they are fastest and most comfortable) three specific conditions currently force us back into "Exploration":

• High Variance in Quality

‍If you check five different AI tools and get inconsistent results (some amazing, some "crappy"), you haven't yet found the ceiling of what is possible. You must keep exploring to find the true range of quality.

• A Changing Environment

‍In a stable environment, you eventually stop exploring because you’ve found the "best" option. But AI models change monthly.
Lizzy: "If things are changing a lot... if I double-checked something and it was bad before, but now it’s good... you should probably also keep exploring."

• The Information Gap:

Shon admitted that his conservative AI tooling use can often keep him in the "Exploit" phase for too long.
Shon: "It’s interesting to think like, what am I losing out there? Do I exploit what I found? Am I really, really happy and safe at where I’m at? Or do I keep on spending time on these new things?"

The "Learning Curve" Tax

The reason this tradeoff feels so heavy is the "switching cost." Both Shon and Lizzy discussed how switching to Git Worktrees (a way to have multiple branches open at once) initially made them slower and was "annoying" to set up.

Lizzy: "To change the workflow, you’re going to be slower at the new thing for at least a little bit. I had to install again in certain worktrees and I was like, 'Why is this not working? I hate this worktree. Scrap it.' But the idea that I could have an unstopped conversation with Claude about that branch, there's no question that’s going to be worth my time."

The takeaway from their framework discussion is clear. In a year where the tools are evolving at a breakneck pace, "staying safe" in a familiar workflow is actually a risky bet. Exploration is the only way to ensure you aren't exploiting a tool that has already been surpassed.

Breaking the "Yes-Man" Loop

A common pitfall with Large Language Models (LLMs) is their tendency to be "yes-men." Because these models are optimized for helpfulness, they often validate a developer’s initial idea… even if that idea is flawed.

AI models are notoriously agreeable. To help break this "Sycophancy Trap," Lizzy uses Plan Mode to pitch competing ideas against each other. By asking the AI to "Design two ways to solve this and tell me why my first instinct might be wrong," she forces the model to act as a critic rather than a "yes-man."

Puppeteer and "Testing the Tests"

Perhaps the most critical phase of the engineering loop is Quality Assurance. Shon and Lizzy both cautioned against the temptation to let AI handle testing autonomously. Shon shared a recurring issue where AI-generated tests were technically valid but functionally useless.

Shon: "I’ve seen it just assert true at times and you have to review the test. You can't just trust... that's how I discovered this assert true test where it was passing after I broke the thing on purpose."

Lizzy echoed this, noting that AI frequently mocks components that don’t need to be mocked, creating a "green" test suite that masks underlying failures. The consensus was clear, an engineer must always see a test fail before they can trust a pass.

Shon suggested pushing this further by using a Puppeteer MCP (Model Context Protocol). Puppeteer is a library for writing end-to-end browser tests by simulating mouse movements and keyboard entries.

Shon: "For QA specifically... I’ll give Claude Code an active web session by copying it out of the console in my browser. So it has the access token and all the headers as if it was a browser... I want to explore driving Puppeteer in a way that... outputs the script that then we can review and then run."

By letting the AI drive a "headless" browser to perform the boring, repetitive clicks of manual QA, engineers can find non-deterministic bugs faster, provided they remain the final reviewer of the script.

"Vibe Coding" and the Dunning-Kruger Trap

The conversation eventually turned to the cultural phenomenon of "Vibe Coding", the rise of non-engineers building applications through prompts. While Lizzy celebrated the "wizard-like" feeling of creation that AI provides newcomers, she warned of the Dunning-Kruger effect -- the tendency for those with limited knowledge to overestimate their competence.

Lizzy: "AI compounds that in a huge way because you can learn about things very quickly with AI... But it can make you think that you have learned more than you did or make you feel like you've got AI absolutely figured out. The scalability of building a whole system is still a huge challenge and you wouldn't want to rest on AI for that."

Shon noted that while "vibe coding" might suffice for internal prototypes, it often lacks the rigor required for production-grade systems. He pointed to the rise of people attempting to perform legal work using ChatGPT, only to be corrected by judges when the AI’s "expertise" turned out to be a hallucination. In the same way, a "vibe-coded" app might look great on the surface but crumble under the weight of a thousand concurrent users.

Vibe Coder + "I built an app, I'm a senior architect." = May lack understanding of scalability, security, and edge cases.

Experienced Engineer + "I use AI for speed, but I verify every line." = Understands that AI is a tool, not a replacement for fundamental knowledge.

"I Am the Author"

As the session concluded, Shon shared a personal "commandment" he keeps in his configuration files, a digital reminder of the boundary between the engineer and the agent.

Shon: "One of the very first commandments I put in was: 'You are not the author. I am the author. You are not responsible.' I am responsible for every line I post and every line I approve."

Ultimately, AI is a tool that enhances the craft, but it does not replace the engineer’s fundamental responsibility. Whether we are exploring new models or exploiting familiar workflows, the human must remain the primary architect. With AI-assisted engineering, curiosity is an asset, but skepticism is a requirement.