Beyond Vibe Coding: Why AI Prototypes Need More Than Better Prompts

A Working Prototype Is Not a Dependable System

AI-assisted development has changed the economics of prototyping. With the right tools, a working application can take shape in days rather than weeks. For technical leaders, that creates an opportunity to explore ideas and validate assumptions before committing substantial resources.

But the ease of building a prototype can obscure a harder question: Can the system reliably execute the business process when users depart from the expected path?

I encountered that distinction while developing Resume Workbench, an AI-assisted application for tailoring resumes and cover letters to job descriptions. The goal was not simply to generate polished documents. It was to preserve factual evidence, identify gaps, solicit human clarification, and evaluate outputs before they were accepted.

I pursued two different approaches:

1. a custom Python application and, later,
2. a hosted AI environment configured with reusable skills.

Both could perform useful tasks. Both also exposed limitations in workflow continuity, human review, and visibility into execution.

The experience changed the way I evaluate AI delivery. The central challenge was no longer whether an LLM could perform an individual task, but whether the surrounding system could coordinate those tasks into a dependable, governable process. The surrounding system is often called the agent runtime.

Two Approaches, One Recurring Problem

First: Build the application

For V1, I used ChatGPT to help develop requirements, design the solution, and scaffold a Streamlit application backed by 13 modular Python services. The system used structured schemas and a local ChromaDB evidence corpus to support job analysis, evidence retrieval, drafting, and review.

flowchart TD
    subgraph MacroFlow ["Figure 1: Resume Workbench V1 Application Architecture"]
        subgraph UI["Presentation & Workflow Control"]
            ST["Streamlit UI"]
            SESSION["Session State<br/>Active Workflow Progress"]
            ST <--> SESSION
        end

        subgraph SERVICES["Application Services"]
            PY["13 Modular Python Services<br/>Analysis · Retrieval · Drafting · Evaluation"]
        end

        subgraph DEPS["AI & Data Dependencies"]
            LLM["OpenAI API"]
            VECTOR[("ChromaDB<br/>Vector Store")]
            FILES[("Local .MD Files<br/>Resumes & Evidence")]
        end

        SESSION <--> PY
        PY <--> LLM
        PY <--> VECTOR
        PY <--> FILES

        GAP["Architectural Gap:<br/>Workflow Continuity Dependent<br/>on Active Session"]

        SESSION -.-> GAP

        class SESSION tollgate
        class VECTOR,FILES storage
    end

Within days, the prototype could execute the intended sequence. The architecture was modular, and generated artifacts were stored locally. Yet the application’s active workflow depended heavily on Streamlit session state and procedural logic tied to the interface.

That distinction mattered when I tried to revisit a decision, incorporate additional evidence, or return after interrupting a session. Files could be saved, but the system could not reliably reconstruct where the business process stood and resume from that point. An isolated LangGraph proof of concept had not become the orchestration mechanism for the main workflow.

This was a design limitation of my implementation, not proof that Streamlit was inherently unsuitable. I had separated the application into services without adequately separating workflow orchestration from the user interface.

Then: Try a hosted approach

Rather than continue investing in a one-off application, I made a deliberate change in direction. Perplexity Computer offered a hosted environment where custom skills could reproduce much of the workflow without requiring me to maintain the entire application interface and supporting infrastructure.

That was a reasonable experiment: use an emerging platform to reduce custom development effort, then focus on the business process itself.

I migrated and refined the instructions through repeated use. Individual steps worked, and I eventually configured interactive clarification experiences to collect evidence and feedback. But the end-to-end process was difficult to manage. Clarification requests sometimes became disconnected from the broader workflow; progress was hard to see; and long periods without meaningful status left me unsure whether the system was working, waiting for input, or stalled. Recovery often required specialized prompts.

Moving to a hosted platform reduced the code I had to maintain. It did not remove the need for explicit user control of UI features, process state, reliable handoffs, and execution visibility. These observations describe the workflow I configured and used, not every capability of the platform.

Four Lessons That Changed My Architectural Priorities

The two implementations differed considerably, but the failure patterns converged. That was the important discovery.

1. Workflow state must survive the interaction

A complex business process isn’t just a conversation or a sequence of screens. It has current tasks, completed decisions, pending approvals, and dependencies between steps.

In V1, durable files existed without dependable restoration of the active workflow. In V2, conversational history did not give me a clear, actionable view of the process. Both experiences reinforced the need to design persistent workflow state and resumption explicitly, rather than treating them as incidental features of a UI or chat session.

2. Human review is part of the architecture

For Resume Workbench, factual integrity depended on a person supplying or approving evidence. Human involvement was not an optional final sign-off; it shaped the work throughout the process.

In both approaches, review was harder to manage than generation. A dependable design needs defined points where execution pauses, presents a decision, records the response, and resumes without losing prior work or repeating completed steps. Human-in-the-loop is a workflow requirement, not simply a prompt instruction.

3. Observability is a user and delivery requirement

In the hosted workflow, the most frustrating moments weren’t necessarily incorrect outputs. They were periods when I couldn’t tell what the system was doing or what remained to complete.

That experience reframed observability for me. Traces, status, failure reporting, and recovery aren’t only engineering conveniences. They determine whether users can trust a process and whether delivery teams can support it. A system that performs sophisticated work but cannot explain its current state is difficult to operate reliably.

4. Evaluation needs enforceable rules

V1 exposed a quieter problem. Its documented readiness framework applied weighted criteria, but scoring was largely delegated to holistic model judgment. One configuration set the readiness threshold at 85, while an evaluator prompt described the ready-to-apply band as 90-100.

That conflict was preventable. Thresholds, schemas, and permitted workflow transitions should be enforced in code where practical. More subjective questions (such as whether phrasing preserves the meaning of a career accomplishment) still require appropriate model evaluation and human review. Governance works best when deterministic rules and judgment-based assessments have clearly defined roles.

From Better Prompts to Better System Design

A turning point came when Thomas John, Practice Lead, AI & Digital Strategy, recommended Victor Dibia’s Designing Multi-Agent Systems. The framework helped me name the issues I had observed and think more deliberately about orchestration, persistence, bounded responsibilities, and human oversight.

It also clarified an important distinction: these failures do not, by themselves, prove that every complex AI workflow needs multiple agents. A conventional workflow engine or a stateful application may be sufficient. Specialized agents become useful when different tasks genuinely benefit from separate responsibilities, tools, or reasoning patterns.

For a future Resume Workbench iteration, I am evaluating a stateful orchestration approach that separates the interface from workflow execution, persists checkpoints, formalizes review gates, and makes evaluation criteria testable. A multi-agent design is one candidate, not yet an outcome I’ve yet validated in production.

The operating principle is straightforward:

flowchart TD
    subgraph GovernedFlow ["Figure 2: A Conceptual Architecture for the Governed AI Workflow for the Final Resume Workbench"]
    
        USER["User Request"]

        subgraph CONTROL["Stateful Workflow Control"]
            ORCH["Workflow Orchestrator"]
            STATE[("Persistent Workflow State<br/>& Checkpoints")]
            ORCH <--> STATE
        end

        subgraph EXEC["Governed AI Execution"]
            TASK["Bounded AI Tasks"]
            VALIDATE{"Validation Passed?"}
            REVIEW{"Human Approval Required?"}
            HUMAN["Human Review & Feedback"]
            OUTPUT["Approved Output"]

            TASK --> VALIDATE
            VALIDATE -->|No: Revise| TASK
            VALIDATE -->|Yes| REVIEW
            REVIEW -->|Yes| HUMAN
            REVIEW -->|No| OUTPUT
            HUMAN -->|Revise| TASK
            HUMAN -->|Approve| OUTPUT
        end

        subgraph OPS["Operational Visibility"]
            OBS["Execution Logs<br/>& Progress Status"]
        end

        USER --> ORCH
        ORCH --> TASK

        TASK -.-> OBS
        VALIDATE -.-> OBS
        HUMAN -.-> OBS
        ORCH -.-> OBS

        class STATE storage
        class VALIDATE,REVIEW,HUMAN tollgate

    end

Across the workflow, state should be recoverable, decisions attributable, and execution visible. The diagram is a target operating pattern, not a claim that the proposed rebuild is complete.

What This Changes for AI Delivery Leadership

The broader lesson isn’t about resumes, Streamlit, or Perplexity. It’s about the distinction between task feasibility and system readiness.

As AI makes functional demonstrations cheaper, technical leaders need to become more disciplined about the questions they ask before scaling a solution:

• Continuity: Can work be interrupted, revised, and resumed without losing decisions or repeating costly steps?
• Accountability: Where must people review evidence, resolve ambiguity, or approve consequential outputs?
• Visibility: Can users and operators see progress, detect failures, and recover predictably?
• Quality: Which requirements can be enforced programmatically, and which demand evaluation or expert judgment?
• Architecture: Does the process actually require multiple agents, or would a simpler orchestration model be more reliable?

These aren’t purely engineering questions. They influence delivery risk, operating cost, vendor evaluation, user adoption, and the investment required to move beyond a demonstration.

I also learned something about leadership judgment. Choosing a hosted platform after building V1 was not a retreat from engineering; it was an attempt to test whether a different operating model could remove unnecessary custom work. When it exposed similar limitations, the right response was not simply to refine prompts indefinitely. It was to revisit the requirements and the architecture.

My prototypes were experiments, not enterprise production deployments. But they provided direct evidence of where apparently capable AI workflows can become fragile—and helped me develop a more rigorous set of questions for future delivery decisions.

The real measure of an AI solution isn’t how quickly it produces an impressive result. It’s whether the surrounding system can deliver that result reliably, transparently, and under appropriate human control.