Defining Your MVP Goals

This is the first post in the "MVP to Production" series — a practical guide for builders who want to ship real generative AI products, not just cool demos.

Picture this: a team spends three months building an AI assistant. The demo is slick. Executives love it. Someone makes a slide deck. Then it goes into a backlog, and six months later nobody can tell you what happened to it.

This isn't a rare story. It's the default outcome.

MIT's NANDA initiative studied 300 enterprise AI deployments and found that 60% of companies evaluate AI systems, 20% reach a working pilot, and only 5% actually go live [1]. The other 95% built something that worked in a controlled environment and then quietly died on the path to production. Zero measurable business impact. Not "less than expected." Zero.

The problem is almost never the model. The models are good. The problem is that most teams skip the foundational steps, and those skipped steps are exactly where AI projects stall. This post is about those steps — and how doing them in order is what separates the teams that ship from the 95% that don't.

Start with the problem statement

This may seem straightforward, but it's surprisingly easy to approach it incorrectly. The bad thing is to begin with the technology and then try to identify a use case. This often leads to a product that, while technically impressive, addresses a problem that doesn't exist.

S&P Global surveyed more than 1,000 enterprises and found that wrong use case selection was cited as a root cause of failure by 43% of respondents [2].

Begin by articulating, in simple terms, the precise problem you aim to solve.

When it comes to AI, opt for them only when a simpler solution truly falls short, rather than as your automatic starting point. The honest development path typically starts with a single LLM call to gauge its effectiveness.

Start small, demonstrate the value. Then, you can justify adding complexity.

A real example: the R&D engineering agent

One of the most common patterns I'm seeing right now is R&D teams trying to solve knowledge sharing and incident resolution across their engineering org. The problem is real and painful: an alert fires at 2am, the on-call engineer jumps between Slack threads, Jira tickets, runbooks, and observability dashboards trying to piece together what's happening and what the fix is. Knowledge of a specific SDK or previously solved RCA is sparse across several systems.

So teams start building their own internal AI engineering agent - something that connects to Slack, Jira, their observability platform (Datadog, Grafana, whatever they use), and internal docs, and can help surface relevant context, suggest fixes, or even kick off remediation steps automatically.

It's a genuinely exciting use case.

Scope the MVP

One well-executed feature is far more effective than five poorly executed ones.

For an AI MVP, pinpoint the single most vital function your product must perform, and focus solely on that. Everything else belongs in phase two. If it doesn't play a crucial role in validating your core hypothesis, eliminate it. This can be challenging, especially with a grand vision. However, an MVP that clearly demonstrates one key aspect is far more valuable than an overcomplicated prototype that fails to provide any insights.

In traditional software, an MVP is the smallest thing that works. In AI, that definition breaks. A model that works 80% of the time feels broken to users who expect it to work all the time. An analysis of 1,200 production LLM deployments found that reaching 80% quality happened relatively quickly — pushing past 95% took the majority of total development time [2].

That's why the real AI MVP isn't just the smallest thing that works — it's the smallest system that can survive production: real users, real data, and real consequences for failure. And surviving production means building in layers, not all at once.

For the engineering agent, that singular focus might be: given an active alert and the last 10 Slack messages in the incident channel, extract the corresponding markdown files from your repo. That’s it. No automated remediation, no Jira ticket creation, and no multi-step reasoning. Just that one task, executed consistently.

It also helps to align on what your MVP is actually trying to prove. The four goals I come back to are:

1.Validate real user value
Prove that the AI actually solves a meaningful problem better, faster, or cheaper than existing alternatives.

2. Test core assumptions

Validate assumptions about prompts, model choice, latency, cost per request, user behavior, and output quality before scaling.

3. Measure unit economics

Understand cost per user, cost per request, token usage patterns, and whether the solution is financially viable at scale.

4. De-risk production scale

Expose early issues around rate limits, retries, hallucinations, reliability, and external API dependency before investing in production-grade infrastructure.

Scoping:

Keep It Simple

My most important takeaway from dozens of use cases I worked with is keep it simple, add complexity.

A plain LLM call, give it context and get a response, will handle the majority of use cases better than a half-baked agent. Summarization, Q&A, classification, drafting responses, extracting structured data from a document. If your workflow is essentially "user gives input, AI gives output," that's probably all you need for v1.

A plain LLM call — give it context, get a response — handles the majority of use cases better than a half-baked agent. Summarization, Q&A, classification, drafting responses, extracting structured data from a document. If your workflow is essentially "user gives input, AI gives output," that's probably all you need for v1.

Think of it as a ladder you climb one rung at a time. Start with a single LLM call and see how far it gets you. If the model needs better or more current information, add retrieval (RAG). If the task requires pulling from external systems at query time, add tools. Only if the task genuinely requires multi-step autonomous reasoning do you need a real agent. Most products never get there — in my experience, one agent equipped with great skills can handle most of the job.

Teams that add RAG before establishing evaluations can't tell if retrieval is helping or hurting. Teams that jump straight to agents before validating with a simple prompt spend months building scaffolding around a broken core idea. Each step up the ladder exists because the step below it hit a real ceiling — not because the next step sounds more impressive.

For the engineering agent, I would start by connecting the required data sources to the model: Slack, Jira, GitHub repos. Keep it simple with MCP tools first. No agentic loop. Just a prompt and a response. Ship that, see if engineers actually find it useful, and let real feedback tell you what to build next.

Bring Your Organization's Knowledge to Make a Difference

Your AI product is only as good as the knowledge you feed it.

Before you think about architecture, the first question to ask is: what knowledge do we already have, and where does it live?
Poor data quality ranks among the top causes of AI project failure — not because organizations lack data, but because it's fragmented.

For the engineering agent example, in most R&D teams it's scattered across Confluence or Notion pages, Markdown files in repos, Slack threads nobody can find, Jira ticket descriptions, and runbooks that may or may not be up to date.

Your MVP doesn't need to connect to all of it but it does need to connect to the right slice of it.

The simplest method I suggest for MVP is to include the relevant documents directly in your prompt as context, without using a vector database, embedding pipeline, or retrieval system.

Select the 5-10 documents most important for your use case, break them into manageable sections, and provide them to the model. This approach often produces good results and allows you to determine if the knowledge is actually beneficial.

After that, you can integrate with your organization’s tools. Start with MCP servers as an easy plug-in, though keep in mind this may not provide the most comprehensive context if your knowledge base spans multiple systems.

Once you see that grounding your LLM in internal documents improves output, that's when it makes sense to consider a full RAG setup. For now, identify the essential knowledge, format it for the model, and run your tests.

Starting easy to advanced

Define what "done" looks like for your MVP

MVP means for the minimum that actually validates something.

For a generative AI product, that validation question is almost always one of these: can the AI reliably do the task? Will users trust and act on the output?

Pick the one that matters most and make it your primary success criterion.

Then set a few concrete, measurable goals around it.

For the engineering agent example, that might be 80% of common questions answered correctly without human review, responses under 5 seconds, and customer satisfaction that doesn't drop below baseline.

Notice these aren't "the model answers questions." They're specific enough that you'll actually know whether you hit them or not.

Moving on

What does success look like with actual numbers? What's the single most important thing your MVP needs to do? If you've got clear answers to all of those, you're in great shape to start building on solid ground.

In the next post, we'll get into setting up your GitHub repo the right way for an AI project, including folder structure, branching strategies, and a few things that are specific to working with models and prompts in source control.

Putting it all together

What did I miss? Drop a comment or reach out. I'm building this series based on real experiences and always happy to go deeper on the parts that matter most to you.

## References

[1] MIT NANDA, "The GenAI Divide: State of AI in Business 2025," August 2025. https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf.
Fortune, August 18, 2025. https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/

[2] S&P Global Market Intelligence, "Why 42% of AI Projects Show 0 ROI," 2025. https://beam.ai/agentic-insights/why-42-of-ai-projects-show-zero-roi-(and-how-to-be-in-the-58-)