Three Weekend LLM Prototypes, and What They Taught Me

By
128 views
Three quick LLM prototypes from 2023, what each one taught me about prompting, latency and context, and why none of them are online any more.

In late 2023, right after I launched jakejh.com and jjhdigital.com, I wanted to know how language models actually behaved inside an application, not how they read in a benchmark table. So instead of spending months on one polished product, I built three small ones, roughly a weekend each: thin wrappers around GPT-4, Llama 3 and Groq's inference API.


None of them were impressive products. The code was mostly plumbing between a text box and an API call, and none of the three is online any more. What they gave me was the first honest look at the questions I still work on: how a system prompt shapes behavior, how latency changes the way people use a tool, and what happens when the context runs out.

Project Chat Project: The AI Salesman

Technologies: React, Next.js, TypeScript, GPT-4, Llama 3, Tailwind CSS

A conversational agent tuned to represent JJH DIGITAL. The interesting part was never the chat interface, it was the prompt: whether a long, structured system prompt could make a general model behave like a specific company, with its own voice, services and limits.

The system prompt had to do three jobs at once:

  1. Provide comprehensive information about JJH Digital's services
  2. Set a conversational tone that matched the brand
  3. Fall back gracefully on questions outside its knowledge domain

It worked about as well as a well-prompted general model can, which is to say surprisingly well on the questions it had been primed for and confidently wrong on everything else. That failure mode is why fallback handling became a habit I still keep.

Key takeaway

A well-written system prompt is worth more than clever code. Nearly all of the value was in the prompt and almost none of it was in the implementation, and that is still roughly true.

Project Prompt: Lightning-Fast Responses

Technologies: TypeScript, React, Next.js, Groq inference API

The second experiment was about speed. Groq's inference API was the story of that month, so I built the smallest possible chatbot around it and tried to shrink the gap between a query and its response.

The implementation featured:

  • A customizable system prompt that users could modify
  • Integration with Groq's inference API
  • A clean, minimalist interface that emphasized conversation flow

The difference was obvious within a minute of use. Below about a second, conversation stops feeling like submitting a form and starts feeling like talking. None of the content had changed, only the latency, and yet it was a different product.

Key takeaway

Response speed changes how people perceive an AI interaction. Same model, same answers, faster reply, and the thing feels natural instead of mechanical.

Project Planner Project: AI-Driven Itinerary Creation

Technologies: TypeScript, React, Next.js, Llama 3 70B via Groq

The most complex of the three, and the one that mattered most, because it was not a toy: AdventureGenie had shipped a feature called Genie Wishes that generated trip ideas, and I wanted to know how much of it I could rebuild in one night.

Using Llama 3 70B through Groq's platform, I implemented:

  • Multi-threaded conversation capabilities for contextual understanding
  • Natural language interfaces for itinerary adjustments
  • Consideration of factors like seasonality and travel times in recommendations

You describe a trip in plain language, it asks a couple of questions back, and you refine an itinerary by arguing with it. The multi-turn part was the hard part, not the generation.

A feature that had taken weeks at a startup existed in rough form by morning. That is the shift the API era actually delivered, and it is why I stopped treating LLM features as projects of their own and started treating them as primitives inside larger ones.

Key takeaway

Modern LLM APIs compress development timelines for anything that is mostly language understanding and generation. That is the whole trick, and it is a big one.

The Value of Rapid AI Prototyping

Three years later the wrappers themselves are trivia. Anyone can call a model API in an afternoon. What has not gotten easier is everything around the call:

  1. Experimentation over perfection: quick implementation instead of a polished product meant three approaches in the time one build would have taken.
  2. Learning through doing: each project taught me a different piece, from prompt engineering to rate limits to behavior at the edge of the context window.
  3. Demonstrating possibilities: a rough demo explained the technology to clients faster than any proposal did.
  4. Comparative analysis: the same idea against different models (GPT-4, Llama 3, Groq) gave me practical intuition about their strengths, and most of that intuition still holds.

The prototypes were never the point. Building three bad ones in a month taught me more than a year of reading about the technology would have, and the questions they raised are the ones I get paid to answer now.

If you want to see where it led, JSPARK3 is the same curiosity with a bigger budget: what does it actually take to run a model on your own hardware instead of renting a wrapper around someone else's.