building my first AI feature in my app | coding vlog ep. 5

Quick Overview

The creator successfully built and deployed an AI feature—a custom code or cursor generator for marketers—into their app, Yorby.ai, over five days, utilizing a double-blind testing process involving multiple LLMs (including Gemini 2.5 Pro and Flash) and a custom internal evaluation tool to iterate and select the best-performing model, ultimately demonstrating the power of specialized AI tooling over general-purpose models for specific tasks.

Key Points: The creator spent five days building and deploying their first AI feature, a code/cursor generator for marketers, into their app, Yorby.ai. The core development involved an iterative process comparing multiple LLMs, including Gemini 2.5 Pro and Gemini 2.5 Flash, using an internal evaluation tool. The creator used a double-blind testing methodology to grade different models on a 1-10 scale based on their ability to generate high-quality, viral-potential content scripts tailored to a user persona. The initial attempt using complex AI workflows and multiple models resulted in poor output, leading the creator to simplify the process to a single, well-crafted prompt. The final, simpler approach yielded better results, with the best model scoring 9.8/10, proving that high-quality input is crucial for good AI output. The video also briefly mentions two core features of Yorby.ai: a Hiring Feature (pay-for-success model) and a Viral Content Database.

Context: This video is a startup vlog (Episode 5) documenting the creator's intense, 24-hour development sprint to build and test the first AI feature for their social media marketing tool, Yorby.ai. The primary goal was to create a tool that could remix existing viral videos into new content tailored for specific business needs, moving beyond simple prompt engineering to incorporate structured evaluation and iteration loops using various LLMs.

Detailed Analysis

The creator documents the five-day process of building the first AI feature for their app, Yorby.ai: a tool that generates custom code or cursors for marketers based on existing viral video scripts. The initial approach involved complex workflows and testing multiple LLMs (like Gemini 2.5 Flash and Pro) side-by-side using an internal evaluation tool, which ultimately led to poor results due to the complexity. A key learning was that a simpler approach—using a single, highly specific prompt based on the original video script and a defined user persona—yielded much better results, with one script scoring 9.8/10. The creator details the iterative process of generating multiple scripts, grading them using an internal scoring mechanism (0-10 scale), and refining the prompts based on the results, concluding that the AI tooling ecosystem is powerful but requires careful input design. The video also briefly covers the existing features of Yorby.ai: a Hiring Feature and a Viral Content Database.