EVERYTHING
According to I-programmer, a research paper authored by Dave Rensin, a Distinguished Engineer at Google, outlines a sophisticated approach to using large language models (LLMs) for software architecture. The methodology aims to move beyond simple code generation toward creating rigorous, verifiable design documentation that serves as a foundation for development.
Training the Elephant: Contextualizing Knowledge
The first phase of Rensin's framework involves 'feeding the elephant.' In this analogy, the elephant represents an AI session saturated with specific organizational knowledge. Developers are instructed to point the tool at relevant source trees or design documents and ask it to generate high-level descriptions. If the model fails to grasp the system correctly, the developer must provide corrections immediately.
Rensin emphasizes that developers should explicitly instruct the AI not to write code initially. Instead, the goal is to have the tool challenge assumptions and propose implementations in prose and block diagrams. Key steps in this process include:
Because current models have output token limits, Rensin advises building these documents in small, iterative chunks within a single chat session to maintain consistency.
Testing with the Goldfish: Verifying Clarity
Once a design document is drafted, it must be validated using a 'goldfish'—a completely fresh AI session with zero prior knowledge of the project. If the goldfish cannot explain the system based solely on the provided markdown file and its references, the documentation is deemed insufficient. The developer then adds missing context and repeats the test until the model can successfully navigate the logic.
Rensin notes that while 30% of an AI's suggestions during this phase are highly valuable, the primary goal is to eliminate ambiguities. Once the goldfish stops asking critical questions and only offers minor nitpicks, the document is ready for human review. For massive codebases, Rensin suggests a bottom-up approach where developers create small, high-quality readme.md files for specific directories, eventually aggregating these into a full system overview.
This methodology shifts the role of AI from a mere autocomplete tool to a rigorous peer reviewer that ensures architectural integrity before a single line of production code is written.