The race is on among leading AI labs: everyone wants to build self-improving AI models. The theory goes that as AI continually refines itself in an exponential feedback loop, it will eventually achieve superintelligence, surpassing human understanding and possibly even control. It’s a fascinating, almost futuristic concept, but I had a more immediate, practical question: could this cutting-edge technology help me with my newsletter?
I wondered if recursive self-improvement could be harnessed to automate some of the tedious tasks involved in publishing. My goal was to train and continually enhance an AI model to handle some of my newsletter’s busywork. After about a week of intensive experimentation, the answer became surprisingly clear: a resounding “yes.”
What’s more, this hands-on experience with self-improving models unveiled a compelling alternative vision for AI’s future. It suggests a path that doesn’t solely rely on a handful of powerful companies dominating the entire industry. Instead, it hints at a more distributed, accessible future for advanced AI capabilities.
The Quest for Self-Improving AI (and My Newsletter)
To begin my exploration, I decided to try training a small language model from the ground up, though “I” actually meant offloading the heavy lifting onto Claude. I leveraged a powerful tool called AutoResearch, designed to help an existing, off-the-shelf AI model build and refine a smaller one. This innovative software is the brainchild of Andrej Karpathy, a renowned AI researcher who played key roles at OpenAI and Tesla, and now contributes to Anthropic.
My setup was fairly straightforward: I fired up Claude with the recommended instruction, “Hi, have a look at program.md and let’s kick off a new experiment!” While Claude meticulously handled the complex coding and model adjustments, I provided the necessary infrastructure. This included an Nvidia DGX, a desktop “supercomputer” built specifically for AI development, and the continuous electricity to run it for several days straight. I also made the possibly ill-advised decision to let the model operate with minimal permission checks – essentially, giving it free rein to “cook.”
Monitoring the AutoResearch project every few hours was genuinely mesmerizing. I watched as Claude autonomously tweaked parameters, adjusted training regimes, and then analyzed how these changes impacted the smaller model’s output. This iterative process allowed it to refine the model further, constantly pushing towards improvement.
My First Foray: Training a Mini Language Model
The initial results from my nascent language model were, to put it mildly, not brilliant. When prompted to complete the phrase “In the beginning…”, an early version produced a nonsensical, repetitive string: “In the beginning of the beginning of the end of the end of the end end of end end end end end end end beginning end end end end…” It was far from intelligent and prone to infinite loops.
However, subsequent iterations, autonomously improved by Claude, demonstrated a clear progression. The models gradually became more coherent and less prone to those bizarre, endless repetitions. While it was certainly no match for advanced models like GPT-5, this experiment vividly showcased a promising pathway toward continual, autonomous improvement, even for relatively small-scale projects.
Automating Content Curation with Custom AI
Having seen the potential, I decided to tackle a more specific newsletter task. I already use an AI agent powered by Claude to help me discover noteworthy research papers, but I wanted to go beyond that. My goal was to create a tool that could not only find papers but also summarize them effectively for my “Elsewhere on the frontier of AI” section.
For this, I turned to a platform from a startup called Prime Intellect, which specializes in using AI to train custom models for precise tasks. I gathered approximately 100 past entries from my newsletter, which typically consist of concise summaries of interesting research. With Claude’s assistance, I then established a Prime Intellect training environment to build my own model, which we aptly named Frontier_Paper_Curator.
Claude played a pivotal role, finding additional relevant papers and generating a wealth of synthetic data to bolster the training process. Another specialized AI model was brought in to meticulously assess Frontier_Paper_Curator’s output, while the training environment continuously improved the model using reinforcement learning. It was a sophisticated, multi-layered approach to creating a highly specialized AI.
Vincent Weisser, CEO of Prime Intellect, which recently secured $15 million in funding, shared his company’s vision: to make recursive self-improvement accessible to everyone. He argues that while models from frontier labs are undoubtedly brilliant, democratizing this kind of AI training can yield equally capable, highly specialized models tailored to specific needs. “Give every company access to frontier training infrastructure, and the collective creativity of the market unlocks far more than any handful of labs can,” Weisser explains. “We don’t want one centralized, almost godlike intelligence, we want a billion intelligences that go into all the niches that create beautiful things.”
Prime Intellect isn’t alone in this decentralized vision. Adaption, another startup, offers its AutoScientist tool to automate AI model training, with CEO Sara Hooker noting its value to large companies lacking in-house AI expertise. This movement highlights a critical shift, especially given the risks of relying too heavily on single frontier models, as demonstrated when Anthropic blocked certain requests to its Fable 5 model. Executives like Palantir’s Alex Karp have also warned against the potential loss of data control and technological autonomy when solely depending on external frontier labs.
The Promise of Specialized, Autonomous AI
The ultimate aspiration for recursive self-improvement is for AI to autonomously generate novel ideas and insights to enhance itself. While the tools available to us beyond the frontier labs are currently more constrained, they are still remarkably powerful. In less than a day of working with Prime Intellect, I was able to develop a surprisingly effective model for discovering and summarizing research papers.
Here’s a compelling example of an entry created by my custom AI model: “Researchers at iFLYTEK have developed iFLYTEK-Embodied-Omni, a unified multimodal AI model that integrates vision, language, and action generation into a single framework. Unlike prior embodied agents which treat visual understanding, future state prediction, and action generation separately, their model uses shared multimodal self-attention to enable close coordination—analogous to a brain-cerebellum collaboration—between a vision-language ‘high-level brain’ and an action-generating ‘low-level cerebellum.’ This approach reduces error compounding and interface bottlenecks common in cascaded pipelines. By training on a large diverse dataset including human and robot-annotated embodied videos and image-text data, and using a staged training strategy, they demonstrate a general-purpose embodied agent capable of joint reasoning, prediction, and control. This contributes a novel architectural and training paradigm toward more integrated, versatile robotic AI systems.”
This is an impressive outcome for a first attempt, offering insightful and well-structured summaries. While the new model is still a bit overzealous, sometimes selecting too many papers I might otherwise skip, and its summaries can occasionally be a touch generic, it represents an incredibly promising start. I remain hopeful that, with further refinement, this self-improving AI will one day significantly reduce my busywork, freeing me to focus on deeper insights and creative endeavors.
Source: Wired – AI
