TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
An engineer has published a complete, end-to-end open reproduction of Surya Narreddi’s viral watercolour-painting language model, using TRL and OpenEnv on Hugging Face infrastructure. The pipeline trains a Qwen model with reinforcement learning against an aesthetic reward mix, with all datasets, environments, scripts and trained models released openly.
An independent engineer has published a fully open reproduction of the watercolour-painting language model that went viral in August, releasing the RL environment, a hand-rated reference dataset, training scripts and trained models under an open license. The reproduction, built with TRL and OpenEnv and running end to end on Hugging Face infrastructure, follows the blog post of Surya Narreddi, whose 23 August video of model-painted watercolours drew over 1.5 million views. The project tests a specific question: whether reinforcement learning can optimise a model against aesthetic taste rather than verifiable answers.
The original project, created by Surya Narreddi from the art and design side, trains a language model to write roughly 150 lines of JavaScript that paint an image using p5.brush, a library the author describes as one that “adds natural drawing tools to p5.js”. The model’s output is code, meaning it can be read, edited and re-run, with the decision behind each brushstroke visible. Style is enforced partly through a restriction: the model is only allowed ten of the library’s methods.
The reproduction implements the reward defined in Narreddi’s blog as a weighted mix of four terms: a gate (0.05) that checks the sketch compiles, paints something and does not cheat; a length term (0.05) softly pushing towards longer code; a pairwise judge (0.60) scoring style against four references randomly drawn from a hand-curated pool; and HPSv3 (0.30), an open 7B preference model trained on human choices between image pairs. The pairwise judge is Qwen3-VL-30B-A3B-Instruct, a general vision model called through HF Inference Providers, which sees the candidate next to four references and is guided by a written description of what to weigh — bleeds, translucent washes, soft edges — with each comparison run in both presentation orders.
Once the two Spaces (the RL environment and the scorer model) are running, the training recipe is a single command launched via HF Jobs against Qwen/Qwen3.5-35B-A3B with LoRA, 110 steps, 240 episodes per step and 8 generations per episode. The reproduction’s new contributions, according to the author, are the open implementation, the hand-rated reference pool, and three reward mixes trained and compared in parallel. All artifacts are gathered in a single Hugging Face collection.
Why RL Over Taste Matters
Most recent reinforcement learning work on language models relies on verifiable rewards — math problems with known answers, or code that passes tests. This project sits closer to the older RLHF tradition, where models learn from human preference, but with a twist: the preference is aesthetic, with no correct answer to optimise against. The author frames the core question of the project as “whether you can do RL over taste.”
The timing also matters. The paintings look loose, imperfect and handmade at a moment when mainstream image models produce what the author calls perfect, statistically average pictures. The author’s own guess is that this contrast drove much of the video’s virality. Because the model writes readable code rather than pixels, the work also offers something image generators do not: inspectable, editable output where every brushstroke decision can be traced.
The open release removes the main barrier to follow-up work. Narreddi’s blog covered an earlier, narrower stage of the project — close-up flowers rather than the full compositions in the viral video — and did not include open artifacts. The reproduction closes that gap for the community.
Products Worth Considering
Bose QuietComfort True Wireless Earbuds with Active Noise Cancellation, Black
Samsung 32 Odyssey G55C QHD 1000R Curved Gaming Monitor
DJI Osmo Pocket 3 Creator Combo 4K Vlogging Camera
As an affiliate, we earn on qualifying purchases.
From DeepDream to Curated Datasets
The author places the project in the lineage of early generative AI art, when practitioners explored the medium itself: DeepDream (2015), a debugging tool turned into art; Edmond de Belamy (2018), produced by artists probing what a GAN could do; and Mario Klingemann’s neural-network portraits. In the same period, artist Anna Ridler photographed and hand-labelled thousands of tulips, exhibited the dataset as an artwork, and later trained a model on it — an approach the author says this project mirrors by hand-curating a reference pool and training against it.
According to Narreddi’s thesis, described in the source material, he started by prompting text-to-image models, where the prompt is the only lever and more detail buys control only up to a point. Training the model itself goes further. His blog post explains the framing and reward design behind the earlier flower stage; the full technical report, which his site says is coming, does not yet include open artifacts.
“adds natural drawing tools to p5.js”
— p5.brush project description, cited in the reproduction
Unverified Results and Missing Reports
Several things remain unclear. The reproduction’s author explicitly labels the virality explanation as a guess, not a finding. The comparison between the three reward mixes is presented visually — each run shown as the median painting per step — but the article does not report a final quantitative verdict on which mix works best in this source material. The original blog covers an earlier, narrower stage of Narreddi’s project (close-up flowers), not the full compositions shown in the viral video, and no open artifacts exist yet for Narreddi’s own runs. A full technical report from Narreddi is promised but has not been published. It is also not stated how closely the reproduction’s outputs match the originals in quality.
Follow-Ups and the Technical Report
Two things to watch: Narreddi’s full technical report, which his site says is coming, and the reproduction’s “What I would try next” list, where the author collected every idea of his own rather than deviating from the original recipe. Because the entire pipeline is open — dataset, environment, scripts, trained models — other practitioners can duplicate the two Spaces, set the reward-mix environment variables and launch their own runs with a single command, suggesting community variants on subjects, reward weights and library restrictions are likely to follow.
Where I land
I think this reproduction is more interesting than its viral surface suggests. The eye-catching part is the aesthetic output, but the durable contribution is methodological: it shows a complete, reproducible pipeline for reinforcement learning against non-verifiable, taste-based rewards, which is a genuinely underexplored corner compared to math- and code-RL. The open release of the environment, reference pool and three compared reward mixes is what turns a demo into a research artifact others can build on.
The strongest counterargument is that the reward is only as good as its proxies. HPSv3 averages many people’s taste, and the pairwise judge is a general vision model guided by a written rubric — both could steer the model toward a statistical notion of “watercolour-ness” rather than genuine artistic quality, and the virality of the output is not evidence the reward is right. The fact that the source material reports no clear quantitative winner among the three reward mixes supports some caution.
What would change my assessment: a head-to-head evaluation against Narreddi’s original outputs, blind human preference ratings comparing the three reward mixes, and the promised full technical report. If the report shows the open recipe matches the originals at reasonable compute cost, this becomes a template for open aesthetic-RL work; if it doesn’t, it remains an elegant but unvalidated experiment.
Key Questions
What exactly does the trained model produce?
The model writes roughly 150 lines of JavaScript using p5.brush, a library that adds natural drawing tools to p5.js. That code, when run, paints a watercolour-style image. Because the output is code, it can be read, edited and re-run.
How is the model rewarded during training?
Through a weighted mix: a 0.05-weighted gate that the sketch compiles and does not cheat, a 0.05-weighted length term, a 0.60-weighted pairwise style judge comparing the painting against four references from a hand-curated pool, and 0.30 weight on HPSv3, an open 7B human-preference model.
Is this the same model as in the viral video?
No. The viral video showed Surya Narreddi’s project. The reproduction follows his blog, which covered an earlier, narrower stage of that project (close-up flowers), and re-implements it openly with TRL and OpenEnv. The full compositions in the video are not covered by the published recipe.
Is everything open?
The reproduction is fully open: reference pool dataset, RL environment, training scripts and trained models, all hosted on Hugging Face. Narreddi’s original project has no open artifacts yet, though a full technical report is promised.
Can others reproduce the training?
Yes, according to the author. Once the environment and scorer Spaces are duplicated, the recipe is one command via HF Jobs, with configurable reward mix, subject, and hyperparameters, and a 48-hour timeout on an H200 flavor.
Source: Hugging Face
