Sketches from the Field

Gathering data by hand.

Picking your dataset is the most important part of a project like this — good data creates good results. It's also easy to pick something with too many visual differences between examples, which will quickly turn a small project like this into a certified dummy mission™. I chose to keep it simple: 300 leaves from one shrub type, taught to a small pix2pix model.

I'm a hobby ecologist, so I'm naturally drawn to the small. For this project, I chose to investigate the vein networks in a leaf — they're vital to life on earth and almost entirely unseen, similar to an ML model's own inputs. I chose the creeping fig (Ficus pumila) as my subject; a little vine that eats fences. It's invasive here in California, so harvesting hundreds of leaves to experiment and troubleshoot felt less harmful to the ecosystem.

Even the same leaf can have so much variance: some are fat and heart-shaped, some long and skinny, some red instead of dark green. I flattened them, laid them on the scanner glass a handful at a time, and scanned every single one before pressing them in a book. This took 10+ hours across many days. I came to appreciate what it takes to gather ML data.

While working away, I reflected on the same lesson generative AI keeps teaching me: we obsess over the outputs while forgetting the inputs — the unglamorous structure it takes to make something beautiful, whether it's a leaf's vein network or a generated image. When we forget the inputs, we forget the intelligence we started with in the first place. In this case, I was examining nature's intelligence — its ability to keep generating itself, over and over, in the perfect way, for a greater good.

I think there's something important to learn from the things we deem small, uninteresting.

Preparing the data, training the model.

Gathering and scanning is only half of it — after that you still have to clean, crop, and grow the data. I augmented my 300 leaves into ~1,200 by flipping them horizontally and vertically, set up a Google Drive to hold it all, and got the training pipeline running.

Then came the decisions that actually shape the model. Epochs: one full pass over all the data is one epoch — I set a max of 200, the established standard for pix2pix. Learning rate: how big a step the model takes each time it corrects itself. Too high and it overshoots and crashes; too low and it's making painfully slow progress (akin to watching a plant sprout for hundreds of hours straight). I used 0.0001 — cautious, maybe overcautious. Save frequency (100): saves the model itself — checkpoints, trained weights you can load and use later. Display frequency (500): saves preview images every 500 steps — the input, the guess, the target — so you can literally watch it learning (this was my favorite part!). I did undertrain it a bit in the end. You live, you learn, you train again.

And then you wait. The logs tick past — discriminator loss, generator GAN loss, L1 loss — three numbers flashing across the screen for hours. Checkpoint by checkpoint, a blurry leaf starts to appear and take form.

checkpoint 99 checkpoint 1,900 checkpoint 43,200

It kind of looks like shit. Here's what I see anyway.

Ok, let's look at this thing. Smeary, dithered, a little haunted looking (why is AI so uncanny?!) — but look closer: it learned green in the middle, pale at the vein, darker at the edge. It started trying to figure out veins, chaotically. A blank network now has opinions.

And I came away understanding what it actually takes to build one — the cost of a dataset, the demands of a model, and the complexity of even the smallest things. I'm excited to keep iterating.

input real leaf it guessed

Can a machine learn a vein, then dream a new one?

That was the real question. Veins are unseen, we don't think about them, but they hold up the whole leaf, and the leaf holds up the ecosystem, and the air we breathe. It felt like a fitting thing to ask a model to learn: the intimate, structural, easy-to-overlook parts. It reminds me of the overlooked dataset driving the models we love to use.

Watching it train, checkpoint by checkpoint, a hundred images at a time, I understood ML differently. The output isn't the point. The inputs are — the data, the pipeline, the thousands of small decisions. The labor is the real star here; whether you're building these datasets, the models that use them, or maintaining them, they're made by many, many hands.

What artists have been saying for so long is true: the purpose is the process, and the labor is the art.

So maybe it hasn't quite imagined new veins yet — or answered the biggest questions of the plant universe — but I do have ideas for the next pass. What's another epoch anyway?