Sketches from the Field

What's different this time.

I came back to this project with new learnings in hand, ready to train this model all over again from scratch. In this second round, three things changed:

1. The dataset got veins, using a vein-tracing pipeline I vibecoded
2. The sketch tool got a rebuild — I created new custom brushes for the drawing tool
3. The model trained longer — I ran more epochs to help the model learn a little more

The dataset got veins.

Instead of feeding the model bare outlines of a leaf and asking it to guess the entire inside of the leaf (which proved near impossible), dataset A now includes the actual vein network traced out of each scan — the midrib and a few side veins. The goal: give the model enough vein structure to learn from, so that when it generates a leaf from a new sketch, it can invent the rest of the network on its own — not hallucinating out of nothing, but generalizing from a few hundred real leaves into something new.

I came to this with a question: if I give the model the same rough leaf outline any adult or child would draw — a silhouette, a couple of veins, nothing more — can it imagine the rest of the structure convincingly? Can it learn the logic of a vein well enough to invent one that still feels real?

Tracing veins by hand across 300+ leaves was a non-starter, and mapping them with computer vision wasn't much easier — veins branch, they vary wildly in size, and no two leaves are consistent. So I built a computer vision pipeline that finds the vein network on its own: a Sato ridge filter (the same kind of filter used to find blood vessels in medical scans) picks out the thin structures, gets thresholded into a mask, and skeletonized down to one-pixel lines — counting 25–50 fragments, then pruning down to the ones that mattered, color-coded live as it worked: green kept, red discarded, blue the midrib.

It worked, but it would've taken 36 hours to run across the whole dataset, so I scrapped it and rebuilt the pipeline around a better question: not “find every vein,” but “find the veins a person would actually draw” — the midrib and a handful of clean side branches. That version turned the skeleton into an actual graph, walked it from base to tip to find the midrib, then outward to find side veins, and dropped anything under 15% of the midrib's length as the fine capillary detail no one sketches from memory. Runtime dropped to about four hours.

Every threshold in it was still a number I picked by eye — the graph theory didn't make it more objective, just fast enough to be worth doing.

Foraging Data by Hand.

Every leaf behind this project was picked by hand, one at a time, from a single invasive vine — creeping fig (Ficus pumila) — eating a fence near where I live. I chose it partly because its abundance and invasiveness made it an easy choice vs experimenting on native plants.

Sure, it's different, but not uncomplicated. As a hobby ecologist, the questions I face in my own everyday harvesting practice were bubbling up during my data collection process too — even though I wasn't foraging fruit at all. I was data foraging.

• What data do we ignore in our everyday lives?
• What data exchange am I making?
• What data am I extracting?
• Why does abundance or availability grant permission?

The logic that abundance gave me permission to take a lot of leaves started to trouble me. These are the same logics used to scrape the web for models. People say, “well it's there, it's available in the public sphere — why can't I use it?” But abundance isn't the same thing as permission.

Robin Wall Kimmerer's writing on the Honorable Harvest came up during the share-out of blog one (a book very dear to my practice). It's the idea that taking from a living thing comes with obligations: ask first, take only what you need, use everything you take, give something back. My own family has fished, farmed, and foraged for generations, and there's a specific kind of obligation that suddenly appears when you gather something yourself, by hand, rather than having it arrive already gathered and packaged for you. You can't help but notice what you're taking. Which means you can't help but ask what you owe it.

I don't think “data” is usually allowed to be a word that carries responsibility. We call it collection, we call it web scraping, or click workers — and either way we mostly look at what comes out the other end — the model's output — without much thought for what was asked of the input to make it possible. Dataset A did a lot of quiet, unglamorous work in this project, but I hope that while using the tool, you remember that it once was a real leaf.

So on my quest of harvesting this invasive plant, I decided that I wasn't only there to scrape or take data, but to be in relationship with it — to properly learn from it. By doing this, I was able to really grasp my hands around my own data, understand it closely, and understand how ML models are built from the ground up.

In this case, I realized the environment would lose a little ecological data and hopefully, from me, gain another kind. Maybe a kind of knowledge. I wondered if I could bring that knowledge with me, make with it, make it useful.

So I saved and pressed the 300 leaves I gathered for future preservation of some kind — which may be announced in a part three — and I'm creating 300 short reflections for every leaf taken, so that none of them passed through my hands unconsidered. That's still mostly ahead of me. Maybe that's what a third version of this is for.

Can a machine learn a vein, then dream a new one?

Well yeah, it can, and my first thought when I saw a real-looking, generated leaf was “wow, that's kind of f*cked up.” Because it actually did learn something about vein networks, and it actually kind of looks like a leaf. I find this so interesting, and maybe a little weird and uncanny. But what once felt like a pipe dream in phase one of my project, actually happened this time. It learned enough of what a vein is — where it sits, how it branches, what texture surrounds it — to generate ones it has never seen, following a hand-drawn suggestion that anyone, even a non-artist, can create.

→ → → →

Like many handmade models, it does fall short in some ways. Some of it is the limitation of the Pix2Pix technology itself — it can never be as strong as a commercial model. Also, the dataset is small in general, but especially small for certain leaf shapes. So when you draw a really fat leaf with just a few veins, you may see empty regions in the generated leaf. Honestly, it's pretty ghastly looking. But as someone noted: those gaps in the leaf look exactly like a leaf you'd find on the ground in the fall — a bit warped and weathered, and I like that it has some “realness” to it. We love happy mistakes. So I'm learning to enjoy the little failures — instead of continuing to fine-tune the perfect model, I think I'm really happy here.

But if you do find yourself wanting to see a fuller leaf, you can always just add a vein in the blank spot of the leaf in your drawing, and re-generate — and voilà! Your leaf is filled in. All in all, I knew there would be limitations, and I'm happy with what I've learned. You can see the massive progress yourself by comparing the output to blog one.

Sketches of Life

I came into this project attempting to replicate something as squishy and hard-to-pin as “life”, and I'm now realizing how many more questions than answers a project like this brings up. We talk about machine learning as mimicry all the time — digital twins, models “trained on us,” always implying a slightly better, smarter, faster, or more efficient version of the original.

But that's not exactly what happened here. This model is, at best, replicating the part of a leaf's vein network that humans can see. There's bias in my own perspective: I can't see inside of the vein network, I can't easily account for all of the ecological inputs, or travel down into the root systems that entangle this leaf with the rest of the ecosystem. But I think that's okay — I don't have to. Even in appreciation for the natural world, this is my very human perspective. My own human intelligence.

Maybe before we ask a machine to imagine something new and daring, we first have to consider the limitations or biases of what we see.

Connected Intelligences: Human, Machine, Nature

I'm becoming really interested in the idea of connected intelligences: human, machine, and nature. Where I'm limited, I can lean on another kind of intelligence to fill in the rest as an inspiration point — similar to how we as humans have practiced biomimicry for eons, learning from and modeling after nature. I think instead of pretending these limitations don't exist, we can recognize and sit with these gaps, and then they become the connective tissue to other kinds of intelligences. The collaborative model I'm suggesting here also takes the pressure off of one form of intelligence having all of the answers (that may very well be a dummy mission, as I suspect). Instead, it invites a more curious, active participation of human, nature, and machine — instead of a one-way collaboration or impact.

Machine intelligence: its ability to synthesize our incredible human intelligence and bring together connections faster than we could.

Human intelligence: our ability to feel and reflect, our intuition, our connection to something greater, and the challenges and opportunities we face as living beings.

Ecological intelligence: the constant stretching toward life, always trying to find balance, connectivity, and build networks and collaboration.

I wonder what's possible when we pause for a second and consider nature's intelligence as part of our investigations? What can we create? What could we learn?

I'm excited to continue thinking about how humans, machines, and nature can be in deeper conversation as we shape the future. Not as superiors with conflicts of interest, but as equal partners.

Power in the Small

Before I go, I want to underscore that this blog isn't all about the limitations of being a human. If anything, as I've flipped through these generated leaves that I hand harvested, I've been reminded of the beauty of being alive in the first place. How special it really is. The unique place we also hold in the making and saving of knowledge.

My mom, whose garden I've been spending a lot of time in lately, told me to watch what a plant does — how it responds when it's watered, how a vine's tendrils go looking for a fence or a pole to wrap around. That's how you know it's alive, she told me, by the way it reaches. I grew up in a pretty spiritual home, but this project is a personal reminder that life is so beautiful, ephemeral, and maybe a little more mysterious than I even recognize at times.

These leaves, for me, are a small meditation on the miracle of humanity. By looking at a leaf's generated vein in a meditative way, I look at the building of life, and think of my own.

If these leaves being successfully generated feels like a little miracle of my own harvesting and effort, then so is all life, really. We are made up of this effort, knowledge, and connectivity.

It's a reminder that a lot is packed into the small — that the small should never be underestimated, and that even a single human — or a leaf — on a large earth carries power and potential. I'm reminded to live fully and to stretch toward life.

My partner reminds me that machine learning, at its most honest, brings us back to mystery rather than trying to resolve it. I agree. I'm also finding that the useful move isn't ranking intelligences — human above machine, machine over human, or even nature over human — but accepting that there are simply different kinds of intelligences to learn from. They can give us perspective. These knowledges, together, create a new kind of opportunity, a new potential way to live and make new intelligences, maybe?

Natural Stopping Points, and the Art of the Epoch

With an intention to set aside the output pretty early on, I came into this investigation genuinely unsure where it would take me: if I'd find any semblance of success, what I'd learn, and whether it would even be surprising or interesting to me. I'm happy to say it was all of the above!

I guess a handful of epochs are like life: we look closely at the same thing, over and over again, wondering what to take away from it, until finally the reasoning comes into view. I myself have sat on this second blog for weeks — much longer than the first blog — running my own mental epochs, wondering if my personal learning rate would set me up to reach a natural conclusion, or if my own logic and reasoning around this entire project would crash from the literal barrage of information I've taken in (Ha! Just kidding).

I think I'm ending up in the same position as my own project's model: a realization that the work I've started isn't done, that there's so much more for me to do and create (stay tuned for part three!), but also recognizing that natural stopping points are more of an art than a hard science at times. I'm liking this stopping point. It's a reminder that the road I'm being taken on is actually a continuous journey of learning and becoming — just me, the machine, and nature.