two ai workflows. 1 practical, 1 experimental


Hey this is Isaac,

Here are a couple demos of things I thought were cool. Then an update on a new project i'm working on with Hamel Husain.

First, I’ve been working on Raw2Draft (my writing tool). I just added D3 diagrams to markdown and html text editing and I use one or the other most working hours. D3 is a low level visualization library that gives maximum flexibility but is very painful to write. But agents write D3 surprisingly well and it's amazing for adding visuals to help with clarity and thinking through a problem.

video preview
​

Second, the experiment. I’m working on an agent memory system backed by Anki (spaced repetition flashcards). I want a memory system where I learn alongside my agents. When the agent finds a useful pattern, correction, or gap in my understanding, that can become a card I review later instead of disappearing i nto the chat transcript. This gives me something to study, provides memory of that to my agent, and creates a clear review surface.

​

I’ll keep sharing personal experiments like this publicly. But as anyone building AI products knows, getting from experiment to a reliable and desirable production tool is much harder. To help builders actually solve those gaps, most of my time is shifting from SpecStory to a paid community (the Advanced AI Engineering Club) I’m running with Hamel Husain.

Inside the community, Hamel and I will be doing weekly deep dives on topics like retrieval, memory, evals, and agents with real prod architectures. We'll also do operator reviews where members bring their product, evals, and logs to get blunt, tactical feedback on where reliability will break. For example last week the founder of Ankihub showed the product, evals, and current focus and we diagnosed his bottlenecks relating to annotator alignment and query disambiguation live with the community.

This means I can focus completely on creating the most complete walkthroughs, live builds, and direct feedback (the first community deep dive is tomorrow!). If that is useful for what you’re building, you can see more details here.

If you’re here for the public writing, keep reading. I’ll still be here.

p.s. We'll be doing a free public talks as a sample of kind of deep dives we'll do weekly in the community. Hope to see you in the Tool Architectures (1500+ signup already) one or the Better RAG with Late Interaction one. Otherwise here's a sample of some of the topics we'll cover in the community, tough we will adapt based on what problems members are facing in practice.

Thanks,

Isaac

Isaac Flath

Every post comes from something I've done on a real project. AI tools, development approaches, how I actually build things. You're getting a curation of my taste, not takes on stuff I don't use. Subscribers also get extras: things that went wrong, how my thinking about AI is changing, hacky workflows I use every day, and the occasional personal update. Stuff I share with subscribers because it's a little too personal or unpolished to blast across the internet.

Read more from Isaac Flath

Hey, Registration for PDF to Production closes tomorrow, November 13. Joe Barrow and I teach this Saturday, November 14, from 10 a.m. to 6 p.m. Eastern. It’s $500, including recordings, notebooks, $1200 of vendor credits, and 30 days of follow-up support. See the curriculum and register: https://isaacflath.com/pdf2production If you'd rather stick to free resources and piece things together yourself, check out the Document Roadmap. It goes through an overview of key misconceptions and key...

Hey, There's been a ton of valid skepticism around benchmarks recently, but there's a lot of good reason for optimism too! Joe and I are going to discuss one of those in the document space, which is how unit tests are being used in benchmarks. And all of this is applicable to your own product's evals. After all, a benchmark is just another person's evals 😅. This Wednesday, Joe and I are discussing why unit tests make strong OCR benchmarks and how to design them. We’ll work through examples...

Hey, I wanted to understand how decision models (like Jev) actually work in detail. I use them a ton, and it's clear they're going to be a key part of software moving forward. Joe and I spent an hour going through all of that, using Perplexity Decider (OSS decision model) to go through the architecture. The talk walks through how this is built: Turning a decoder model into an encoder by changing its full-attention layers. Replacing the language-modeling head with a small head that produces...