The grass was actually greener

3 months ago, I wrote an enthusiastic blog post about a new research project I was excited about. Today, I have a paper! It’s not published (yet), but we have a full manuscript under submission that I’m quite proud of. It’ll be posted to arXiv soon as well.

In 2024, I spent 10 months working on discrete diffusion models without any positive results. A trial by fire. I learned how to create and test principled research questions, stay organized, and (most importantly) I simply became faster, smarter, and better. In 2025, I spent 3 months and wrote a paper. I know what it took to get to this point, what I could have done better, and what I did really well.

Of course, here’s a link. I’m happy to receive any comments or critiques. Even typos. Condensing the last three months of work, hundreds of GPU hours, and thousands of lines of code into 10 coherent pages was by far the hardest part of this process. It’s a preprint for a reason.

I’m going to clean things up for a better public-facing code release, but you can check out my research codebase too. Everything’s in there, full nude. Last year, I made ~1400 commits and thought that was a lot. Since the start of 2025, I’ve made >1100! I went 13 straight weeks without missing a commit and only broke the streak to spend a full weekend writing. I roamed for data on top of a mountain in Zion National Park to check on experiments in the week leading up to the TACL deadline.

What’s next?

A good research project is a gold mine for new ideas. There’s 1000 more experiments I wish I could have done, so I can always start there. There’s also a new idea that came up in our discussions that we call “causal cross-attention.” It seems elegant and useful. I could also spend the summer building Prompt Choreography into a full-fledged serving system, or maybe just integrate with vLLM. Good ideas leap abound.

I also want take a step back for a bit and look at what other opportunities lay around me. Some of my best friends just dropped out/quit their full-time jobs to pursue a new startup. I probably won’t join them, but now is the time to truly grab life by its balls. There’s a trove of papers I put off reading until I finished my manuscript. I never got past the first chapter of Sutton and Barto. I recently got excited about using MCMC to sample adversarial prompts. Children of Dune is collecting dust on my bedside table.