Slow feedback loops rule every (living) thing around me

Or how I came full circle on the need for faster feedback loops in bio.

When I got started in bio, my Big Idea was that slow feedback loops were the biggest rate limiter on progress. As I learned more, I started to think that maybe I started out naive and things were more complicated. Now I've come full circle: slow feedback loops make progress in bio harder and psychologically much less rewarding.

The magic of fast feedback loops

Fast feedback loops are like psychological alchemy. They are the dark matter that make video games, programming, woodworking, and many of the more addicting sports flow-inducing. They're the spark that ignites our minds' ability to engage in mechanical sympathy by extending our sense of self into our tools.

Fast is fun. Fast is exciting. People gravitate to programming because they can conjure software from the ether while basking in programmer flow and then iterate quickly. In the context of programming, many people have written about how even small differences in interaction latency can transform a computer from an external artifact to a cognitive augment. In James Somers's words:

Google famously prioritized speed as a feature. They realized that if search is fast, you’re more likely to search. The reason is that it encourages you to try stuff, get feedback, and try again. When a thought occurs to you, you know Google is already there. There is no delay between thought and action, no opportunity to lose the impulse to find something out. The projected cost of googling is nil. It comes to feel like an extension of your own mind.

Fast feedback loops are also key for effective learning, as shown by Kahneman and Klein's adversarial collaboration. Kahneman and Klein were famous for studies showing opposite seeming results. Kahneman became famous for demonstrating the weakness of human intuitive judgement in clinical and legal decision making, whereas Gary Klein's work argued for the power of intuitive decision making by policeman, firemen, and other high stress professionals. To get to the bottom of this, they collaborated for 7 years on an adversarial collaboration, which resulted in a paper, "Conditions for intuitive expertise: A failure to disagree". One of their main findings was that humans develop valid intuitions when feedback loops are fast and high signal and invalid ones when loops or slow and noisy. Intuitively, this makes sense. Learning requires trial, error, and revision. Outside of purely intellectual domains, the more trial and error cycles you can perform in a given time, and the more accurate the error signals, the better you'll learn.

Feedback loops in bio are especially slow

Everyone knows feedback loops in software are fast. That's why so much writing about the magic of fast feedback loops focuses on programming. But even relative to other physical scientific and engineering disciplines, bio stands out as having especially slow feedback loops. In hardware, for example, cost reductions have enabled much faster tinkering. Rasberry Pis, FPGAs, Arduinos, CNC machines, combined with powerful electronic design automation enable fast, low cost iteration in the early stages of projects.

On the other hand, despite rapid progress on tools, feedback loops in bio (even preclinical or non-therapeutics work) remain slow and costly, especially without fancy, hard to set up equipment. A simple bio experiment like getting E. Coli to express GFP can take a day, but few interesting experiments take under a week. In therapeutics, it's near impossible to get a full experimental cycle done in under 2-4 weeks assuming you're working with mice. On top of this, even simple experiments are relatively costly. Glassware is famously overpriced and even with dropping costs, sequencing runs still add up quickly for a lone tinkerer. People like Sebastian Ciocoba, David Ishee, and Josie Zayner are the exceptions that prove the rule.

Hardware iteration cycles are longer at industrial scales but still beat bio's. In the Acquired podcast on Nvidia, they talk about how a major impetus for improving their electronic design simulation tooling was the fact that tapeouts from TSMC had a month turnaround time. Having to deal with multiple months long turnaround times warranted a major strategic shift for Nvidia, but in therapeutics month+ turnaround times are routine and seen as normal. A full exploration for why we see this divergence would require a longer post, but I believe bio's heavy focus on high margin businesses means it lacks the same pressure for cost reductions and speed ups that have driven this progress in hardware.

Faster feedback loops in bio would accelerate progress

Evolution is an existence proof that, with enough iteration cycles, bio can do incredible things. To paraphrase Archimedes, give me a diverse enough population and enough generations and I can move the world. Take life extension as an example. Part of the reason extending lifespan is so hard is because iteration cycles are proportional to one human life. By contrast, if we had the will and got the cost of running the evolution experiment low enough, I'm quite confident we could evolve a C. Elegans (or yeast) with a 10x longer lifespan just through selection. If you want more examples, just take all the different organisms in the world...

Evolution's power is that, while it's basically the dumbest possible approach, it's an extremely general approach to steering biology towards a given outcome. As long as you have a way to select towards the outcome you want and it's reachable through repeated small (genomic mutation) steps, you can get there eventually. Even if we could just speed up cycles and reduce costs for directed evolution methods like PRANCE by 10x, we could accelerate progress in areas like protein and cell line engineering dramatically.

Evolution works well for a certain class of design tasks, but doesn't generalize to scientific discovery or areas like medicine. Fortunately, the lesson about feedback loops applies more broadly. Many bio problems are hard because you only can run 1 serial experiment per month or even year but would be trivial if you could run one experiment per hour (or minute) with minimal risk and low costs. For example, if it were possible to test thousands of therapeutic candidates in humans safely per day and get results, I'm willing to bet we'd make much more progress on complex disease cures even with our current levels of understanding. Similarly, if end-to-end cloning, cell culture, and assay cycles took minutes rather than days to months, biology research would speed up by an order-of-magnitude.

How to improve: Make fast feedback loops a priority

Making biology experiments work requires determination, attention to detail, and a slight masochism. As a result, especially relative to us lazy ass software people, biologists have an impressive pain tolerance for difficult work and an accompanying patience to wait for results from multi-month experiments. While I deeply respect them for this, I think it can also hold back the field by allowing it to stay stuck in a state where many things that should be easy and fast remain slow and hard.

So, the first step to improving is establishing faster, cheaper feedback loops as a priority. This has worked well for sequencing and synthesis, where maintaining Carlsson curves became a rallying point for a large swath of the field. I was recently at an event where a synthesis provider bragged about how their turnaround time for orders was a week faster than competitors'. Yes! I love this! Now we should extend this same mindset to as many other areas of biology as possible. Biology needs more laziness!

Every difficult to use tool is an opportunity for improvement. Every expensive reagent is an opportunity for (chemical or protein) engineering to optimize cost and efficiency. PCR should be dirt cheap and zero touch. Cloning should be easy and fast or, as the synthesis providers are hoping, let's make it so we don't have to do it at all in a lot of cases.

For therapeutics, the goal should be to be able to predict trial outcomes using experiments that take a month or even a week. For SynBio, I want to be able to do a grad student's year+ project's worth of (painstaking) generic circuits work in a week.

These things sound crazy, and some of them probably are. But others aren't. They just take will!

The predictive validity critique

One critique based on Jack Scannell's great work could be that focusing on faster feedback loops (and scale) in therapeutics would lead to even lower success rates due to lower predictive validity. In their paper introducing predictive validity, Scannell and his collaborators used a simple model to show that in a multi-step process such as the clinical trials process, achieving higher throughput can actually be detrimental for success probability of it comes at the cost of a lower correlation between early stage tests and eventual success (clinical trial success). A similar argument applies to faster feedback loops. Being able to do 10 iteration cycles towards the wrong proxy goal just means we can get further in the wrong direction faster.

I acknowledge this criticism as a real risk (if anything am hyper aware of it from being immersed in the gene therapy field, which places heavy emphasis on in vivo results) but view it as something to solve for rather than a reason to not aim for orders-of-magnitude faster feedback loops in bio. We need to be able to iterate quickly while maintaining high predictive validity. This probably involves developing better model systems and then ideally developing powerful ML predictors which can accurately predict how a result in a set of model systems will translate to a human (or whatever the target application is). Fortunately, many people who work in ML for drug discovery are already aware of this problem and exploring various approaches to mitigate it.

Conclusion

Stop accepting slow cycles. You (biologists) work way too hard to be stuck with bad, slow tools. Slow is fake. Expect better! Demand better! Someone no smarter than you made the tools we have and you can make them better.

Tech has many vices but laziness and impatience are virtues that biology can and needs to learn from. So let's get started.

Acknowledgments

Many people, including some at SynBioBeta, with whom I’ve discussed this topic at various points.