Python as Orchestrator: When to Glue, When to Compute - Freya Bhushan Mehta - 2026

It's a fairly common topic of of talk at Python conferences about when you should be doing things in rewriting things in Rust or in C or whatever. And this one really stood out to us this year as something that really goes into the why and when of when you should be uh when you should be doing things in native code and when you should be doing things in Python. And I'm really excited to see this talk. So thank you for that. Hi everyone. Uh I think this one is a bit more technical. This it's it's a bit more on the technical side. So I'll do my best to keep everyone awake. Um my talk is on Python as orchestrator, when to glue and when to compute. Um my name is Freya Mehta. I'm a software engineer at Bloomberg in London, where I work on the derivatives pricing library engineering team. Uh my work lives in the intersection of software engineering and quantitative finance. Uh we're building C++ and Python systems that price and structure complex financial instruments. Um so whenever I tell people I code in Python at work, I usually get a very skeptical look. Uh and I know exactly what they're thinking. Um isn't Python too slow for the kind of work I do with the real-time analytics and low latency. Um I mean slow compared to the compiled languages like C++ here. Um and if you're talking about raw computation, they are absolutely right. Um so I like to give this an analogy to people that if you think of CPU as a high-speed chef, uh C++ gives it a pre-written optimized recipe in its native language, whereas Python is more like someone standing outside the kitchen reading instructions out loud step by step while the chef is waiting. Um also quick note, I'll be using C++ as my compiled language throughout the talk, but the framework applies equally to Rust, Go or anything else compiled. Um the questions and the talk is more about where to draw the boundary and not which language is on the other side. Um so yes, if you care about about squeezing out maximum performance from the hardware, Python is at a disadvantage and there are real reasons for that, right? First, it's compiled language translating everything into optimized machine code ahead of time, whereas Python interprets bytecode at runtime. Um second is static typing, right? It static typing lets the compiler generate tight specialized instructions. Uh and Python inspects types at every operation. Um C++ arrays sit in contiguous memory that the CPU can blast through, whereas Python lists are arrays of pointers to scattered heap objects. Um and fourth, now historically the GIL has limited CPython to one thread of Python bytecode at a time. Um although Python 3.13 has introduced an experimental free-threaded mode that starts to address this, but it's still not very well adopted. Um so yes, Python is slower at raw computation, um but then why is Python everywhere? Python powers Netflix's recommendation engine. Um it processes petabytes at Spotify and it supports financial systems handling trillions of dollars as well. Um the key insight over here, I'll give you the key takeaway right now. It's we use Python for the logic, but the heavy computation is done the compiled code does the entire heavy lifting for you. >> [snorts] >> Um so the question isn't is Python fast enough, it's when should Python compute and when should it just glue things together. And that's what this entire talk is about. We'll walk through the while we walk through the talk, we'll also build a decision framework for choosing that boundary, when to compute and when to orchestrate, because getting it right will let you build systems that are both fast and easy to evolve. Um so before we get into specifics, let me just show you the shape of the problem over here. Uh because if you've written any kind of data pipeline, you've already seen this. Um there are four stages. Acquire data, which is basically you're just pulling external sources, APIs, databases and message queues. Um transform, you're preparing that data for computation. Uh it's coordination work here, not the math. Uh compute, this is the heavy lifting stage where you can do Monte Carlo simulations, there are matrix operations, there are neural network trainings as well, the actual work your system exists to do. And output, you're packaging results and delivering them to consumers over here. Uh but the same shapes shows up everywhere. ML pipelines scrape data, clean it, train models and serve predictions. Video processing ingest footage, normalizes formats, encodes and streams. Um in my domain, the pipeline looks like this. It we acquire market data, uh we calibrate mathematical models to that data. We run pricing simulations, compute risk sensitivities and finally show them to the on the on our UI as well. Um now here's what makes this a Python problem. At every stage, you face the same question. Does this piece of logic exist in should it belong in Python or should it belong to a compiled code? Um and here's the key insight. The answer will be different at every stage. So let's just walk through the pipeline from left to right and see what we can learn. Um on the left side of the pipeline is data acquisition. This is the inlet where your system first meets the chaos of the outside world. Now the here the choice between Python and compiled code at this stage comes down to just one question. What is your system actually waiting on? If the bottleneck is the network, then choose Python. In this world, um your code will spend most of its time waiting. Um waiting for the server, waiting for uh the database query to return, wait the actual work, uh which would be translating schemas, uh normalizing formats, it's all coordination, none of it is computation over here. Uh Python handles this very naturally. Uh with asyncio especially, a single thread can manage thousands of open connections juggling all those IO weights without burning CPU. Uh and the coordination logic uh itself, right? Um it's genuinely complicated, but it's not very compute intensive. Uh it needs to be correct, readable and easy to change, not necessarily fast. So at also at this stage, fast isn't about um CPU cycles, it's about like the throughput and the responsiveness. Um and Python is fast enough to saturate your network uh network pipe uh while providing the um operational speed that's that's needed to survive and adapt to the mess of the external APIs. Um and if the bottleneck is CPU, choose or consider compiled code. Um this is a different world. Over here, your data arrives in a raw binary stream, a high-frequency hardware uh sensory telemetry, a dense binary protocol. You're not waiting on the network anymore. You're racing to decode bits as fast as they're arriving. Um there's also a hybrid approach, by the way, worth knowing about. Um you can have like a small small compiled uh service and which handles the binary decoding, pushes clean data into a shared buffer and your Python orchestrator reads from that buffer to handle all the heavy um high-level logic. Um throughput where it matters and flexibility everywhere else. So what should you ask yourself, right? Um here's the framework for uh data acquisition. Uh basically questions that you I want you guys to take back to for your own projects over here. Um first, what is the bottleneck, the network or the CPU? If your code spends most of its time waiting on IO, it's coordination, Python excels here. If it's racing decode to decode bytes, that's computation. Consider compiled code. Second, how is the data structured? Uh human-readable formats, JSON, CSV, SQL results, they all play to Python's strengths, uh which has got mature libraries and readable parsing code. Uh dense binary formats with bit-level packing lean towards compiled code, um which can basically operate directly on the raw memory. Um third, how complex is your landscape, which means how many sources or how messy are they? If you're pulling from a dozens of different APIs, each with its own schemas and failure modes, Python's readability and ecosystem make that manageable. Uh if you have one or two high-volume fire hoses, the problem is throughput, not coordination. Um and fourth, and this one is a very easy one to look overlook. Um how often does the ingestion logic change? So data schemas evolve constantly, right? Um APIs get versioned, new sources appear. If this logic lives in C++, every change triggers a cycle, recompile, relink, rebuild packages, a full QA, coordinated deployment. Days, sometimes even weeks. Uh in Python, a developer can update a schema mapping, test it against production uh data and even ship it in the same afternoon. So, next before we get into the heavy computation, there's a stage that often gets overlooked, which is data transformation. Here you're just reshaping the data, you're cleaning strings, you're converting types. It's not very glamorous, but it's often where performance quietly disappears. The interesting thing about the transformation stage is that the decision isn't really about the speed over here. It's about data volume and logic complexity. If you have 500 if and else rules for cleaning messy data, handling edge cases, normalizing fronts, that's Python's strength. The logic is complex, it changes often, and the readability matters far more than the raw throughput. But if you have two simple rules and 10 billion rows to apply them to, the equation flips. At that scale, even a small per row overhead adds up. Now the good news is that for most transformation work, you don't actually have to choose between Python and C++ because libraries like NumPy and Pandas let you stay in Python syntax while the actual work runs in compiled code underneath. So the rule of thumb I would say is that if your transformation is vectorizable, you can basically express it in as an operation on a whole column rather than a loop over individual rows, stay in Python with these libraries. Um You get compiled code performance with Python's readability. So the transform stage reinforces a similar two questions. Is the bottleneck in the logic complexity or the data volume? And how often does the transformation logic change? Complex, evolving cleaning rules belong in Python. High volume, stable transformations belong in compiled code. Or in Python libraries that depend or delegate to compiled code for you. So now we move to the heart of the pipeline, the computation. This is the stage that justifies the existence of everything else. Um whether you're training a neural network, encoding video frames, or like me running a Monte Carlo simulation, this is where C++ genuinely shines, but it's worth understanding why. C++ gives you control over over how the data sits in the memory. So the CPU can grab it without waiting on the RAM, that's cache locality. It It lets you also use the SIMD instructions that process multiple data elements per instruction. And it gives you more predictable latency. There's no garbage collector that might pause your code mid simulation over here. But here's a very critical question that a lot of people skip. Is the math already solved in a library? So if you're using a standard model, a known neural network architecture or like a linear regression or a well-established pricing formula, there's a very good chance that a C++ backed Python library already does it. PyTorch, NumPy, scikit-learn, OpenCV, these give you compiled code performance with Python syntax. So you don't need to write the C++ yourself. You only need custom C++ when you're writing a proprietary algorithm that no existing library supports. And even then, not all of it belongs in C++. And now there's the final stage, output. The results are ready. Now they need to reach whoever or whatever consumes them. This stage has surprisingly a very clean decision rule. Who is the consumer? If the consumer is a person, a trader looking at a dashboard, or data scientist reading a report, a user getting a API response, Python is the clear choice. The ecosystem for making data useful to humans is unmatched. We have Matplotlib for charts, fast APIs for fast API for web APIs, Streamlit for dashboards. That's that's completely Python's territory. But if the consumer is another machine, a low latency automated system, a raw binary stream to another server, keep it in compiled code. Don't convert data to a human-readable format just to send it back to your machine that doesn't really need it. Another practical consideration is the serialization tax. So if your compute engine just finished a massive calculation in let's say C++ and the result is like gigabytes of data, copying it across the Python boundary just to write it to disk is wasteful. Stay in C++ for the output, but if the result is like a summary, like a final price, or like a classification label that you want to share, then hand it off to Python because that'll be cheap and the flexibility could be worth it. Okay, now that we've talked about when Python should glue and when it should hand off to compiled code, there's a critical detail that I've been just glossing over so far. I keep saying Python calls C++ like it's free. Like you just write a function call and it works. It does work, but it's not free. There are several ways. First of all, Python can connect to compiled code depending on your language. For C++, you have PyBind11 or Boost.Python. Yeah, for Rust you have PyO3 and Rust-CPython. And for Go you have Cgo with Python bindings. Now all of these share a common characteristic. Crossing the boundary between Python and compiled code has a real measurable cost. Every time you cross from Python to compiled code and back, you pay a tax. Conversion types, which is Python int to like int64, Python string to char star. Second would be reference counting and memory management. Global interpreter lock management. Memory layout translation, where you're translating from Python list to arrays or Python dictionaries to structure. For a single call with a large payload, where you're saying here here a million points of data, go compute, the tax is negligible. You cross once and the compute dominates. The computation dominates. There's no problem over here. But here's where it gets interesting and where teams often run into trouble. Consider a common scenario. You have a fast numerical computation written in compiled code with Python bindings. User calls it from Python, clean interface, and everyone's happy. But someone writes a loop in Python that calls this function thousands of times with just slightly different inputs. Now here the individual call function runs in microseconds, which is blazing fast, but the boundary crossing, the Python to compiled code and compiled code back to Python thousands of times in a loop, this will dominate your wall clock time. So when you profile it, you'll realize that the computation just took 20% of your run time, whereas boundary took 80% of your run time. The computation is found fast, but the boundary is slow. Now here the fix isn't about making the computation faster, but the fix is redesigning the interface. So instead of calling compute single and just passing one input, maybe call and calling that thousand times over, maybe create a compute batch and pass all those thousand input parameters and just send it once. There's one boundary crossing, one big payload. Python bundles up the work and hands C++ the job and gets back a single batch of results. A lesson I would let you guys take from here is that design your boundaries to be crossed infrequently with larger payloads. Have chatty interfaces. I mean, no, don't have chatty interfaces. Many small calls will kill your performance gains. So the Yes, so here's another thing. Making this boundary work well is a real investment. It's not a one afternoon tutorial. You need building tools. You need to build tooling to compile extensions alongside Python code. You need packaging infrastructure so users can pip install the results. You need binary compatibility testing across Python versions and platforms. And finally, you need documentation for both the Python APIs and the compiled internals. This is a lot of infrastructure work. It takes time, it takes a huge team. But once you've made that investment, the pattern will pay off everywhere. Python will orchestrate, compiled code will compute. The boundary is crossed deliberately in larger chunks if you design it well. So finally, this gives me last two questions of our decision framework, which is what is the computational profile? Not just is it heavy, but what kind of heavy? Batch parallel work should cross the boundary once with a large payload. Iterative work that someone might call in a loop, you need to redesign the interface so that the loop moves inside the boundary. And second question, which is what does the boundary cost? The interface between Python and the compiled code is not free. Measure it, profile it, design chunky interfaces, not the chatty ones. Make sure you're not trading a computation bottleneck for a boundary bottleneck. So this brings me back to where we started. Is Python too slow? Orchestration, coordinating services, translating data, managing workflows, handling logic that changes frequently. Python is not too slow, it's exactly right. For heavy computation, like numerical loops, Monte Carlo simulations, GPU kernels, yes, Python is too slow. But that's okay because Python's superpower was never raw speed. It's It's being the best orchestration language we have. When I started working on rising systems, I thought Python would be just used for scripting and testing. And I thought the real work will be done in C++. But I was wrong. Python is the reason developers can install a library with pip install, write three lines of code, get a production quality result, even though gigabytes of compiled C++ are doing the heavy math underneath. And that's not scripting, that's orchestration. And it's one of the most important layer in the stack. The entire scientific Python ecosystem is also built on this pattern. NumPy, SciPy, PyTorch, TensorFlow, Pandas. Python orchestrates, compiled code computes. This is a very well-established pattern across the industry. Yeah, so the next time someone ask you us or tells you Python is too slow, tell them what is Python actually doing in your system? If the answer is orchestrating, coordinating and just gluing components together, then it's not too slow, it's it's doing exactly what it's supposed to do. But if the answer is tight loops over large data sets, yes, move that loop and but keep the orchestration. Because Python doesn't need to be fast, it just needs to be smart. Thank you. >> [applause]