Jeremy Howard: Deep Learning Frameworks - TensorFlow, PyTorch, fast.ai | AI Podcast Clips

Duration: 9:01 1409 transcript words EN summary

Video summary

Jeremy Howard outlines the evolution of deep learning frameworks, tracing his team's journey from Theano and Caffe to TensorFlow before settling on PyTorch and eventually integrating fast.ai. He identifies a critical flaw in early systems like TensorFlow: their reliance on static computational graphs that require defining all operations upfront. This rigid structure prevents interactive debugging within environments like Jupyter notebooks and hinders the iterative research process essential for modern AI development. In contrast, Howard praises PyTorch as a revolutionary entrant that allowed researchers to use standard Python syntax while automatically handling GPU execution when necessary. This dynamic approach eliminated significant boilerplate code related to training loops and gradient management, enabling rapid experimentation which was vital for projects like ImageNet competitions and academic research. Despite the advantages of PyTorch's flexibility, Howard notes a barrier to entry: newcomers often struggle with writing their own training loops from scratch without guidance. To address this, fast.ai developed a multi-layered API that allows users to train state-of-the-art neural networks in just three lines of code at the highest level. This design philosophy provides an accessible starting point for beginners while allowing them to dive deeper into lower-level controls as they gain expertise. The framework has become instrumental not only for Howard's students but also for participants winning major competitions and researchers publishing academic papers, effectively bridging the gap between ease of use and deep technical control without forcing users to manage low-level details immediately. The discussion turns sharply critical toward TensorFlow 2.0, which attempted to mimic PyTorch's interactive capabilities through a feature called "eager execution." Howard argues that this was merely syntactic sugar added on top of a fundamentally flawed runtime rather than a genuine architectural overhaul. Consequently, operations in the eager mode version of TensorFlow run approximately ten times slower than equivalent code in PyTorch because Google failed to retool its foundational components. He describes the underlying codebase as disorganized and riddled with technical debt resulting from rapid development by many contributors over time. While he acknowledges that Swift for TensorFlow could potentially offer a different solution leveraging Apple's compiler optimizations, he remains skeptical about current progress given historical issues within the Swift community regarding numeric computing performance priorities. Looking toward the future of language support in deep learning, Howard expresses cautious optimism about Swift but highlights significant hurdles before it becomes practical for widespread use. He estimates that achieving super-practical utility with Swift is roughly three years away due to the need for a robust data science ecosystem and better handling of specific architectures like recurrent neural networks (RNNs). Currently, Apple's low-level libraries are not well-optimized in Swift, though he notes that Swift for TensorFlow can directly utilize existing Python code and libraries as an intermediate step. This interoperability allows users to run notebooks entirely in Swift while still accessing the vast resources available in the Python ecosystem, serving as a transitional bridge until native performance catches up with industry standards. Ultimately, Howard's recommendation for new students entering the field today is clear: they should begin with fast.ai and PyTorch to master core concepts quickly before worrying about framework migration costs. He emphasizes that switching between frameworks like MXNet or TensorFlow takes only a few days if one understands the underlying mathematical foundations, making the choice of initial tool less critical than conceptual understanding. However, he warns against investing time in learning outdated workflows due to technical debt in major libraries unless absolutely necessary. The consensus is that while Python remains dominant for now, the industry must eventually evolve beyond its current limitations, and frameworks like fast.ai provide the most efficient path to becoming an expert practitioner capable of achieving world-class results without being bogged down by inefficient infrastructure.

Read the full video transcript

the from the perspective of deep learning frameworks you work with fast AI tickler this framework and pi torch intensive flow what are the strengths of each platform your perspective so in terms of what we've done our research on and taught in our course we started with Theano and care us and then we switch to tensor flow and care us and then we switched to PI torch and then we switched to PI torch and fast AI and that that kind of reflects a growth and development of the ecosystem of dig learning libraries siano intensive flow were great but we're much harder to teach and do research and development on because they define what's called a computational graph upfront less data graph where you basically have to say here are all the things that I'm going to eventually do in my model and then later on you say ok do those things with this data and you can't like debug them you can't do them step-by-step you can't program them interactively in a Jupiter notebook and so forth pi torch was not the first 4pi torch was certainly the there's the strongest entrant to come along and say let's not do it that way let's just use normal Python and everything you know about in Python is just going to work and we'll figure out how to make that run on the GPU as in when and necessary that turned out to be a huge a huge leap in terms of what we could do with our research and what we could do with our teaching and because it wasn't limiting yeah I mean it was critical for us for something like dawn Bench to be able to rapidly try things it's just so much harder to be a researcher and practitioner when you have to do everything up front and you can inspect it problem with pay torch is it's not at all accessible to newcomers because you have to like write your own training loop and manage the gradients and all this stuff and it's also like not great for researchers because you're spending your time dealing with all this boilerplate and overhead rather than thinking about your algorithm so we ended up writing this very multi-layered API that at the top level you can train a state-of-the-art neural network in three lines of code and which kind of talks to an API which talks from OPI which talks from API which like you can deep dive into at any level and get progressively closer to the Machine kind of levels of control and this is the first day a library that's been critical for us and for our students and for lots of people that have one big learning competitions with it and written academic papers with it it's made a big difference we're still limited though by Python and particularly this problem with things like recurrent neural Nets a where you just can't change things unless you accept it going so slowly that it's impractical so in the latest incarnation of the course and with some of the research risked out now starting to do we're starting to do stuff some stuff in Swift I think we're three years away from that being super practical but I'm in no hurry I'm very happy to invest the time to get there but you know with with that we actually already have a nascent version of the first AI library for vision running on switch 19 so far because a Python for tensorflow is not going to cut it it's just a disaster what they did was they tried to replicate the bits that people were saying they like about a torch the is kind of interactive computation but they didn't actually change their foundational runtime components so they kind of added this like syntax sugar they called TF eager tons of logging in which makes it look a lot like PI torch but it's 10 times slower than PI torch to actually hmm do a step so because they didn't invest the time and like retooling the foundations cuz their code base is so horribly company yeah I think it's probably very difficult to do that kind of rejoin yeah well particularly the way tensorflow was written it's written by a lot of people very quickly in a very disorganized way so like when you actually look in the code as I do often I'm always just like oh god what were they thinking it's just it's pretty awful so I'm really extremely negative about the potential future if it by the flaws of the FET Swift for tensorflow can be a different beast altogether it can be like it can basically be a layer on top of M lar that takes advantage of you know all the great compiler stuff that Swift builds on with LLVM and yeah it could be a think it will be that's what you fantastic well you inspired me to try Evan Roo truly felt the pain of tensorflow 2.0 Python it's fine by me but yeah but it does the job if you're using like predefined things that somebody's already written but if you actually compare you know like I've had to do because I've been having to do a lot of stuff with 10 so far recently you actually compare like okay I want to write something from scratch yeah like I just keep finding is like oh it's running ten times slower than pi torch so is the biggest cost let's throw running time out the window how long it takes you to program that's not too different now thanks to chance to flow eager that's not too different but because because so many things take so long to run yeah you wouldn't run it at ten times slower like you just go like oh this is taking so long yeah and also there's a lot of things that are just less programmable like TF data which is the way data processing works and tensor flow is just this big mess it's incredibly inefficient and they kind of had to write it that way because of the TPU problems they described earlier so I just you know I just feel like they've got this huge technical debt which they're not gonna solve without starting from scratch so here's an interesting question and if there's a new student starting today what would you recommend they use well I mean we obviously recommend fast AI and pi torch because we teach new students and that's what we teach with so we would very strongly recommend that because it will let you get on top of the concepts much more quickly so then you'll become an extra and you'll also learn the actual state-of-the-art techniques you know so you actually get world-class results honestly it doesn't much matter what library you learn because switching from China to MX net to tensor flow to PI torch is gonna be a couple of days work as long as you understand the foundations well but you think we'll Swift creep in there as a thing that people start using not for a few years particularly because like Swift has no data science community libraries Oh basil wing and the Swift community has a total lack of appreciation and understanding of numeric computing so like they keep on making stupid decisions you know for years they've just done dumb things around performance and prioritization that's clearly changing now because the developer of chris Christie at developer of Swift Chris Latner is working at Google on Swift Potenza flows so like that's that's a priority it'll be interesting to see what happens with Apple because like Apple hasn't shown any sign of caring about numeric programming in Swift so I mean hopefully they'll get off their ass and start appreciating this because currently all of their low-level libraries are not written in Swift they're not particularly swifty at all stuff like ml they're really pretty rubbish so yeah so there's a long way to go but at least one nice thing is that Swift for tensorflow can actually directly use Python code and Python libraries in a literally the entire lesson one notebook a first AI runs in Swift right now in Python mode so that's that's a nice intermediate thing you