Free yourself from your ORM with mypy!

as the user sent in bad data they can you can the application can error before it even gets to the database possibly tell the use of something a little bit more useful in terms of what they did wrong if they're using an API and then there's a lot of simple patterns that just come with the rme that make it easy to build out on your application quickly that's why things like Django and rails and stuff are so popular because you can type a few things in and you get like an application really fast and that's definitely cool but there are other ways to get at this so composition is I wrote functional composition here but I mean even object composition is is good of generally being able to compose things makes things more flexible and also lets you hide things so if instead of a subclass you contain a class you can hide that class from the external world and you can trust that I've got my model class but only my model class can talk to the database nobody else can talk to the database and I don't have to worry about queries leaking in somewhere I wasn't expecting or things like that so composition is a great tool for abstracting away complexity that you don't need to know about in the function you're writing or in the module you're writing there's a lot of built-in data structures in Python that provide a lot of these have a lot of power to them you can also like I said you can just write an out you can just write a class you can build your own class hierarchy etc there's one more thing which is going to be the main talk of my main focus of my talk that you can use and that is my pie so talks about my pie it talks about our m's what do they have to do with each other they actually have nothing to do with each other my pie has nothing to do with databases my pie is a static type checker have been has anybody used my pie mostly on that side of the room ok a few people like that so it's a static type checker you can use it to tell it will tell you if you're doing something wrong in your Python code without actually running your Python code and it uses type annotations if you're using a version of Python that supports type annotations in the syntax alternatively you can run it against old code you can run against Python 2 code and you do your type annotations in comments and it can parse the comments out and figure out the types so let's show off a little bit of what it's going to look like so I'm gonna open up file here sure no hitting the wrong character okay so I'm going to write a function and I'm going to annotate its type so this is just from the mypie example but just so we see what it is I've got greeting it's gonna take some name we're gonna give it a type which is a string the colon means this is its type and then the function has a return type and that's what the arrow means this is what it's going to return it's also going to return a string and there we go okay so right now there's no errors because the function isn't actually doing anything even though if you call this function or return none my PI is saying that's okay you haven't actually written anything yet if I returned an integer it will tell me incompatible return type I expected a string but I got an integer you're doing something wrong I can say okay well how about just saying hello and it says okay cool one thing to note here is that I'm taking the name I'm not doing anything on the name that's probably not what I intended my PI is good at checking to make sure that you're handling conditions that you're not accessing things that aren't there stuff like that what it can't do is tell you if you're doing the right thing because it doesn't know what the right thing is so there's a you know there's a lot of discussion about types and tests and which is better you really kind of need both if you want to be really confident that your tests your code is working correctly so it's not a replacement for writing unit tests but it I do find that it's helpful for guiding you saying like hey you forgot about this case maybe you need to handle this case probably want to test to say that the right thing happens in that case but it will at least tell you that you handle the case you so here similarly if I say hello +5 it's gonna say I can't do that because you can't add strings of numbers but if I say hello + name it's totally fine so down here I just have my editor telling me what's wrong with the current file I'm working on so let's start using my PI instead of an ORM so the very first thing we want to do is define sort of our model I'm gonna use I'm gonna use something called a typed dictionary comes from my PI type extensions I think at some point in the future this is going to be merged into the standard typing module but for now it's an extension while they're sort of solidifying the API and I want to I'm going to define let's say we're building an application for dealing with books cuz books are you know really cool and modern and etc so I'm gonna use a typed dictionary type dictionary is just it says look in at runtime represent this object as just a dictionary but I want to say what attributes it has to have and what types those attributes are and so folks have authors sure and they've got titles okay so there we go I should probably type out the code correctly every time okay so that's useful maybe haven't done anything with it yet let's write a function that can take a dictionary maybe it's coming from the user JSON request somewhere else and return one of these things so we get a dictionary coming in call it request it's a dictionary but we want to get a book back out so we can return a book by the brook and it has an author okay so it says that's all right so we've eliminated all of our typos and it says everything's cool now one thing you'll notice here is that this function won't always run to completion if I pass a dictionary in that doesn't have these arguments it will raise an exception and won't return anything it won't return a book won't return anything maybe that's fine maybe we want to do something a different way we can talk about other ways to do it at the end of the talk if I get that far but just sort of be aware that static type checking checking works great on functions that return values it doesn't work great on things that raise exceptions it can't tell you that you mutated the state necessarily it's really about pure functions or what it handles best so we've got this function we can give it a dictionary a generic dictionary we have no idea what's in it and it will either return a book or it will raise an exception okay so let's say we want to be able to save this book into our database so I have written a fake database here but I'm gonna import and it just has it has an insert function so I'm going to I want a function that saves a book it takes a book to save the book has to be a book not a book and it doesn't return anything and then I'm going to insert the book into my database I'm going to pretend like it's sort of II except I called it insert instead of instead of at insert or whatever whatever it's got a I want to save the title and I want to save the author in the database okay everything's good cool now authors are good and all but the publisher is where the money is so we really need to keep track of the publisher because they are the people with the money so we are going to first let's pretend like you know what we need to remember to save the publisher okay oh oh that's interesting it's actually telling me that I can't do that because books don't have publishers so I can come up here and I can say okay books have publishers now oh but now it's saying I can't create a book without a publisher because I said books have to have publishers so now so you can see what it's doing here is it's kind of guiding my refactoring it's letting it's sort of telling me you forgot about this you forgot about that it's keeping track of what you can and can't do with the values that are passing through your your code and it's telling you if you're doing something that will potentially fail at runtime so that's kind of cool so let's add something else so some books come in series not all books come in series but some books are a book in a series so let's try to model that so we're gonna start out by just defining a new class book in series and it's kind of book you can inherit from it it's not really inheritance because remember a runtime these are just dictionaries but it's sort of a hint to the type checker um by inheriting from book it gets all of the attributes of book and then it can have new attributes so I can say I also have a series it's just the name of the series the ID whatever I'm just going to store it as a string okay that is cool but it's not really doing anything useful right now but we can use it to add a new feature let's say our website we want to be able to find things books related to books recommendations engines were a really big deal 15 years ago still are probably people don't brag about them as much because they all just bought them so we're gonna write a function called fine related and it's gonna find one related book or it's going to tell me that there aren't any related books so it's gonna take a book its type is book and we want to return we want it to return the book makes sense right but I don't really know how to implement it yet so I'm just gonna pretend I don't know I'm gonna figure something out okay so how could we go about finding a related book well if the book is in a series then we can just find another book in the same series right they're presumably related so we're gonna write a function find in series that's gonna take a book but that book has to have a series so book in series and it's going to return a book in series okay so we want to somehow do this I've got this database class it has the Select function or not class module has a find I actually called it find and see I want to say again sort of pretending like we're riding against is interesting in this case because it doesn't enforce anything so you really need to enforce things at the application level if you want anything enforced at all so I'm gonna find a book here okay I've got a return the thing I find okay this is interesting so the find implementation I wrote says I'm not always going to find the thing you're asking for maybe that thing doesn't exist in the database so it uses something called optional so I'm gonna go import optional here because I'm gonna need it optional okay and then this is returning an optional book in series still not right because I'm trying I'm trying to return an optional let me if you look at the very bottom here I'll tell you the whole whole problem it's it got a optional dictionary of anything to anything but I told that I was going to return an optional book in series so that's because I need something similar to a parse book but for a book that has a series attached to it so just copy-paste that's a good programming practice so I'm gonna do this book in series and I want it to return a book in series okay I'm probably done right no I'm not okay so we need to return it says I need to I said I would give it a book in series but I gave it a book so book with a series okay and then I'm gonna also break the summer starting to long so I should be good right no okay so book and series needs a series I told it I needed a series it's telling me that saying key series missing from type dik-dik to booked in series so let's do that ok so now our spoken series is written correctly but this still isn't written correctly so what I'm gonna do is I'm going to say well I'm gonna parse book in series and that will turn my JSON object thing dictionary into a book in a series seems like it should be a good idea still won't let me do it because I typed it because I typed a book twice book ok but it really wasn't supposed to let me do it anyway because Parsa book and series expects to get a dictionary but the database may not return a dictionary it may just return none optional means you're either gonna return this or you're gonna return none it's a sort of a subclass of another type called Union which is one or the other but the other is always none so here again it's telling me hey you forgot that sometimes the thing you're looking for isn't in the database and you don't get anything back and if you run this you'll get like a null like none type has no blah blah blah very annoying error so let's save this off save off what we get and then we can say if Rho is not no returned parce book and series row otherwise we'll return none okay cool so we can actually find go to the database and say hey find a book in this same series if you don't find it return nothing if you do find it turn it into this book and series thing and return it back to me okay that sounds cool but we still haven't implemented this find related things so let's figure out how to implement that so what we want to do is be able to say hey look when you asked me to find a related book if the book belongs to a series I know how to do that I can just find another book in the same series and tell you to buy that book if if it's not in a series I haven't figured it out yet we've got to do some machine learning thing or something and I whatever we'll just say nothing so I am going to here say let's try this let's just say find in series book okay there's a couple things wrong with that so trying to return the wrong thing but also find in series wants a book that has a series attached to it we're not sure if the series is attached to it because we haven't checked so let's try this if series in book let's do that otherwise our return none is gonna find anything okay so the first thing is sometimes we return none and we said we'll always return a block but we actually know that we're not always gonna return a book sometimes there is no nothing related to the book it's just like a magical unicorn book it's Jean refining and nobody has caught on to the hot new genre okay so fine related has to be optional we don't know we can't always find something related to it there's still a problem the problem is that find in series wants a book in series a book with a series attached to it but we only have a book so this is actually one of the places that my PI kind of has trouble we know by looking at this code that add this line here the book has a series attribute but my PI doesn't it's gotten confused fortunately there's a way around that which is cast so if you've ever in any C++ code or Java code you know what that does and we have to import it okay so we've told my PI look trust me I checked I'm very certain that book in series is the right type one thing we didn't have to tell it though is that a book in series is a book because we inherited the types from each other even though here I'm saying I want to return a book and here I'm saying I'm gonna return a book with a series attached to it my PI knows that a book with a series attached to it is also a book so it lets us do it so that is that and then one last thing that we need to do we need to be able to save the book so so I want to save the book I want to save the book in series and I should probably save the series huh right no as I forgot to update the type so there we go so the type checker is kind of assisting me as I'm refactoring my code and it's saying hey you forgot about this you you need to check for this attribute this attribute doesn't exist so the other thing it does is it says you can trust that this attribute exists if this was some whole very complicated sub that dealt with books they were belong to a series and it needed to find an image that's the series image instead of the book's cover image or give you a listing of all the books in order you you can trust no matter how many other functions you call that the book has that attribute and if you make a mistake as I did many times up here the checker will check it for you so that is it and I don't know if there's time for any questions or not [Applause] feel to your just fix the problems until yeah so if you go add a field manually any code that doesn't access the field MiFi doesn't know you're supposed to be accessing accessing the field this is a little hand wavy here I think that what you would probably want to do is build out that interface layer between your code in your database you want a unit test that right and so you're kind of doing this in conjunction with writing tests so at some point you'd have a test that says hey look if the user gives me a book that has a series attached to it I better save the series attached to it in the database you're gonna have to go update the database saving code to write the series out at that point that sort of drives you to go start adding the type annotation and then my PI helps you find all the places that you might have broken and go in and sort of fix them up so it's not it can do everything for you but it can help guide you and show you places where maybe you forgot that you did something over here or whatever so that's what it's really good at is discovering those things okay I think that's time [Applause] [Music]