Creating a command-line app in Python [PythonLiveLessons #4]

to join a live broadcast of python live lessons tune in to my youtube channel sebastia amato every tuesday 10 p.m new delhi time 6 pm central european time and 9 am los angeles time hope to see you then and for now enjoy this episode [Music] hi everyone welcome to episode 4 the fifth episode of python live lessons my name is sebastie amato hi everyone i eduardo ole oye oleole and what i want to do today is pick up where we left off during the last we do episodes of the previous episodes in which we made a python module called a markdown code exec um yeah happy to be here too happy to see that there are some some viewers um and what we're going to do today is actually take the python module that we made during the previous episodes and turn it into a command line application so something that you can run from the console or the terminal or the command line however you want to call it so let's actually really very briefly say what our project does mark down code exec and as always so the main reason for me to do this live rather than to actually just record it and put it online is to uh to to allow a few questions right so if you have questions interrupt me at any time and i will try to engage with with the questions so what does our project do markdown code exec so it essentially takes a markdown file as input and markdown is text and with a particular simple markup syntax that allows you to indicate for example that something is code so here between the three backticks we have a bit of python code that says print hello world and the attributes between the curly braces indicate that that is actually python code and that we want to capture the output now in the capturing of the output that's where our markdown code exec application comes in or a module right now it's just a module and what does that mean well it literally means that it just captures the output of the print statement print function call which is hello world in this case and actually inserts it into the markdown after the code that was executed that's what it does um so so um what have we done so far so we in the first episode we created the sketch so sort of a basic outline of what the applic program is supposed to do approximately then we turned that sketch into a nice python module or python package that you can import as you are used to doing with other python modules then we actually uploaded our project to get in github just to keep track of the versions and everything and today we're going to turn our program into a command line program so what does that mean it means you don't no longer have to start python and then import it but rather that you can actually execute it as as you can execute a normal command line program um and how do we actually want to execute our program well let's call the program markdown code exec um and then i want to be able to call it in two different ways i want to be able to sort of use the hyphen i or hyphen hyphen input argument and then specify an input file and a hyphen o and then the output file right and that would mean read from in dot md and write the process output dot output to output md and i want to have a slightly fancier for the sort of the code phonetics fancier way of being able to actually run this application i want markdown code exec to be able to read directly from the standard input and to write to the standard output if that doesn't mean much to you basically the idea is that in linux and also mac os you have a sort of concept of sort of a standard input that is sort of fed into your program and a standard output which is what the program prints out and then there are tools in which you with which you can connect these things this sounds a little bit abstract but the example that you see here that get in dot md means print out the what the contents of in dot md then sort of the vertical line means channel that into what comes next what comes next is markdown code exec so our program that will print it out to the standard output and then this thing indicates that we want to write that output from the standard output to the file called out.md in other words in a slightly less user-friendly way it does the exact same thing as the thing before but the reason i want to be able to allow also this more complex way of using the command line app is that then you can use this the the app in in sort of uh in a chain of other tools that you might use right but most people would of course do the thing above okay um so let's actually switch to uh rapunzel the code editor and let's get to work so what we have here is on the left hand side we have the application as we've been working on so far we have our module called markdown code exec and you know what let's actually take a look at how we can use it so here we have a file called demo md i will change the working directory oh and also go back to yup there we go go back to this mode and then in order to actually read from demo.md we created in our module markdown code exec a function called parse file here and i can use it in following way you can watch back to previous episodes if you want if you're unclear on how that works but basically i can say from markdown code exec i'm typing this directly in the terminal import parse file and then parse file and then input file demo.md and let's say demo out dot md and then it will actually process the contents from demo.md and write it to demo.md here we are and what you see here here we have this thing that should be captured and indeed the output is captured and inserted below um so this is the the way you can use it as a module how are we now actually going to turn this into a into a an application and there are a few ways to do it i will show you my preferred way and the first thing that you need to do is so create a so-called entry point and the entry point is just a function that is called and that basically is the command line app and i'm going to put that in a separate module that i call underscoreapp.buy the underscore indicates that it's not intended to be used from the outside that's what the underscore means and then in this i'm going to actually define a function and i'm also going to call it app right now i'm going to leave the function empty so i just say app um how can we then so this is just adding a function to the module so it's not yet a command line app if we want to turn this into a command line app i'm going to create a file a top level file and i will call it markdown code exec dot by up there it is and i will very simply in that file mark down code exec i will say from markdown code exec up dot underscore app import app and then i will do something that is a little bit special i will say if name equals main up call the app that's all sounds a bit mysterious but let me explain what this means so basically this is simple right this is just an import statement uh that imports this function from the from the add module then what we're doing here is we're checking whether the magic variable main name sorry has the content is has the value underscore underscore main underscore underscore and why well because python is hard coded is built such that this is only true for a for a script that you call directly so here in the here in this this this module there will also be an underscore name variable but it will have a different value and only if you call a script from directly from as a top level script will this condition be true i hope that makes some kind of sense if not please don't hesitate to ask questions and then we can execute it actually so um let me actually open a terminal here up open tilex yeah should have prepared that better there we are so now i have a terminal here and i will if i now say python and i say markdown code exec.buy i execute that file that works but it does nothing of course because we still have to actually do everything right so basically it doesn't crash but it also doesn't do anything so let's switch back so what we need to do in the in our in our app is basically take control do a few things yeah yeah that's what okay happy to clarify that's the clarified the underscore underscore name thing for you yeah um we need to do a few things basically we want to from the from this module what we want to do we're actually not going to use the parse file function but we're going to use the simpler function called parse text and that takes text so not a file name but just text that's already been written and it returns new text that has been processed so it takes the original markdown as input and and and then x returns the process markdown as output um so that's what we want to do now let's say that we already know sort of what the input file is so we say in markdown in for example is demo built md and markdown out is demo out dot md of course we don't know we this is just for explanatory purposes because later we don't know this right exactly how this works uh how what these files are called because they are specified on the command line but let's say that we actually do uh we do know what they are then what we can do is say okay we're going to read let's call this actually path in and pass out then we can say okay with open path in sfd md in is fd.read so we read all the contents from the from the from the input path then we actually need to call the parse text function so we say from markdown code exec import parse text um and i say md out is pars text md in and then we have done the heavy lifting already and then we simply need to say with open path out sfd and we need to specify actually that this file is for writing up fd build right md out so this would actually work to the extent that oh i should have two empty lines here for style this would work to the extent that it would actually read from read from demo.md and it would write to demo.out.md but of course we don't always want to do that we want to be able to specify that on the command line so how are we going to do that for that we actually need to use the so-called import so-called arc parse module from python this is part of the python standard library and it is and it allows you to basically read get the command line arguments that were passed to your program in a standardized way um and nippodam is saying just as a reminder i'm still interested in how django routers work the rest framework well thank you um perhaps perhaps i should switch to django and beef up the number of live viewers um so it is noted i'm also just i i'm also not personally so familiar with django so i'm it's not just out of uh and i need to know of course how something works in order to be to explain it and then atishe asks what's the significance is does that refer to the comment about django or does it refer to something that i did please clarify if you want so um what we're going to do is use the arcbars module i'm picking up here to actually read the read the command line arguments that were passed to our script and it works like this i will type it and then you will sort of understand how it works because it is relatively intuitive let's actually close this because we don't need it anymore so up there we go so we create a parser object um and then um so at the show you say see my comment i don't know what comment that refers to actually if you specify that a bit more detail then i will try to answer and then to the to the parser object we add some arguments um and it goes like this so what we're saying is that our uh our command line app should be able to accept uh an input file either specified by hyphen i as i showed in the example or by the more elaborate way of saying hyphen hyphen input it should be a string and this is the help thing i will show you in a bit what it means why why why this is useful if you don't spec um and the magic is such that it will automatically go to a variable that is called parser.input you will see that in a bit oh i want to do something very similar of course for the output output output markdown file that's it there are only two arguments and then i say arcs is parser dot parse arcs and this this these lines will do basically the magic of parsing the arguments that are passed to the command line you can do many more things as well here i'm having a very sort of standard criterion but you can have all kinds of checks and different ways to specify arguments the arc bars module is very versatile but in this case in this case we're just basically saying you can specify input or you can specify output and then we have the path in which will be rx dot input and path out which will be arcs.output um this is actually really enough already for our the basic way of calling the function with these arguments so let's actually go here to the terminal and then if i say python markdown code exec dot pi right now i'm still doing it in kind of an ugly way by by first calling the python interpreter and then the name of the script with dot py once we actually create a python package that you can install with with pip we will do that in future episodes this is actually no longer necessary and we can just call it as markdown code exec without anything which is nicer but the the general idea would be the same if i execute this oh we get a oh wait yeah did i yeah okay sorry if i execute this and i say help that's what i wanted to do you you see now the help right and that's why it makes sense to actually specify the help in the in the specify the help when creating the arc parser and that's that will be used here and you see also that you get some explanation of how you can actually use use the the the command line app now if we want to actually run it we say by park code exec.by input demo.md output demo out.md and it will do basically the same as we did before right actually demo.out demo out.md already existed but if let's see actually for just for the sake of argument say demo out 2 dot md if i switch back you see that now we have demo out 2.md that was created this however doesn't allow us yet to actually also read from the standard i input and write to the standard output because you see here that we're assuming that path in is actually a file name and it doesn't necessarily need to be a file name so what happens if we don't actually specify an input or an output well then these variables variables will actually become non and we can make use of that so we can say if path in is non then here we're going to do something that is go that we're going to read from the standard output input sorry else we're going to read from the file all right so here we need to we didn't specify an input so we need to write read from the standard input how do you do that well up there are a few ways to do it but basically i see your comment morpheus and we'll get back to it in a second we will say uh md in is first an empty string and then for line in sys dot std in md in plus equals line so basically systoled std in is like the content of a file and you read it line by line we start with an empty string and we just concatenate the lines that we read to the standard input until the standard input is exhausted and that will go automatically we don't need to check and then we've basically read everything from the standard input something something similar we can do with um wait actually let me answer morpheus would you say that arc parse has advantages over similar libraries except for being in the standard library i used dokopt many many years ago there are probably better ones today i actually the only two libraries that i know for argument parsing are opt bars and arc bars where arc parse is basically the improved version of opt bars both of which are in the standard library i think i don't know dog bars so i cannot really doc opt so i cannot really comment as to whether it is better or worse i can't imagine that there are many things that you would want to do that arc parse doesn't do actually so in that sense it's very uh very convenient but maybe doc opt has some advantages that i don't know all right so and in the same way right here we're assuming that we're going to write to the standard to to to a file that was specified but if we didn't specify an output file then md out or sorry path out will be num so we say if path out is null then we need to do something to write to the standard output and else we write to the file that was specified right here now how do we write to the standard output that is very simple actually really we just say print md out right so what kind of logic do we have here maybe i should comment the code a little bit it's always good to comment the code parse command line arguments that's maybe not actually should i comment that i always feel when i'm commenting code you want to add comments but only things that are not already obvious from the code itself right because if you add obvious things it just takes space if no input file was specified dot read from the standard input input else read from the input file okay do the actual parsing if no output file was specified dot right to the standard output output else right to the output file okay so now we have a little bit of useful comments so let's see whether this actually really works right so um how can we check that well so remember that cat demo dot md will actually print the output print the contents of demo.md here to the output just to the terminal if we say let me clear if we say getdemo.md and we pipeline it as it is called with this vertical line that's typical unix linux mac os stuff basically this will not work on the windows unless of course you install the linux subset for windows or whatever it's called and we pipeline it into our program dot pi then it will actually print out almost the same thing but you see that now it has actually done the parsing right captured the output all right if we now do again almost the same thing but rewrite it to demo out three dot md we have an alternative way of parsing these uh parsing the code of basically parsing a file and writing it to a new file if i switch back to to rapunzel you see here we have demo out three dot md so we have demo out dot md which we created by using a markdown code exec as a module demo out2.md which we did by by sort of specifying an input and an output file and demo out 3.md which we created by using this kind of linux pipelining thing and it works very well i think so um now we have so if you have questions if it's our questions in the chat about uh about actually how to to to do this aspect of creating a command line app i hope it's kind of clear i feel also this is bad this is already the first few episodes were kind of rough in terms of difficulty but here we're actually really doing something that i think is quite doable right um while waiting for that let's actually now because we're doing the entire um the entire code workflow python coding workflow and part of that is of course after you've made some kind of notable improvement then we're going to commit this to git so i'm going to use my favorite tool git gui but there are many other tools and we talked about that in the previous episode so i start get gui up just to remind you here are the unstaged changes which are the things that git doesn't know about but that you might want to do something with well the demo out things we don't want to do anything with it but here's the script the sort of the top level script that we created with the name is main that's what we want to get and here is our module the app module so we also want to get that we click on it and by clicking on the changes we move them from the unstaged changes to the stage changes which remember are the changes that we are going to commit that we're going to give a name and actually log as such so turn module into command line app so this is the name of the commit that identifies what i say up commit push i push it to github there we go it's gone um and actually maybe so if i go to github now say markdown code exec you will see that here we have our commit turn into command line app right so that's the thing that we just did it would also make sense i think now to actually tag this let's say that this is release 0.1.0 we're going to polish it later but this is already a release worthy thing that we did how can you do that in git how can you tag a release well you can say i'm going to do this now not in the gui but on the command line but it's the same thing i typed this slash just so that the cursor goes to the next line that's a bit easier for you to see i feel that way i say git tag release 0.1. oh up there it is git push origin release 0.1.0 up it pushes it what is does it mean to push a tag well it basically means if we switch to back to github i reload the page it means that here under releases you see there is one tag if i click on it tags you see that this is release 0.1.0 and this sort of gives us a point that is anchored to the current state of the code right and that's what a release is right it's sort of a snapshot of what the code is like right now nippodom asks my employer requires me to make command stuff with powershell are there so much different differences versus python except that power um yes i think well i'm not i'm not a power user of powershell so maybe i'm underestimating everything that powershell can do but my understanding is that powershell is kind of like a batch script like language right for very simple for a very simple operations whereas python is a full programming language that allows you basically to do everything and as morpheus is basically seconding that by saying that there are a massive number of libraries for python so but uh i'm saying that with some uh caution because maybe actually i'm underestimating what powershell can do but i think for real sort of first i would say for scripting type of things which are basically a few few commands that you want to execute in a row things like powershell or bash are fine for things that correspond to actual programming you don't want to use those kinds of things you want to use something like python i think not necessarily only python of course but something like python now with that what have we done today we've actually accomplished quite a bit i think again in 30 minutes we've taken our module turned it into a command line app the most important things to learn from this are basically that to create a command line app generally speaking you create you create a function that serves as an entry point sort of the thing that is called when you run it from the command line later when we're going to create a package you also see that this entry point will also be used in the packaging process in this entry point you parse the command line arguments generally speaking with the arc bars library but we've heard there's also dog opt and maybe a few others to do that and we've created the top level script that doesn't very little else except importing the app function checking whether we're executing this from the top level where rather we're executing this as a top level script by checking the name variable and then calling it and that's it that's everything that we did now if you're watching this back later this is where the episode episode ends so thank you very much for watching python live lessons if you are watching live then i would i'm happy to take a few more questions if there are any [Music] you