PBS_2026_08_01

[music] Well, it's that time of the week again. It's time for Programming by Stealth, and this is installment, well, we should call it Tidbit 19, recorded August 1st, 2026. I'm your host, Allison Sheridan, and of course, I'm joined by your co-host, Bart Bush. How you doing today, Bart? >> I'm doing fine. Um, we're not going to mention my foot because I don't need it to podcast. Feet [laughter] are not required for podcasting unless they're pod feet, however that goes. >> Oh, fair. Fair. [laughter] >> Well, this is our last uh recording together before we get to see each other in person. Uh but we won't be doing a programming by stealth from Ireland. We will be doing a security bit, so I know this is an evergreen thing, so we should uh leave it there. But we're excited. >> Absolutely. I'm also shocked it's already August. Where has the year gone? Anyway, >> yeah, but we we're applauding ourselves that we didn't really take so much of a hiatus this summer. Um, we may have stretched the schedule here and there, like it's going to be a month till the next one instead of uh a fortnight, but uh still that's pretty good for us. >> It is. And I think this one skipped a week as well because you were off in was it Maxtock two weeks ago? >> Where was I? No, I think I was in uh Houston. >> Oh, okay. and then straight to Miles because you have >> It's hard to tell. [laughter] >> Five of six trips completed. So, >> wow. Do you get to have like a seventh one for free on your miles? [laughter] >> We probably should. All right. Well, we have a really fun story here. I uh I love this. I've pre-eread the notes and I'm ready to ask all kinds of questions. >> Oh, good. So the name of the episode is building a JavaScript CLI command line interface app with Node.js. And it's kind of a an example of my thesis for doing this entire series ever, which is that coding is empowering. If you can code, then you can make the computer do something you want. If you can't code, you might get away with it with an Excel formula, which is kind of coding or with automator, just kind of coding adjacent. And nowadays with generative AI, you might get halfway there. But if you can code, you can solve the problem and you'll make that AI agent do more because it's helping you do something you can already do. And I ended up spending quite a few weeks scratching a very itchy itch of my own that's very much related to everything I do really because it it started to interfere with my creating show notes which is something I do very often. So it was quite annoying >> to all of us. Yeah. >> Yeah. So we are going to talk about the problem to be solved and we are going to describe my solution. Uh but actually the main focus the thing that makes this worth sharing with everyone is my experiences of dealing with no.js dependencies which has changed since we would have last mentioned no.js JS goodness knows how long ago because of AI, let's face it. >> Um, and also we've never really talked about using JavaScript on the command line. And I have written a command line app in JavaScript. >> So that's a conversation we haven't had yet. >> So I thought this would be a very quick tidbit and I'd be struggling to find enough content. >> And it turns out it's a two-parter. So the blog post is one part, but we're going to record the podcast as two because actually there's a lot of meat here. So if you would like to see what it is I'm talking about, I have created a command line app that I've called Linkify. And like I always do, I release it as open source. So you can go to GitHub and you can peek under the covers and see my code or judge it, your cool. But I've also gone a step further and I haven't just released it through GitHub. I've also released it through npm, the node package manager. >> So if you have NodeJS installed, you can install my command line app with just a command in the show notes and then you can use it with the other command in the show notes. So it's just npm install minus global and then the name of the package is atbarfisher slinkify. Huh? And once you have it, you can use npx to run the app. So npx linkify generate. Give it a URL and it will make you a pretty link. Wow, that's pretty crazy. >> Yeah. So, let's describe the problem I set out to solve again. >> Oh, before we go, uh, so we've talked about it being a two-parter. What we're going to learn here is the process that Bart went through to do this. And then the part two will actually be learning about linkify. >> Yeah. So I'm going to describe the problem to be solved and what it is I was trying to do and then I'm going to describe the biggest problem I hit which is dealing with dependencies safely in 2026. Then we're going to take a break and then we're going to come back and I'm going to describe how the code for linkify actually works because it's a nice example of JavaScript code and how to make a terminal app out of JavaScript code. So that's part two. >> Okay. >> Just want people to have their expectations set where they're going to know where we're going to stop on a cliffhanger. >> It is kind of a tease. I Yeah, I may or may not. Well, to be honest, I struggled to find an order. I changed the order for this about five times while writing the notes. This is the the order that I think works least bad. >> Kind of all over the place. >> Yeah, I'm sure it'll be fine. So, we are going to start with a problem to be solved. So, I write a lot of show notes, particularly two very specific type of show note require a lot of work and a lot of links. And that's let's talk Apple and the security bit segment I do for you on the Nosilla cast because both of those sets of show notes are basically lists of links because they're news shows. What has happened? Here's all the links. So if you want to see what I'm talking about, I have links there to the June 2026 let's talk Apple, which was the most recent when I started writing these notes. and at the 5th of July security bits which was the most recent at some point. And because I'm doing commentary on news and a summary on news, I'm actually very picky about the format of those links because to me they need to communicate two things. Which part of the text in those show notes is my voice and which part is somebody else's voice and who am I trusting as the source for the story? Because you can't judge and use headline without knowing the organization who has told you that because >> right >> I'm not I'm going to use the word bias not as a peorative but let's use the word point of view. Every publication has a point of view and you have to factor that in. So I need people to see this is a headline from RS Technica. This is a headline from Tidbits. So if we like Adam, which we do since he's a friend of the show, we're going to treat that one way better. >> Well, actually, you would include him in your notes because he knows what he's talking about. [laughter] >> True. >> Whether or not he's a friend of the show, it's also good for that. This is a piece where I wish uh AI had this ability is very often I'll be looking at the results of some you know like a medical question it'll come back and if I look closely I go wait a minute that's a Facebook link or that's a Reddit link it's like no no no no no no no I want something I don't know in the NIH sort of you know national institute of health kind of category for that one. >> Yeah. Or the Mayo Clinic they they come up as a lot their SEO is amazing >> I'm good with Mayo Clinic. Yeah, I just but I finally had to tell it never ever ever give me a Facebook link to anything ever. That's never going to be what I want. >> Bonus tip. That's why Kaggi is great, the paid search engine, because I've just told it these sites I don't trust and they just invisible to me. Don't exist. Never darken my door. Love it. >> Okay, so my format is basically if I'm talking, it's not a link. And the links to stories consist of the headline the people who wrote the story gave it followed by an m dash followed by the domain name of the website the story came from. So tidbits.com macobserver.com ourstechnica.com etc. So an example might be lta 154 june 2026m www.letalkapple.ie And I don't include anything more apart from the domain. So I just replace that with an ellipsus because >> by the way your website is let's talk let's talktalk.ie not let's talkappleie. >> That's a really good point and you think I might know that by now. So I do my notes like you do in markdown which means that what I end up with is open square bracket the headline m dash the domain name forward slash ellipsis close the square brackets open round brackets the URL close round brackets and I used to assemble those by hand which means copy and copy the URL start the link paste the URL type the m dash type in the domain name go back to the browser Copy the URL and paste that in in the round brackets. Tedious. >> Only slightly better with a uh clipboard manager, right? >> Yeah. Yeah. Then I could copy copy paste paste. That would make it mildly less annoying. Pretty annoying. >> Mildly. >> Yeah. And then in recent times, I've started to get a little fancier with my links to make just in case this problem wasn't difficult enough to solve. I've realized that one particular source for Let's Talk Apple and there are a few similar sources sometimes. Basically, a press release is a very different animal to a news article. A press release is a company literally putting a spin on things, >> right? >> And so I've decided that press releases should get an emoji of the loud hailer thingy, whatever you call that. to very late. The megaphone, that's the one. >> And also, they don't get the domain name, they get the name of the company followed by the letters PR. >> So, when I link to an Apple story, it's the loud hailer and then it says Apple PR, which means th this is the most positive way to look at this story. There is no no nuance here at all. >> Okay. Okay. Or all nuance. >> Or all nuance. So, I would you I would basically use my tooling I'm going to describe in a sec. And then I would go in and manually put in the loud hailer and stuff. A lot of messing around, but I've been podcasting for a long time, a lot longer than this app I wrote a few weeks ago has existed. So, this is not the first time I've scratched at this itch. In fact, this is the third. This is take three. Very happy with take three. It's the best take by far. But for a very long time, I got away with a much simpler solution, which is text expander, our old friend text expander. Because not only can the replacement in text expander be a standard piece of text with placeholders, it can be the result of executing a script. And one of the languages it supports is JavaScript. So, a long, long time ago. Go on. [laughter] I'm laughing because you've actually done this four times because I'm using linkify that you wrote in Pearl. >> Oh, I forgot the Pearl one. Oh god. >> So maybe I should put a zero. >> Put a zero and then it was it was Pearl. [laughter] >> You're right. Yeah, that was before I switched over to JavaScript in >> But I'm still using it. >> Text expand. >> I I don't see any problem. Yeah, [laughter] >> I'd forgot all about that. Wow. I really have been solving this problem for a while, haven't I? >> You have. You have. I love this script. I use it daily. Mine is very simple. It's just the URL and then, you know, the shortened URL like podfeed.com and then square bracket or round brackets to get to the full URL. Use it every day. >> Yeah. Which is perfect for what I was about to say. So my text expander snippet does the domain part. So it uses well the first version used Pearl and then we use JavaScript to take the domain name the full URL sorry and strip out all the bits that aren't the domain name keep the domain name put the ellipses after it and then put the full URL after that inside the round bracket. So it makes you most of what I need for my nice links but not the headline. So I'd use my text expander snippet to get me most of it. I would use text expander's feature of moving the cursor to put the cursor where the headline has to go. >> Uhhuh. >> And then go back to the browser, copy the headline, paste it in. >> It's a lot better. >> It sped me up a lot. And I did that for a long time. But I got ever more cranky that I there has to be a way to pull the headline out. And then I discovered a JavaScript library that exists in node in npm. So, you know the way that on the actual web we in programming by stealth use jQuery to find parts of a web page using a nice dollar syntax. >> Mhm. >> That only works in the browser because you're reading the DOM. But there's a command line version. It's called Cheerio and it's inspired by jQuery and it lets you give it just raw HTML. It builds the DOM without ever showing it. It just builds it internally as a virtual web page that no one ever sees. And then you can use the jQuery syntax exactly like I know it >> to pull things out of that web page like say the headline. >> Okay. >> So I wrote an npm JavaScript which I called linkify.js. And the script basically downloaded the URL, loaded it into Cheerio, pulled out the headline, and then printed it. And so on the terminal, I would use a pipe to chain together the wonderful Mac OS command PBaste, which takes the content of your clipboard and outputs it. So I would use PBaste, chain that with my script to send the URL into the script. The script would make the full link including the headline >> and then output it where I would then pipe it into the other wonderful Mac OS command pcopy which takes input and puts it in the clipboard. So the end result was I copied the link, I ran my script and then my clipboard magically became the link. >> That sounds perfect. >> It does. But there was always an initial niggle which is that the act of all of these plumbing as I call it on the terminal inevitably put a new line character on the end of every link and I never got rid of it. I spent years >> never got rid of it. There was always a new line character I didn't want. So I'd hit back to it. [laughter] >> I never figured out a way to do that. You know, if I were to find the uni code character for back, that would have fixed it. That actually would have fixed it. God darn it. >> But what a what a hack. >> What a Yeah. Yeah. And the other thing is that whenever something would go wrong and the command would fail, the pipe would always happen. >> So I would lose my link. I would end up destroying the [laughter] link. So instead of getting the full thing, you didn't even get half of it, >> right? I got nothing. >> Oh, no. Nice. Nice. >> And then I wrote that JavaScript a long time ago before I dis before anyone discovered ES6, which means it's the really old way of writing JavaScript code with the prototype function for dealing with classes and things before the class keyword. If you look far enough back in the PBS archives into the double digits, you will find examples of this type of JavaScript code. It was horrible. You hated it. [laughter] Um, yeah, it was evil. Which meant that once I'd written that script, I was in very little humor to do anything more with it because it meant going into old icky code. Don't like that. But regardless, the script worked, so I used it for years, and it didn't cause me any problems until around about this spring when it started randomly to not work because some websites would answer my script with a 403 access denied. H I know why. AI bots were flooding the internet, eating up everyone's resources, causing website owners to get massive hosting bills. So, they started to deploy counter measures about against anything that wasn't a browser. My Little Script is technically speaking a bot. It is a piece of code fetching a web page. So they weren't trying to stop my little script, but everything they did to stop open AI and all of these guys stopped me dead in my tracks. >> Oh, and remember when the script fails, it destroys the URL, >> right? >> So that really got irksome. And I think the crisis point came when one day I just counted and a third of the links failed when I was doing a security bits. It's like, "Oh god, this isn't an edge case. This is a really serious case. I need another solution." And around about the same time, one another one of those pennies dropped. And I realized that there is something I could do if the download failed because most websites for the purpose of search engine optimization encode the headline into the URL as the slug. Oh, >> so if I could reverse the slug, I could get a headline without downloading anything. So if download fails, reverse the slug and use that instead. The slugs are often truncated or something. They're never they're never as nice as the actual headline from the actual website. But as a fallback compared to an empty clipboard, an actual link with a passible piece of text that I could manually fix if needed was a big improvement. >> Oh, yeah. Yeah. Yeah. Okay. >> So, I thought, okay, I need to go back to this code. It's very old code and I don't not only is it horrible old code using very old-fashioned JavaScript. I wrote it a decade ago. Well, actually nine years ago. I checked the commit date of the first commit. 9 years old. I don't remember how it works. How am I going to do open heart surgery on a piece of code I wrote nearly a decade ago in a flavor of JavaScript I hate that you refuse to even look at? Yeah. So, I ended up rewriting the whole thing in modern JavaScript. And then I didn't stop because I had a couple of other niggles and I had a stretch goal. So, I kept going. >> Now, wait a minute. You didn't write comments for future you to know how to uh to fix it. >> I did. And I'm glad I did because the first step zero of my big rewrite that took me a few weeks was not to add a single new feature. It was to translate the prees6 JavaScript into a JavaScript module using the latest JavaScript approach which meant I could every line of code in that old script was preserved. It was just put in a different structure and the comments made it possible for me to reassemble everything. And the act of breaking the jigsaw apart and rebuilding it in a new shape, maybe Lego is a better example, the act of taking the Lego car apart and putting it together into a nicer Lego car meant I actually understood how the code worked. So my starting point for fixing my problems was modern code that I understood and the comments from 9 years ago made it possible to get from point A to point B. So I never had a blank page. >> Oh, okay. Good. So while I was being niggly, the other niggle I had is that the mechanism for invoking the old code wasn't a terminal command, but it was these three terminal commands piped together and then you had this training new line character and it was just icky. and the mechanism. Wait. Yeah. And I didn't have a mechanism for applying my megaphone emoji and stuff like that. So, there was no when I wrote the script 9 years ago, that didn't occur to me that I would want to make an exception for press releases. So, the script didn't do that. So, >> it's not just Apple. It's anybody's press release gets the megaphone. >> Yeah. >> Okay. But I never thought of that all those years ago. So there needed to be some extra logic in the code. And if I was going to rewrite it all, well, why not stick that bit of logic in? Compared to the AI problem, that should be much easier to fix. So why not? And then I gave myself a stretch goal because one of the things I've started doing more in particularly in security bits is in the pallet cleanser section. I often like to include links to things that are actually images. The two that spring to mind are the apod, the astronomy picture of the day from NASA and of course the wonderful XKCD web comic. And so I would have to do those links manually. But if I'm downloading a web page and pulling out all the pieces using jQuery like syntax, aren't all the pieces in there? So, couldn't the script pull together all the pieces from the XKCD or whatever and build the full link, including the markdown for the image tag? Of course, it could. Just my script couldn't, >> but JavaScript can. >> So, that was my stretch goal. So, there's an example in the show notes of a link to an XKCD comic from a recent security bit segment that I did make with my new command line app by just pasting in the URL. So, you have uh separate fields like where you can add a third thing that would include the image link or or a uh an emoji. I we'll talk about how it's implemented next week because basically >> the act of making a link I have broken it into three steps and you can control each step independently and that gives flexibility that I'm kind of pleased with. >> Okay. So as I started to do this, the very first hurdle I hid was that my very old script was relying on some very old JavaScript libraries. So if I was going to modernize it, I was also going to have to modernize my dependency tree. And that reminded me given that security bits is one of the show notes I write that right now in 2026 a massive attack vector is the baddies are using AI to find bugs in commonly used libraries that everyone includes in their code or using AI to just hack the developers and inject ject vulnerabilities into the libraries everybody's using. But the end result is that you need to be really careful about managing your dependencies. And I've always mentioned it in passing, but in 2026, it needs more than in passing. We should have a look. And so that's what we're going to do for the rest of this episode or this part of >> It's sad, but you've got advice. It's sad, but I have advice. So, the reason I use NodeJS is precisely because it has this amazing infrastructure for including open-source libraries into your own code, npm, the node package manager. Like, the reality is I couldn't write half the I couldn't write any of the stuff I write from scratch, right? You you were always standing on the shoulders of giants here. That's kind of the power of programming. And it used to be the case, the innocent old days, that you could just go to npm, pick any module that looked like it might work. And the worst thing that would happen is that the module was boogie and didn't really do what you want. And then you went and you got another one. And if it solved your problem, job done. And you never thought about it again. You didn't update it. You didn't do anything. and you were fine for years and years and years. Nah, not today. That's not a good approach today. Uh we call this supply chain attacks and they are oh so fashionable at the moment among the cyber criminals. So just thinking of stuff we've talked about on security bits here are some of the ways in which this npm and all the other ones by the way this is not a node specific problem. If you use Python pi has the same problems. If you use PHP the various libraries for PHP modules have the same problems. If you use Java the Maven and all those have the same problems. Everyone has these problems, but for me, it's npm that matters because I love my JavaScript. So, they're using AI to find vulnerabilities, particularly in modules that have two properties at the same time. They've been around for a very long time. So, they're in lots and lots of people's code and the developers of those open source modules have moved on. So, the modules are not in active development. So, they're both commonly used and not being actively developed. So if you find a bug, it won't get patched quickly. >> So that's a powerful bug. >> They're also pretending to play the open-source game and submitting helpful pull requests into open-source projects, which usually do genuinely do something useful, fix a known bug, add a new feature, but they come with a little sting in the tail. They also sneak in a little bit of malicious code. And so if the maintainer is a little bit rushed or whatever, oh yay, a fix for this bug. Does it fix my bug? Oh yes it does. My test passes. Yay. Oh no. Push to production. Auga, right? Etc. There are maintainers of open source projects who run out of steam and they end up with helpful volunteers offering to take over or to become their assistant only to inject malware. We almost lost uh Open SSH that way. What is it two years ago? >> I forget. >> Yeah. a helpful person who spent was it a year basically there were Chinese agents we discovered and they spent a year pretending to help this project trying to get commit privilege. >> Yeah. Yeah. Like terrible stuff. Or you just hack the developer's account, right? You just just hack the account and stick the malware straight in. You are the de developer. Hey Presto, you're now updating the actual module. or >> more efficient. >> This is an even more efficient way is you hack the developer secretly so they don't notice. You don't do anything obvious. You don't draw attention to yourself. And you make a subtle edit to the CI/CD pipeline. So the pipeline that automatically compiles the app and releases it every time you do a push. So GitHub actions if they're using GitHub. And you sneak an extra little thing into the action. You don't break the action. you just add an extra thing that says and slip this malicious code in as well as the latest push. And so the developers keep working away and fixing bugs and releasing their code without noticing for an amount of time that their pipeline after they're finished with the code is injecting malware between them writing the code and people downloading the code. >> And that wouldn't show in their uh their differences in git. You wouldn't see that because it's part of the something like GitHub actions. It's buried in there. >> Oh, >> exactly. It's after the push. So, they push good code and they look at their code and go, "It's fine." They can ask an AI gen, "Check this code for vulnerabilities and the AI will go clean." But the finished product has been compiled and packaged, put into a zip file or whatever is done. Maybe it's just zip it up, put the documentation in it, and publish it to a web page. Well, if you add some malware then zip it up, right? What people do load is malicious. It's ah so annoying. Or you hack the actual development tools. >> There are open- source editors that are really popular like VS Code and they have a marketplace of plugins. So like browser plugins, VS Code plugins are the place where attackers have gone, oh this gets us privileged access. Why do we hack some of these? And so that's been a thing. So there are lots and lots and lots of problems. And then just in case this wasn't depressing enough, and yes, I am very much telling you all the bad news all at once so that I'm done with that and then we can go to solving the problem. But there's one more piece of bad news. Just like when I'm writing code, I need other people's code to make my code work. When people are writing that code I'm using, they're also using other people's code. Dependencies very often have dependencies which might have more dependencies and more dependencies and you end up with a branching structure. It's kind of like a tree which is why it's called your dependency tree. And so you might say npm install one module and what you actually get in your dependency tree is 5 10 20 50 in the case of some like was it webpack had 200 and something dependencies. Now, Webpack is huge. >> Webpack is massive, but >> it can be a lot. >> Which means that if any of those packages have vulnerabilities, well, you now do probably or maybe thankfully there's a bit of nuance there. So, there's a lot going on here. Now it is important to say that your dependency tree contains many many more lines of code than your code will ever access. So imagine you need a function to do some trigonometry and so you get a module that does trigonometry and the module does s cosine tan cotan all the stuff and you only ever use s cos and tan. You never use cosine, co- toss, whatever the other one is or cotan, right? You never use any of those. And there's a vulnerability in the modu in the functions you don't use. Well, the vulnerability exists in your dependency, but your code never calls it. So your code isn't really vulnerable. It's only apparently vulnerable. >> Right. Right. Right. But that requires to understand that. >> Yeah. Yeah. To know what >> that involves some nuance, right? You have >> And when you're fourth level down dependency, I don't know about you, but I don't know what all my dependencies do. >> Oh, yeah. Yeah. At the top level, you have you can make an educated guess, >> but no, when the further down the tree you go, it becomes mystery me. You're right. Uh, so your code is not actually vulnerable to every possible vulnerability, but it's going to be vulnerable to a lot of them. So you are going to have a little bit of work to do. And there isn't a simple solution because you might be tempted to say, "Oh, I'll just stop using modules." Well, not only is that really impractical, which it is, it's also less secure. No one can possibly be an expert in everything. So just because you could write your own encryption routine doesn't mean that makes your code more secure because your encryption routine is based on your rough understanding of cryptography not a cryptographers's understanding of cryptography and your little function was written by you and used by you and nobody else has never been seen by anyone. and the module that's been around for 20 years has had a lot of eyes on it and has had most of its vulnerabilities knocked out of it. Your code is less secure regardless of the fact that it's not a dependency. I have no dependencies, just a million bugs. So [laughter] that's not better. >> I'm I'm a bug. >> Right. [snorts] Exactly. I haven't imported your bugs. I've made my own. [laughter] So what this all means is that we have to do something that I know you dislike. You like rules. This calls for judgment and nuance and that's never pleasant. Well, but to be fair, having cheated and read ahead, I think you do give us some rules. You give us some guidance and and it clears out a lot of the muck. It may leave a little bit of muck that you have to do some judgment on, but overall I think you gave us some steps that are are are useful. I think this is it's gonna work. >> Yeah, I agree with you. But the end result is it can never be zero muk, right? There's always going to be some judgment here. >> You just hope it's low severity or medium severity muk. >> Precisely. The other thing is you're not alone in facing this problem. Everyone who writes code today is facing this problem. So the entire open comm community open source community is extremely aware of this problem. They live this problem every day. The people who run the parts of the supply chain that do all of this work like GitHub and npm and Microsoft running VS Code, they're aware of this problem too. So there's a lot going on to tighten the defenses. So npm and GitHub in particular have been tightening up their rules immensely. So today I can't log into GitHub without my pass key. So that makes my account much harder to hack than it used to be. In order for me to publish my new linkify tool to npm, I had to set up two factor on npm, which wasn't needed the last time I published a module five or 6 years ago. And not only do I have to have an account with multiffactor authentication, I also can't publish without entering two or there's two gates basically because you basically the way the publish process works is it's a two-step process and on both steps I have to do my multiffactor authentication. >> I cannot push an update without proving myself twice. >> That's way better, right? the bar has gone right up. >> And there's also tooling. So the actual npm tool we use to manage our dependencies, npm install, that tool has builtin features to help you manage vulnerabilities. Because one of the things that Microsoft have invested a lot of money in is that npm maintain a database of every known vulnerability in every npm module. >> That's a lot of vulnerabilities. >> So there's humans who work for npm isn't just a library people shove into. It's it's people behind it. >> Yeah. Like every internet site, it takes some humans to make it go. I don't think it's a site. I think of it as a repo. >> Yeah. Well, there's a lot of human beings making that joke. >> It's like an a store. >> Yeah. And a store has employees whether you're selling biscuits or selling stuff online. Yeah. >> You know, it's quite a big It's quite a big thing actually. Um and it used to be run by the same people who run Noode.js JS and they were finding it strenuous to run. So Microsoft have actually taken it over. So no.js still does Node.js but Microsoft now pay the bill for npm. >> Ah okay that's >> which is good because it means the GitHub people are heavily involved in securing npm and the GitHub people are very good at their job. >> Yeah that's very interesting. >> Yeah, that wasn't even in my show notes. This I this is why I love recording with you. We go on these little side trails. >> So the point is >> okay. >> Yeah, there's tooling. >> So for us, what matters is npm. >> So they have this database of all the known vulnerabilities and they don't just know that there's a vulnerability in a certain module. They know the first version that's vulnerable and the last version that's vulnerable. And because npm very wisely have been forcing developers to use sever as the versioning system. So we've talked about sever >> that's the thing where you've got like the major version dot the minor version dot and then I was goofing around version like three there's three numbers >> patch version. Yeah version is called patch major minor patch. >> Okay. >> Yeah. And the way the rules work is >> the patch version doesn't add or remove functionality. It only fixes bugs. The minor version is allowed to add functionality but not break anything. And major versions are allowed to have breaking changes. So that's really important because it means that npm can know if an update is safe or not based on version numbers. You have version 3 something point something and the bug is fixed in version 3 something else something else that's a safe update for you. >> Oh right. >> But if you had version three and the bug wasn't fixed till version 4 not safe for you. Manual intervention needed. NPM alone can't help. And Sever is great for that because it lets npm's tools really tell you a lot of useful information. Yeah. >> And it lets them shout at you, auga, breaking change, breaking change. >> I remember the Senver stuff really well because it it just made me happy that it was so controlled and specific and knowing that four versus three was a breaking change. And I mean, maybe you're going to have to accept that breaking change and go fix the things that broke, but at least you know what's going to happen before you do it. >> Yeah. Yeah. Exactly. So the other thing then is we have this massive database of vulnerabilities and so the question is well how's that database structured and they there are four levels of severity low moderate high and critical and they have a very cyber security- like description of what they mean. So I I asked my favorite privacy respecting AI assistant Lumo to give us a summary for human beings. So this is how Lumo summarized it. A low severity issue, minor security issues that are unlikely to be exploitable in most environments or require very specific conditions to trigger. So they're darn hard to do anything with, if not impossible. Moderate vulnerabilities could lead to security issues, but typically require a particular Yeah, sorry. a particular setup or user interaction to exploit. So, they're more likely to trip you up, but still not that likely. You, you know, the lines in the holes in the Swiss cheese really have to line up quite badly. High severity then is a serious vulnerability that poses a real risk and are often directly exploitable, potentially leading to data exposure or code execution. They are not good words. Neither of those are good words. >> No. >> And critical is basically everything worse than that. So the most severe issues typically allowing remote code execution, authentication bypass or other high impact exploitation. This will hurt. >> Remote being the word doing a lot of heavy lifting in that. Not just code execution, but remote code execution. >> Yeah. That's the really scary stuff. >> Yeah. And you can access all that functionality with a very simple command npm audit. You just go into in your library. >> Yeah. So you're working on a Node.js project. So you have a folder that contains your package.json file. And in that folder, if you type npm audit, it will give you a report. And what it's doing is it's reading your package.json file and your package.lock.json JSON and comparing what you have to the database of problems and doing all that math and figuring out what is that funable, is that funable? And it follows the whole dependency tree all the way down. It does all of that math and then it gives you a summary and says there are x amount low, medium, low, moderate, high, and critical vulnerabilities. And it tells you which ones it can fix without breaking changes. So I tried this on my uh time matter application for the web. Not my not my app app but the other one and you've done a lot Barton. It's a whole lot. >> It's probably been a while. >> Yeah. >> It's been a while since you ran npm install. >> Yeah. >> Yeah. So well you'll be able to use this little process here then. >> Mhm. So the first step to dealing with your problems if if npm audit shows you a lot of stuff, my first step is not to immediately go to npm audit fix, which we will get to, >> but instead to take a step back and ask myself the question, how many of my modules can I patch safely? Because not only will that deal with my security problems, I might get some bugs fixed, too. Oh, so actually my code might become a little bit more reliable. So I should just do this anyway. So the first command that's great to remember is npm outdated. That will look at your package.json and tell you everything you've explicitly added. So it's not going the whole way down the tree. It's just looking at your dependencies and telling you of your dependencies how many of these have an update. And it doesn't tell you two versions. You might think current version, latest, but it actually tells you current, wanted, and latest. >> Wanted. >> Wanted. Yeah, wanted is the version you can go to without crossing a major version. So, it would fix some stuff. You don't know whether it's going to fix it fix it, but it's it's better than what you got. >> Correct. So basically if you use npm install to add a dependency, it uses special syntax to say allow this module to update over minor and patch versions only. And that's the default. So you're if you look at the actual content of package.json, the version number has some special characters in it. And those special characters tell npm don't do major versions automatically. If you manually write your own package.json, you can allow silent major version updates. You'd be an idiot, but it is possible. So, the word they use is wanted because the wanted refers to what your package.json says is allowed. By default, that says safe. And so, unless you've done something weird, wanted means safe. So current wanted latest and so if you see some of mine uh in that example. Yeah. Okay. So Webpack- CLI in my example output the current is 702 wanted is 721. Latest is also 721 but latest could have been 8 something right? But that means that I can safely update that package because I'll go from 70 to 72 safe. And the way you update is you go npm update name of package. So npm outdated to give you your list npm update everything you're happy to update safely. And you don't have to tell it the version number because it won't cross the major version without you doing a minus force or something, right? you need to do effort, >> okay, >> to break things. So, it's great, right? By default, everything's just safe. Just do it. Yay. >> So, you put in the show notes npm update and then the package name. I accidentally hit enter after writing mpm update and it just did all the ones it could. >> Yeah, it will do that. I like >> I wanted to do it surgically, but >> yeah. >> Oh well. >> Yeah, you're right. That is safe. And because it won't cross the major versions, that's >> not as interesting. >> Yeah, I prefer to be a little more careful. I like to know what I'm doing. >> Um, so after you've done that, you could run npm audit again. And the chances are the number has gotten less. Chances are also the number is not zero because >> mine got way shorter though. >> Good. >> Way shorter. I mean, it was scrolling pages. I was like, wait a minute. Why is my scroll bar so small? >> Yeah. Yeah. But remember, it's been a long time since you did the npm install, and you probably haven't done an npm update in 2 or 3 years, >> right? Right. >> Or more. How how long have you been tinkering with that app? Because that app has been something you've wanted to do for a long time, and it's existed as a version for a very long time. >> Well, my my YAML file was 2023. >> There you go. At least three years. >> Yeah. actually the the uh the actual math engine was 2023, October of 2023. So that that's the piece that actually makes it go. So it's somewhere before that. >> Yeah. So that's three years worth of updates you just got all at once there. Three years worth of safe updates. >> It's six months worth of AI. [laughter] >> That too. >> Yeah. God knows how many dependencies were added by your AI friend cuz they don't like to reinvent the wheel. >> And why would they, right? They like to just No. use a dependency. Yay. Okay, so at this point you will have fixed everything. Sorry, I'm getting one step ahead of myself. So the npm audit is going to look deeper down and it's going to offer you, hey, do you want me to fix more of this for you automatically with the wonderful command npm audit fix? And what that's doing is it's applying the process we just applied to at every level of the dependency tree. So every one of your dependencies has a package.json file that lists its rules for which can and can't be updated automatically. So npm audit fix applies those rules all the way through your de dependency tree and it updates everything that every package.json JSON file in your dependency tree says it's safe. It fixes all of them. And that will cut down way more vulnerabilities. >> Right. Right. Okay. And after you've done the npm audit fix and then you run another npm audit to see what's left. Now is where the human judgment comes in because you have now done everything that it is safe to do unthinkingly. You could write a script that says npm update npm audit fix safe to do >> and then the last thing the script would do is npm audit and then say over to you >> because that's the output you cannot automate you have >> and if you see high or or severe it goes >> it should yeah >> o script tell finder window [laughter] how >> make yeah Um, I did know there there is a command to make a Mac make all of the standard system bongs because I used to stick that on some of my text expander snippets to let me know when they were done. I thought it was an OA script. No, I think yeah, use OAS script to call something which says like alert and then one of them is called frog I think and it goes [laughter] and one of them is called bell like a really nice ding. I use that one a lot. >> Anyway, you're getting lost here. Okay, so we now get to the human judgment bit and this is where context matters immensely. And I'm going to give you what I'm going to describe what I do and I'm going to describe my context because what I do is not advice at all. It's just an example of decisions I have made that you may think are reasonable. But not only is it not intended as advice, even if it was, it would be advice that only applies to people in exactly my situation. >> Okay. >> And it's not advice because I don't give advice. >> You mean so it's Let me make sure I understand. This is advice for people who are in charge of say the uh gas line infrastructure for a major country, right? That's what you mean. the opposite of those. >> Yeah. [laughter] >> Okay. >> But actually, I can give very easy advice for them. You need to get to zero. >> You're going to have a lot of work. >> Unplug that Ethernet cable [laughter] >> or that or that. Yeah, that works too. >> In case the computer in concrete, throw in the river. Go home. Okay. So, these days I only code for my own personal use. So my level of caution is really different to if I were coding professionally. Also my focus these days is on terminal commands I run locally. So that means JavaScript running on a specific computer which means it's not visible from the rest of the world and it can't touch anything outside of this computer. or I'm doing clientside JavaScript in the browser which means there is no server. So that means that none of my vulnerabilities can take over someone's server. It's my code running in the browser I am using. So if you take my code and you run it on your terminal, you could use it to hack yourself. >> Okay. >> Okay. I'm not going to hack myself. If you want to have at it, if you use some of my client side JavaScript code on on your web page, then all you can do is attack yourself through your own browser >> because it's stuck in the browser. >> Yeah. Yeah. >> So, the blast radius here is very small. I'm still going to be careful because there are still things that even in that very very constrained context could do could be really quite bad, but it's a lot less than if I was writing code that was going to be used on a server because serverside code can be triggered from somewhere halfway across the planet and attack me. And that's bad because that's not me attacking me. That's some random stranger attacking me. or if I publish the code, some random stranger attacking some other random stranger and I've now created chaos across planet Earth if my code is popular and that's different. So context really matters. So given my context, my approach is that I generally ignore the low severity ones. If I can patch them by doing my npm update or my npm audit fix, I do because >> you're not gonna risk a breaking change on a low priority >> or low severity. >> A moderate one, I will actually take the time to read the description, not just the title. I'll read the one sentence, right? It is a one-s sentence description. So when I say I'll take the time, it's not a lot of time, but I will do it. I'll have a quick read. And that means that my attention is left over for the criticals and the highs and I can put my energy into actually triaging those thinking about them carefully. Um there's another very important piece of context here. There are two types of dependency. There are regular dependencies. So when you do npm install, your dependency gets added as a regular dependency which you will find in package.json under the dependencies array and the running code uses those dependencies which means that those dependencies are in play every single time the code runs whether it's me running the code or someone else running the code. So those are they're just always in play. Developer dependencies on the other hand are dependencies you install with npm install minus - save minus dev and they show up under a different array in package.json. They're the dev dependencies array. Those dependencies are not used when you execute the code. They are only used when you build the code. So when you do an npm run build, your developer dependencies are in play. But if you just execute the final code, those dependencies do not get executed. So a developer dependency has way way way less ability to do harm and attacks way fewer people than a regular dependency. Because when I publish my code, my linkify app to npm, the download people get does not contain my developer dependencies. >> So, >> okay, >> it's only someone who clones [snorts] the git repo gets the developer dependencies. >> That makes sense. I I have I have a perfect example of this, Bart. I've run all this on my on my little code and uh one of the things that is a uh high severity is serialize- JavaScript. >> Well, I ran a command that you haven't told us yet how to do, but I'm going to just sneak ahead a little bit. And it's in copy-webpack-plugin, which I without even looking knew from the way you just described it that that's a dev dependency because Webpack is how I'm packaging it and then shoving it up. That never ends up being in something somebody would download. So sure enough, I look in my uh package.json, there it is. Dev dependencies. >> Yeah. So that's much less scary. >> Yeah. >> Than a real depend than a regular dependency. They're all real, but regular and dev, >> right? >> Yeah. Perfect. So dev dependencies are less dangerous, but they're not safe. You do have to read the piece of text describing what they do because sometimes they can do things you really don't want. Some examples that come to mind is they could corrupt your build process if they inject malware into your built app. Okay, they only run when you build your app, but they've now made your built code vulnerable. So those kind of vulnerabilities, they will be critical by the way, right? They're not going to be listed as low, right? How would they how would they get to it if it's only on my development machine? >> Okay, so you have downloaded a bad version of some plugin for Webpack because Webpack can take plugins too, right? >> Yeah. >> Well, every single app that bad plugin builds contains a piece of malware. >> Okay. >> So, you are now spreading someone else's malware >> by publishing your perfectly safe code. They're corrupting your code, injecting their poison into your perfectly good app. But like I say, they will be listed as critical and they will say something really scary in the description. They're you're not going to go, I wonder if, right? They'll they'll be a guying at you very loudly. >> Well, this one was and it says vulnerable to remote code execution via regular expressions, >> right? So that would mean that someone had to edit your code to add a specially crafted regular expression to attack you while you're executing. >> Okay. >> Your code. >> Got building your code. >> Now you're getting down to those reduced probabilities that you were talking about. >> Exactly. So the second the second type of developer dependency that I worry about a lot that I do see is dependencies designed to find secrets on your computer and sneak them out to the baddies. So, the baddies have compromised something you're using to build your code, and it doesn't stop your code building, but what it does is had a good look around your computer for private keys and anything else of value, and then it secretly sends those to the attackers in the background. Your code is fine. What you publish is fine. Your private key for something or other is gone or your crypto wallet has left the building or your one password vault has been emptied or something like that, right? Bad stuff. So, you don't want the word secrets is always bad when you see that in a description, but they're not going to be medium or low. They're going to be critical to be honest. And the other type are just people being mean. Uh wipers. There have been a few cases of cranky developers. So developers who have decided the open-source world stinks. I'm out of here. I'm going to intentionally poison my legitimate plug-in as I leave the building by having it delete everything on the developers computers. >> That has happened once that I know of. And it wasn't long until npm deleted the the module from the repository to stop it being spread. But it did happen once. So, you know, >> that's basically like somebody pooping on their boss's desk as they leave, except you're pooping on everybody else's desk as you leave. >> Yes. Yes. Yes. Yes. Yes. Exactly. It's the entire open source community that they're cranky at. So, that's the desk they poop on. >> I'm glad you brought up that example. I was thinking it but I didn't think [laughter] I could say it out loud. I have very strict uh you know language things on this show. So to try to put some structure on this, let's have a look at what actually happened when I was looking at my dependencies for linkify while writing these show notes. So I ran uh let me see. So I did an npm okay hang on let me read my own show notes to remind myself. So I'd done my npm audit minus minus fix and I'd done my basic npm update. So what's left here is the stuff where I have to apply my judgment, right? So I didn't bother showing you the easy stuff, right? Just what's left with the judgment. >> So when I run the npm audit, I get some output that's in the show notes. And at first glance, I was a little confused because it says I have two vulnerabilities, but it only mentions one module. That module is called Showdown and it's never been fixed. The vulnerability is in the version star. So the vulnerability is in all versions of Showdown. It's moderate vulnerable to regular expression denial of service in link or anchor. In other words, oh pars, sorry, link or anchor parsing. Now I will explain what that means in a moment. Um, and then it mentions it again. It's like, why is it here twice? I got a bit confused. So, this is a good opportunity to explain a very useful npm command. npmls. It will show you where in your dependency tree a module exists. And once it showed me that, the penny dropped. I was like, "Oh, okay." >> So, show because there was something above it. >> Yeah. Oh, >> so showdown is vulnerable. One, showdown is required for clean JS doc theme, the theme I use for my documentation. That means clean js doc theme is vulnerable two. >> That's why it counts two. >> Okay. >> Now [clears throat] that immediately allowed me to implement my brain say hang on a sec. So this is a denial of service. In other words, exhaustion of CPU or RAM when parsing a link or an anchor in my documentation. So if I wrote an intentionally malicious regular expression in my own JS doc comments, I could eat my CPU or my RAM. [laughter] >> Okay, fine. Fine. Don't care. Not a problem. Because I >> if you're doing that, you're having a psychotic uh episode where you've uh a psychotic break. [laughter] >> Exactly. I'm not putting anyone else at risk. I'm not putting me at risk. Fine. We can live with this. Now, I did look, could I fix this? Well, there is no fix for showdown, which made me worry. So, either there's been a big update to JS Doc to remove the dependency completely, or I'm going to be out of luck. And I did a look and npm outdated shows that oh yeah there is a new version of cleanjs doc theme. I'm on 433 and the fix or the latest version is 57. That's one of those breaking changes changes. So I out of interest it's like maybe it's not a big breaking change. So, I wandered over to the release notes and read and the first sentence told me all I needed to know. Cleanjs doc theme version 5 is a ground up rewrite. [laughter] They started from scratch. >> So, there'd be a lot of backtracking to figure out what went wrong. >> They also had a list of every breaking change. It was a long list. >> Oh, wow. >> Okay. So, I did actually make myself an GitHub issue to remind myself that at some stage when I have time, I do actually want to do this update because the new version is better. >> Right. >> But I'm going to do it at my pace when I have the time. >> Yeah. >> Because this is not a problem that needs emergency fixing. >> Okay. And so that basically is an example of me triaging a real bug that I was seeing while I was writing these notes. So I have some general guidance because you can to some extent nip this problem in the butt. You can make it less likely that you have problems which means you have less to fix later. And you do that by well there's two pieces of advice. install exactly as many dependencies as you need and no more. If you need help with something you can't do yourself, you need a dependency and you should install it because otherwise you're going to write some hacky rubbish. But if you're on a whim, oh, that might be cool. No, it's like the same advice I give with WordPress plugin. >> That's what I was just going to say. It'd be just like that. But but you end up with dependencies maybe you're not not needing. You know, I needed to install Webpack. Well, who knows if I'm using this this dependency. Maybe that is Yeah, that's that's an adjacent sort of I guess a piece of advice that as your codebase ages and you remove features, take the time to figure out if you've also removed a need for a dependency. >> Okay. Yeah, it's it's a nice sort of a piece of ancillary advice because yeah, the exact amount you need may vary if the code has a long life. And this makes me think about other languages like if you're writing in Swift, >> that's not all open- sourcy stuff, right? I mean, that's not an npm isn't watching that stuff for you for libraries and and dependencies. >> Maybe there's a open source. Remember, Swift is also released as open source um and is available on Linux and Windows. >> And there are certainly git repositories with pieces of Swift code other people use, >> but I have no idea what the tooling is. >> If there's an npm equivalent that watches for you and Yeah. >> Yeah. I have genuinely no idea. I know that Python people use pi and the pi repo have been very busy applying tools and stuff just like npm and github have been >> right >> so if it's a major thingy for managing dependencies they are thinking about this because they all have this problem >> right >> I just know the npm one really well >> yeah well and this is this is good I mean this is really well done >> excellent so After you decide to have exactly as many as you need and no more, the next thing is that you usually have more than one choice. If you have a problem to be solved, npm is big enough that you don't usually have one module and one module only that could do it. There's probably a few to pick from. So you should basically look for I I sort of say look for good signs and bad signs and find the module with the most good and the fewest bad because none of them are going to be perfect but you know maximize one minimize the other. >> So what I look for is just start off on the npm page for the module. Is it actually a healthy looking npm page? Does it have a high number of weekly downloads? like that's right up there in the top right corner. The easiest thing to look at. If it says 3 million, it's got a lot of eyes on it. >> If there's a vulnerability, it will be found quickly and patched quickly. So, that that's a good sign. uh if someone has taken the effort to write a nice description that isn't just some copy paste or two sentences but is actually a nice about page that shows care and attention which implies it's probably good code. So that's actually a very positive sign. I really like when my npm packages link to a GitHub repository so I can see the code and so I can see the code's history because being able to look at the GitHub repository gives me a whole extra view of how active the development community is. Is there one GitHub contributor or many GitHub contributors? Right. I love it when there's GitHub to go look at. >> Yeah. Yeah. By the way, another thing you can look for, sorry, I was just poking around in npm looking at that, is one of the things you can do is you can do a an analyze security with socket and it'll tell you supply chain security, vulnerability, quality, maintenance, and license. And you get a little meter from, you know, red, yellow, green. >> So, that's kind of cool. >> I don't know when that showed up, but that certainly wasn't there six months ago. Well, maybe a year ago. Interesting. A fresh pair of eyes sees new things because I'm so used to the npm page. I just look for the things I know are there. I never even noticed that. >> Yeah, >> that's cool. >> Um, a recent release history. So, you may see that in npm or you may see that in GitHub, but either way, if there's a nice history of changes and stuff, this is under active development. So, if there is some sort of a problem, it's going to be picked up. That's a a good sign. I generally find that if there's decent documentation, the module is run by people who care. The module, if it's well documented, it's probably wellmaintained. If it has little or no documentation, it was probably something someone wrote in an afternoon once, published, and then saw it off. >> Yeah. >> So, good docs are just a good sign. And to be honest, not only are they a good sign from a security point of view, they're also a good protection for your own sanity. You're going to be using this code. Good ducks. You're going to want those. So, that kind of that's a win-win, that one. >> I might put that even higher than uh than downloads [laughter] if it's useful. >> Yeah. I mean, I guess if you're ranking it for usefulness, it would definitely go higher. But when I have my cyber security hat on, I think the downloads probably should be higher. probably >> and then the other one is if after all of this it is actually good to minimize the dependencies in your tree. The smaller your tree is the less problem branches there might be. So every npm package lists its number of dependencies in this little strip of statistics across the top. And some of them actually make a point in their description of saying zero dependencies and they put it in bold and in big writing. And that's because minimizing your dependency tree is a thing these days. >> Yeah. >> And so that's not a bad metric at all. And one of the things I did which I'll talk about in part two or part B or whatever we call it um is there were a couple of points where I had been using a module to do a specific task and the module I had been using wasn't the only choice and I actually switched. And I'll tell you the exact one. There's um there's a module for doing colored terminal output. >> Oh, >> and I used to use one called chalk which has loads and loads of downloads, but it also has a history of some really nasty vulnerabilities >> and it has a lot of dependencies. And there was one called CL, which is the Flemish word for color, which I just loved cuz hey, Flemish, hello you up. Um and it has zero dependencies and it was a drop-in replacement for chalk. They duplicated the API. >> Oh, it's >> like, well, I don't even have to update my code. I'll just take this instead of this. Yo, >> dependency tree shrunk. Win. >> Yeah. Excellent. >> This is uh really logical. >> Yay. Good. The like I said, this is iteration eight or nine of these show notes. Sometimes what you see is iteration three or four. This is iteration nine. And this took me a while to come together, but I'm glad you enjoyed it. And this is where we're going to draw a line under it for today. And the next thing we're going to start is designing Linkifier because do you know something? I had a lot of fun solving that simple problem. I need some links with a nice headline. Because actually finding the right headline in a web page in a way that works everywhere is not nearly as simple as I've led you to believe so far in this installment. >> Hey, uh I'm going to take a chance on this being a correction you want to hear before we finish. I don't think it's called Linkifier. >> Isn't it called Linkify? >> Yes and no. [laughter] Let's leave them on that cliffhanger. It's not >> I should explain this actually. So the command line app I think I called linkify but that uses JavaScript classes that you can use without ever using my command line app because all of the brains is in a module an ESX module called linkifier. So, Linkifier is the generic library that does the work [snorts] and linkify is the terminal app that uses Linkifier to do its work. >> Okay. Okay. Good. 100 bucks says that I correct you again when we come back in in four weeks. [laughter] >> Actually, yeah, it's four weeks. That could be a very easy bucks. >> Well, now I'm I'm betting I will ask you again. So, >> Oh, and I'm not taking that bet. You can keep your [laughter] money or I'm going to keep my money. >> Who would have thought we would have giggled and enjoyed ourselves this much talking about vulnerabilities in in uh dependency libraries? That's we're an odd lot here. But I had fun. Yay. Hey, why do we keep doing security bits? All the news is bad and we still have a great time and the listeners still love it. So, it is possible. Exactly. Okay. Well, until next time, folks. Lots and lots of happy competing. >> If you learn as much from Bart each week as I do, I'd like you to go over to let's-ashtalk.ie and press one of the buttons over there to help support him. You know, during most programming by stealth episodes, Bart does 98% of the work while I'm just the stoogge who listens and asks the dumb questions. If you go over to let'stalk.ie, you can support him on Patreon, you can donate via PayPal, or you can use one of his referral links. I really hope you go over there and help him out. You can join other programming by stealth students in our Slack community at podgy.com/slack. You can contact [music] me directly at allisonpod.com. And I encourage you to check out all of the shows we do over at podfeat.com. Thanks for listening and stay subscribed. [music]