nf-core/bytesize: nf-core newsletter

Hi, Maxim here. I'm the senior bite size. So today Phil working at Secura will be presenting us a new feature in NF core which is the newsletter. So as usual you'll be able to unmute yourself at the end if of the talk if you have like any question. Or you can also ask them on Slack later later on. So now to you Phil. >> Thank you very much. Thank you for the intro Maxim. For anyone who doesn't know me, my name is Phil Yews. I work at Secura as project manager for Nextflow and helped co-found NF core back back in the early days. This is a bit of a excuse me, a surprise bite size talk. I think Maxim suggested it about 40 minutes ago. So I haven't prepared a great deal but it is all fresh in my mind on the topic. So hopefully I can just talk you through a little bit about what I built, why, how you can use it and then a little bit on the background of how it works under the hood for those who are interested. So basically the core problem I'm trying to solve here is that the NF core community has grown a lot in recent years. We are I think over 14,000 people in Slack now. We have hundreds of pipelines and anyone who monitors the new pipelines channel for example will see that we're getting new pipelines added every week almost. New RFCs, proposals to asking for comments on changing standards. There's a lot of like these bite size talks and hackathons and stuff happening all the time and we're pushing out blog posts with updates. It's just a lot of stuff. And it's really hard to keep up track with all of these different new pipelines, new releases of pipelines across the whole community even if you spend all day every day on Slack like I do and like Maxim does. So for a while I've kind of been thinking about it would be nice to have a newsletter. We started a a newsletter with Sakera with kind of next layer updates and and community updates there and it's it's gone down really really well. So, for the past year or more I've been thinking about trying to do something for NFCore. But, we're all volunteers here and um you know, time is precious. And so, the question really was how do we do that in a way that doesn't require someone to spend a bunch of time doing it every month because it's kind of a dull [clears throat] task and and it's just going to fall by the wayside eventually. So, I wanted to see if I could automate it. So, um basically what I did was I built a chunk of website which is and we have all this new content in the website already basically. Um so, it's a new page or section of pages in the website which aggregates all of this information. And then I built a new repo which takes basically scrapes a specific web page on a specific day of the month and sends it out to an email list. And I'll talk through a little bit about how it looks now. Let me share my screen. >> [clears throat] >> Hopefully everyone can see the NFCore website. So, um >> [clears throat] >> just move that over here so I can look at it. So, if you go up to the top now, you'll see a couple of things there. You can see I'm giving a bite-size talk right now. Um >> [laughter] >> and then there there's a new link here that says um newsletter underneath community. Uh this is kind of the hub of everything and there's a couple of things here. Firstly, you can go back and see historical newsletters. Obviously, we haven't actually been sending them since 2018, but all the content has been building up that whole time and so we can retrospectively build a newsletter for all past dates. So, we figure why why not? Um you can click through if we look at the most recent one um which hasn't gone out. We'll be a new one in a week uh for June. Um you can see it aggregates basically all of the stuff that happened in the previous month. So, this would have gone out on 1st of June 2026 and there was a new blog post that was at the end of May. So, it gets highlighted at the top. It also highlights all of the new pipeline releases that happened in the previous month. You can see that some pipelines had multiple releases cuz of a patch or whatever and they're all kind of aggregated together. So, you can see at a glimpse all of the major pipeline updates. We have upcoming events. So, you can see what to what to join in. You've got bite-size or our newsletter. >> [laughter] >> Um and then at the bottom is kind of some lesser of less obvious things. Um there's a recap of the last 2 months previous of blog posts and events just in case you missed it just as a reminder. Um and then at the bottom we have brand new activity for nascent pipelines which have not yet been built. So, we have pipeline proposals which have been opened. Um this one's been accepted already. This one's brand new. And then these are the ones pipelines which have moved on to the next step after that um and they've got a new GitHub repository created. So, you can see the bleeding edge of new pipelines and and what's happening there which is otherwise quite difficult to see. Um and we also can see here new RFCs with request for comments about changes in the community. And then yeah, you can skip back through basically any month and see see the kind of wrap up for that month. This one's got Maxime giving a thumbs up to a pigeon. Particular highlight of mine. Um so, this works well as a web page. Obviously, email clients are a bit rubbish at handling HTML so, you can't just send a whole kind of web page. So, we have to have a special HTML version for email clients which basically takes away all the nice things from web pages. So, like all the CSS is in line. Um images are all kind of absolute URLs or bunch of stuff like that is done under the hood so, that so that it hopefully renders okay um in your uh your email clients. It's also mobile friendly. So if I view this on a mobile phone it will kind of collapse into a single column. Hopefully it looks okay. It's email clients are famously difficult to work with. So I mean I've checked this in Gmail and a couple of others and it it looks good. Um but obviously if you see some horrible thing then let us know and hopefully we can try and fix it. It should also if I switch to light mode we'll see if it crashes my computer. There you go. It should also work in light mode and be be responsive. Uh what else? Um there's a couple of other flavors. If you still like your RSS feeds there's an RSS feed you can subscribe to. This is useful for dumping into I don't know Slack for example or other kind of like read aggregators. Um there's a markdown version which doesn't really kind of show up anywhere. I'm curious if anyone finds this useful either to again I don't know maybe you want to read it in your terminal or give it to your LLM agent or whatever but there it is in markdown in case that's useful. And then there's a super simple stupid HTML version that has zero styling which is basically for internal use um with the LinkedIn uh newsletter which I'll I'll mention in a second. Um Okay. That's it really apart from that you'll see a couple of new bits just asking to say you can sign up and there's like a sign up page here tells a little bit about how it works and stuff. And uh I've also written a blog post which has just gone out describing all the same stuff in blog post format um and kind of describing a little bit about what you can expect. So I'm really on the kind of features front I'm really hopeful that this is useful to people um and is gives a convenient way for people to keep up to date with what's going on in community. Um one more thing and the LinkedIn um what again going on experience with Sekira we saw that their email newsletter is moderately popular. Um but the LinkedIn newsletter was massively popular. And basically, I think a lot of people these days get so much spam and so much stuff in their inbox that newsletters are a bit kind of stale and a bit old. Um but people do spend a lot of time on LinkedIn and the interface here is quite nice. Um and so we're going to try that as well. So we will mirror the same content into LinkedIn in a in a newsletter format. So you'll hopefully still see these pop up as well. A big thank you to Lizzie who works in marketing at Sekera who is going to do this for us cuz it's a manual process, unfortunately. I spent several months getting my hands on a LinkedIn API key which took ages only to find out that newsletters cannot be automated via API. Almost everything else in LinkedIn can be. But um the newsletters they have this kind of fancy wizzy wig editor and it can't be touched with an API. So you have to manually type everything in here. Uh which is why we have this uh this um this kind of simple version. There we've we've plain XML so that she can easily copy and paste that into LinkedIn for us. So uh hopefully that works well and is another way to subscribe. Okay. Um that's basically how all the the clever stuff for the the newsletter works. It's just a webpage. Um the only remaining part of of how this works is talking a little bit about um the back end that actually sends emails. Um and this I've basically kind of it's quite easy because Amazon, thankfully, has a service which just handles emails. So um it's called a simple email email service, SES. Uh one of the very many numbers of AWS services and it kind of handles everything you need for a for a newsletter. So um we have credits provided to us from for NF Core from AWS which help with many things we do and they will now also help with sending our newsletter. So um so when you hit subscribe, in fact, I can do it. I don't think I've done it for my email address so early testing. I type in my email address here or in any of these subscribe boxes and hit subscribe. I've just got an email which you'll have to take my word for cuz it's not going to be in the screen share. But I get a nice kind of confirmation email so it will be an NF-Core newsletter. I hit confirm and then that takes me to a web page with a really ugly URL but you know, it works. That confirms that my email address and now I'm subscribed. And that's really all there is to it. Everything is stored on AWS. They have their own member management service. There's unsubscribe links in every email and also in the email headers so you get those nice kind of buttons at the top of Gmail saying do you want to unsubscribe? And we don't have to worry about managing all of that infrastructure. I say that there is actually a new GitHub repo I've created called newsletter which has all the infrastructure. It basically has the has all the architecture for this so I'm not going to go into too much details cuz it's kind of boring. Basically there is a few AWS services which do these things. So here's here's SES service in the middle which actually maintains the contact list and and sends the emails. And these are the different flows. And so it's like a lambda and a few other things to do with making the subscription work. But this is just a pure infra repo that basically just needs to sit there. Has all the infrastructure as code scripts in here. And and should never really ever change. The website has all the dynamic content. Right. That was staggering detail for any feature. Sorry if I bored you all. But I hopefully this is a useful thing. Tell all your friends. Let's see how many people we can get signed up and hopefully makes everyone's lives a little bit easier in staying up-to-date with NF-Core. Let me check the questions. >> I know thank you very much. That was like amazing. I'm super happy about like learning more about all of the technical side. A quick question, do we know about how many people have subscribed to the newsletter? >> It's a follow-up task for me which I have not yet done is to expose this information. I want to stick this onto the end of course that's page. Exactly. Um so I haven't done that yet. It should be possible. Um I haven't even thought about how to do it yet, but the SES service basically has these statistics about number of people signed up and stuff. So it'd be nice to have a little graph kind of showing the number of subscribers. >> And you assume that we could have that. We don't have access to the email so we don't know who subscribed it who subscribed, but we know how many people. >> Yeah, exactly. I mean technically as an admin I can go in and I can see who subscribed, but I don't think there's any need for that. Um we definitely don't want to obviously expose that publicly. So just a count is fine. >> Well, that's that's perfect. Yeah, no. I think you covered like every question I could possibly have. Like you went very deep into the the technical details and so on. Uh uh Yeah, I see Matthias was mentioning of course like the statistics. So I think we are good. Does anyone else has any questions that feel I don't covered already or are we good to go? >> I have a question for all of you. Have we have I managed to cover all of the different content types? So we have blog posts, advisories, events, pipelines, proposals. I think those are the five content types that it scrapes, but is there anything else that we could have in an email? >> Matthias is mentioning the components that are missing. And Kevin just appeared online. Kevin, do you have any questions? >> Uh no, just suggestion also including the maintenance uh [snorts] news updates. >> The maintainers have they published blog posts um with notes. >> Exactly as I included especially in the newsletter. >> Yeah, because all blog posts are included in the newsletter. So, it will it will show up as a blog post. >> That I might have missed. Otherwise, it would be nice. I mean, what would be cool is if you could select what you would actually like to get, right? >> Uh it was it was it was looking like I was going to finish maybe one day. But, um there's another [clears throat] level of complexity. >> right? Just thinking. Never mind. >> [laughter] >> That would be a fun That would be a fun feature, but honestly, I don't think that's going to happen anytime soon. It would be a a pain. >> Yeah, we have a suggestion on on the chat. The modules. Do you want Do you want to make like any the new modules because we have new modules coming like every day. >> I Yeah. Sorry, I didn't mean to cut you off. Um yeah, I I did think about that when I first started. Um I think I even looked into it. It was a few months ago now, so I forget. If I remember correctly, my main problem with that is there's just a a lot of them. Um and so, I wasn't sure how useful it was if you just have a list of like 50 new modules or 100 new modules every month. Um Well, I I don't know how many new modules we have any every month, but we have a lot. Um so, kind of I felt like pipelines was a manageable list. It's kind of, you know, 10 or 15 every month. But, when you add my modules, it was just this kind of huge list. So, I mean, I could have a play around with it again if people really really want it. Um >> No, for me I think I would like more the modules to be like just in the stats to see how many new modules we have, how often do we update the module, and so on. Like more I think it would be more interesting just as statistics as just having like to know which module are coming. Okay. Yeah, Nicolas is suggesting like a counter. >> Yeah, exactly. Matthias was already saying can have newsletter stats in the newsletter bit. I mean, I guess in theory we could have all all kinds of stats in the newsletter like current number of pipelines, current number of modules, maybe some plots and stuff. Um that that would be doable. We could do that. >> Um I think I have just one question. Why did you choose the region in in Europe? Or was there anything else? >> No strong reason. Nearly all of the AWS infrastructure we run for NF-Core is in Europe and I'm in Europe, so >> Yeah, that that makes sense. No, that makes sense for me. >> Yeah, honestly, it was it was just a default zone for me. So, I didn't really think of too much about it. Also, generally EU is slightly more privacy conscious than non-EU. Um so, if in doubt for because we're storing people's email addresses with GDPR and stuff like this, I think it's kind of probably safer to be in Europe by default. >> I fully agree. Okay. Thank Thank you very much. I think we can stop recording for now. >> Okay. Thanks, everyone. >> Thank you.