EasyBuild State of the Union (2024)

all right so this is the the traditional easybuild State of the Union talk which we usually start with at the easybuild user meeting um it's a bit similar as it was in at previous Edition so looking at results from the easybuild user survey that we've been done been doing for a while now but I have some side steps left and right and some spoilers for upcoming talks as well I try to limit that a bit but I I couldn't resist them to some extent um the first thing I'll start with is explaining a bit more about the additional logos here so there's easy builds but which I'm I expect everybody here knows about at least or knows what it is um then there's the easy project as well and there's the multiscale European Center of Excellence so let me start with that easy build I guess I don't have to explain so I'll skip this slide it's just there for people um that are picking up the recording maybe later so they can get an idea what's important here is to discriminate between easy build and easy um so to some extent we sh ourselves in the foot there by using a name that's very similar or at least very related there's a bit of confusion about that um so easy at least in my point of view to some extent is an evolution of the easybuild project so it's like a Next Step um rather than working on a tool together um we can work on the actual software stack together and try and share that in a way that it works uh pretty much everywhere that you want to to use it so we call it a community project which started um in 2020 shortly before the the pandemic hit um I was invited to a meeting in Del in the Netherlands because some Dutch universities were interested in working on something together they weren't really sure about what but one common pain they had um was dealing with software installations even though most of them were already using easy builds um they were also uh keeping a close eye on what compute Canada was doing in the time with their shared software stack and they basically wanted to Kickstart something like this in um in the Netherlands at least and maybe in Europe and beyond that so at that first meeting it was only discussions and some ideas um but at the end of the meeting we sort of agreed okay let's let's try and make this happen and maybe even try and get some funding for that and see where it goes um then the pandemic hit so suddenly we had a lot of time uh to work on stuff so by the end of summer 2020 we essentially have had like a very early proof of concept quick and dirty puzzle together of um of what Canada was running already for years on on their system so we were pretty confident that we could we could pull it off if we really um wanted to and if we had the let's say the Manpower and the funding um to do so um so the main goal of the easy project is to build to build a shared software stack so the actual software installations themselves in some shape or form um which is optimized for different types of CPU so we don't want to cut Corners there like people often do with container images for example um so the the installation should still be optimized for particular CPUs um and we want to make it easily accessible and publicly accessible so anyone who wants to um can access that so that's very similar to what computer Canada is doing as well except maybe the community aspect so they're mostly focused on the systems in Canada not so much um beyond that um the focus of the project e of use performance is is a a given I think software testing as well so we don't want to just install that software but we also want to make sure it actually works and keeps working um so that's why we're building a test suite and we're we're doing lots of um effort on that it should be automated as much as possible because we're expecting thousands of software installations on this for different types of CPUs so if we're doing that by hand we're not going to get uh very far and that's also why we need to collaborate more together so to some extent this is an answer to easy bu not being good enough anymore easybuild is a tool but even if you use easybuild you often run into trouble with installations because of site specific changes or or um uh specific aspects to your systems and yeah it's time to take a next step and and work together more on something that's a bit more shared um all of this is is supported or powered by open source software easybuild is one of the projects in this but it's definitely not the only one um elmot is used to give the um the user environment certain vmfs is used to distribute the software Gen 2 prefix is used as a middle layer to to make sure we give installations Network on any Linux distribution reframe is used for testing and so on um so L will cover all of that in detail on on Thursday morning um so I won't spend too much time on on this here um the connection to multiscale so to the other logo on on the slid is um is that the multiscale Euro HPC Center of Excellence is essentially funding the current effort of of the easy project um so after we started in 2020 the first thing I think we did was write up um a paper an Open Access paper that um explained our ideas and then we started looking for funding so we applied for European funding um together with uh some scientific partners and then in the end got accepted and became the multiscale um Center of accidents so we'll we'll go um into detail on all of that on Thursday I won't spoil the fun uh too much here a little bit more on on multic scale um so it's a Euro EUR HBC project or I'm not really supposed to call it the project it's more a Consortium of of a bunch of Partners it's 16 Partners in total um get University through the HPC team is one of the partners um and there's really two aspects to the um to the uh to the project there services to facilitate development and deployment of scientific software that's just a nice way of saying we're going to sponsor easy um and that's basically the six Partners mentioned there including HBC now uh University of Bergen in Norway University of Barcelona Kent university University of Groningen and the surf um Center in the Netherlands are involved with that activity and then there's um a bunch of other partners involved with the scientific side of the project which is the development of software for efficient large scale multiscale simulations which I yeah we're not going to go into that too much we're more focused on the the technical part um what this really boils down to is it's a source of funding for the easy project so to really make easy happen the way we envision it and hence indirectly to some extent it's it's also funding um easy build because for example lar and and Pedro here in the room are quite new um to the community and they only came into the to the easybuild community Through the multiscale uh project so there's there's definitely some connection there as well um what's also important and that has already has some impact and will have some impact in the near future future as well is that this gets us a seat at the table in the Euro HPC community so people are now listening to us where they maybe were not listening to us before um we are getting very good feedback on the idea of U using easy as a shared software stack across Europe um so that's definitely uh snowballing into something that that may be popping up at hopefully all the Euro HPC systems in the future and then of course also the smaller systems that are inspired by that so we hope that actually happened all right and then yeah just some things to mention here in the in the scope of that um the multiscale project and easy on the side was mentioned as an as a as an example of a center of excellence that aligns well with the goals of the Euro HPC joint undertaking that was um a surprise to us we didn't know that they were going to raise us or put us in the spotlight like that at your HPC us today but it happened so that's a good sign of um of them being supportive for the ideas that we have um we did a a half day online tutorial on cmfs and easy as well in December last year where we got over 200 registrations and over 130 people attending the zoom session live so that was way beyond what we expected um so that shows there's a lot of interest in um in this topic in in serfs in general but also in the easy project more specifically that was very good news and also at the Euro HBC Summit last month in um in March in Anor um we tried to promote multiscale and easy as well um to the your HBC Community we got involved in several sessions there including the poster session including in some presentations um and we even managed to show at least the easy logo and the M skill logo on the on the very big screen in the room where they were doing some demos so that was interesting and this is probably going to snowball into into more interest for um for multiscale for easy and indirectly also for easy build all right then back to the more traditional part of um of the presentation is the easybuild user survey so how many people here participated in the survey you can be honest almost everybody large didn't okay yeah that's fine overall work quite happy with with the participation here so we getting over 100 people um to fill out the survey um I have to push for that a little bit so send out reminders again and again but that's that that's the same thing every year um so we're actually quite happy with that um I I actually cut out a couple of questions from this survey maybe you didn't mention because it's still a lot of questions um but I'm trying to keep it focused on the stuff that we think matters um the full results will eventually be available through the the easy documentation as well they're not there yet because I'm not finished with with preparing that um but I'm hoping to finish that let's say the the next couple of weeks so you'll see all the details um in there I will cover the highlights of um of the survey and the most interesting results or the things that I I think we should be aware of or should maybe discuss um this week um those will be covered in the talk um what's very good about having seven years of surveys is that you start seeing Trends um where where we're coming coming from where we're going um and that's definitely very interesting so I hope you can keep up keep this up um and keep um using this as a way to get some insights into the community um what is very important if you're doing a survey is that there's a huge amount of pias and to be aware of that so people who fill out the survey or maybe people who don't fill out all the survey um you need to take into account that that happens people who are very busy are probably not going to take I want to say half an hour maybe it was longer to fill out survey um so you're missing some of those people who are who tried easy build for a short while were not happy with it and then chucked it away there probably won't pop up in the survey so if it if it feels overly positive that's probably just um um a natural thing for the survey like this so taking all of that into account when looking at the results is important just keep that in the back of your head I try to do that when I looked at the results maybe I forgot every now and then so feel free to point that out as well if I'm a bit too positive or if I'm misinterpreting something according to you um a new question I added because I felt I had the room is if people have participated in the survey before um what's interesting here is that it's a nice mix of people who are experienced participants in the survey so over half of them have at least filled it out once or multiple times and about 40% are are new to the survey at all so that's already to me that's good that's a mix of of different different types of people new to the survey maybe not necessarily new to the community um and people who have filled it out quite quite a bit already um demographic so what kind of role do people have this is pretty stable over the years um the biggest part is still sis mins or people who consider themselves sis admins um that's maybe not a surprise I guess that's we we know that's the type of community where we have an easy build um what's interesting is that the user support um ratio is going down but the rsse so the research software engineer role is going up it's a kind of new thing and I guess people are just rebranding themselves as rses rather than user supports I'm not sure it makes a lot of difference in practice um but that's definitely popping up here what is interesting to me is that only 1% of the people here uh primarily consider themselves as software developers and this is known to be very different from SPC for example where they've done Service as well where they easily see over half of of the people I think um considering themselves to be software developers so that's that's one reason why I said before I don't see spec as competition not because it's a it's not a good tool but just because the community is very different than the focus of of the of the project is very different where are people geographically also no surprise here mostly in Europe uh a little bit more than 10% in North America what is interesting here and I'm I don't want to overly interpret this but the European part is going up is easy B becoming more European than it already is are people in in the US moving more towards P I don't know maybe I'm I'm over interpreting here but it is interesting that it is going up see next year yeah maybe next year it will go down again I don't know maybe just bias um I have no idea but it is yeah it's very clear that we're very European which is okay to me at least how long have people been using easy build well again a big mix um the two to let's say between two and 10 years is is the biggest um part of the community so there's a lot of experience um overall that's that's very good that means if there are any questions there's probably or there should be a lot of people who can answer your question to me that's a that's a good thing there is still an influx of new people um to some extent the easy project is is responsible for that as well a bit at least um and there is a very small group it's now about 5% who has over a decade of experience with easybuild that's interesting we never expected this to happen when we open source easybuild initially um certainly not that it was going to become as big as it is today and that people are going to stick around for uh for the long run let's say so that's to me that's a very good sign as well if anyone has has any remarks or any questions shout if I'm saying something stupid or something you disagree with I'm I'm very happy to hear it we have a microphone where's the mic yeah we have a microphone to pass around as well so people who are remote can pick up on that too um yeah in terms of experience with easy builds um one thing I I think we should be very aware of because there that's clearly a bias towards people who have a lot of experience um what those experienced people should stay aware of or or take into account that there's also people new to the tool coming in pretty much all the time not in big volumes but they are there um and that we're try to be welcome to the to these people so people who are not very experienced yet who are maybe not aware of some hidden policies or hidden best practices that we have um who may ask interesting questions I don't want to call it stupid questions because those don't exist um but who may ask questions we say okay how how did you not figure this out already um usually when those kind type of questions come it probably means we're not doing our job right in terms of documentation in terms of errors that easy bu produces I don't know what so that's usually a sign of something that we could improve and it's good to keep that in the back of your head um as as a more experienced easy Builder um since we have a lot of experience there's a lot that we can help these people with but also the other way around if they give feedback it's very very good to to listen to them and um and try to take that into account one way or another and to just be open to questions now I think in general we're quite good at that especially in slack I think we're fairly welcoming if questions are raised there usually somebody is answering those questions it's also different people answering questions um at least that's that's what I'm seeing as a very um a very high level thing uh and I think that's very good I hope we can we can keep that up um we don't have an official um what's it called code of conduct for easy build I don't really feel we need one it would be good to have one for sure but up until now in that 12 years of Open Source easy build I I'm not aware of any serious problems in the community hopefully that never happens but so that's also to me a very very good sign sign of a very healthy Community uh yeah so experienced easy Builders definitely go blind to certain things we're so so used to doing something and in a particular way I always use the example of creating patch files to most of us it's very obvious to people who are new to the project it's like I have no idea where to even start or what the best practices are um and that's probably one thing we still haven't documented well so there there's definitely lots of things there that we should we should be aware of take into account um and again here easy and multi scale I think is is part of this for the influx of of new people that's very good what what's also important to realize there is the the people coming in from that angle are funded to work not really on easy build but at least with easy builds but are also uh for doing that in a specific context in the context of easy so maybe they're focusing more on on problems or or features or um easy conflicts that are very relevant there and not so much to the broader easy build community so that's something to be at least aware of as well maybe this is something we can get back to in the discussion part if anyone has ideas um on this they're definitely welcome how did people first learn about easybuild it's a bit jumpy left and right but what it will boils down to is is word of mouth so people talking to other people people giving talks people doing Hands-On demos um a big part and a growing part as well is that's the red line the red part in the in the pie chart as well is people who learn about easu on the job um so there's a big that's also to me a good sign it means that sites are have adopted easy bu are sticking to it and feel the need to train their people the new people coming in um into easyb as well how many people here are in that category that's even half of the room pretty much yeah that's interesting okay so again I think that's a good sign of a healthy Community people s stick to easy build and um realize they need to train their people and in using it as well you have to blame me for for getting you into easy what was it the beer that did it or uh yeah the new easybuild logo I I kept this slide because we had it at the at the last meeting as well where it was very fresh um now it's not so fresh anymore um so we swapped to a new easybuild logo the old one is on the on the top left um that's for you right so someone here mentioned during the welcome that he managed to flip around the old and the new logo so he answered in the survey that he didn't like the new logo but he was actually thinking about the old logo all right so the the 6% here is more like 5% right okay but and I know there's some people here and I'm not sure some you were very critical of the new logo has it grown on you or I've buried my ex so it's okay okay so you're you're fine with it now right it felt like a nice way to celebrate 10 years of of uh of vild to me and in the end I don't think it has really backfired it's pretty okay um there's a small group who doesn't really like it 12% maybe a bit less fine they can come up with a new logo that's better than both than both and then we can see maybe for 10 years user meeting yeah I mean will definitely try and have some swag there if you want a coffee mug with a new logo Sam you should come to to UL maybe next year then you can smash it against the wall no overall I think this has worked out quite well this was actually a lot of efforts in the end when I think back to this it was like a lot of back and forth with a friend of mine who was designing logos for a living so he helped me quite a bit there in the end it looks very simple but that's that typically what happens with the with the logo as well I have to keep it simple to to keep it good so I spend way too much time on this I'm happy it's behind me if somebody else wants another logo it's it's their problem not mine it's a challenge yeah maybe you can give a talk next year on a better logo for okay um getting a bit more technical maybe um and interesting and operating systems in easybuild community it's still and that's probably not going to change anytime soon very heavily uh red head based um now that sest 7 line is dropping like a stone that seems very good there's one detail though in this survey um the last one you were only able to pick one operating system the most commonly used one and the previous ones that was different you could make multiple choices so that's why that black line is there you cannot really compare left and right to each other so be a little bit careful on how you interpret this one but it is clear that people are stepping away from C7 it's end of life uh it is it on the slide mid mid this year so you better step away from it right if you're not doing that yet be aware that there won't be any updates after June of this year um and currently still about 20% of the community says that this is their major operating system maybe they're planning to move away from it I hope they are um but it's important to be aware of this at least uh um it looks like Rocky Linux 8 in combination with redhead Enterprise Linux 8 is replacing that so that's the other two dots right uh well the blue one is Rocky Linux the green one is um is redhead Enterprise Linux that's where most of the community is Shifting to but what's already happening and that's what we're starting to look at in G as well is already making the jump to real 9 um so we're we're very soon going to have a mix of um real eight based and real n BAS based um operating systems that people are working with um it's very good to be aware of this it it means we know what to test for essentially and easy build we better come up with something that at least Works in boats and ideally still works in in c 7 as well I'm looking at Adam I know Adam does a lot of testing in Cento 7 and sometimes runs into runs into trouble that we Overlook in the other ones it looks like it's still important enough to try and keep that up although you're I think a bit of a lone wolf there in terms of testing on C7 yeah you're for the he just for the heck yeah yeah you're also in in containers I guess yeah wor yeah but it's good to be aware we definitely shouldn't only test with one particular uh operating system that would be not a good idea so mostly ra based there are others out there I didn't actually mention most of them on the slide so everything left of of the line is well uh well based um cental 6 is now gone happily but finally it's the first year that nobody answered cental 6 um the ones on the right of the line you BTU 2004 and 2204 are those are the only ones that at some point reached over uh 10% of adoption in the easyb community everything else so newer versions of yuntu older versions Debian sou whatever other operating system you can think of is definitely below 10% and very often below 5% % of adoption so we're very heavily uh well based I personally don't see this changing any time soon if somebody else thinks it will let us know because then we should definitely be aware of it but I I don't see it happening change question can you pass the mic for C 6 it's due to so maybe there are still some C six and some systems but not as the main one some dying clusters running sento 6 or something it's not my case but yeah yeah hopefully not because sento 6 is end of life since years right but okay maybe people opt into taking that risk let's see where this goes we'll definitely have to deal with at least a mix of of rad based stuff um for years to come that's probably going to stay like that then python versions this is a very nice graph I like this one a lot um so which python version are you using to run easy build you had to pick one here so this adds up this should add up to 100% unless I messed up something um Python 2.7 is now at 5% um people should be getting deprecation warnings if you're doing that because easybuild will drop support for running on top of Python 2.7 finally I should say um in Easy bu 5 so P python 27 is end of life since end of 2020 already uh but we are we are quite slow with dropping support for that because we know too many people were still relying on it so we kept it as long as we could um the biggest uh python version being used now is python 3.6 which is no surprise that's the default python in anything real 8 based which is fine I think officially it's already end of life or close to end of life it's already end of life yeah but but yeah but red hat is going to maintain that for for a while so that's not a big issue that it's in of Life there it's also no surprise that it's uh being adopted but that really only gets us to let's say about 60% that means everything else is splintered across a bunch of Python 3 versions that's annoying we can't really change that um but that's something to to be very aware of I want to say that's fine because there's no breaking changes between Python versions but that's very very not true right we know that it's it's it's a huge pain especially python 3.12 which we're going to support in Easy bu 5 was a was a massive pain because they dropped some stuff from the standard Library there um it's only not going to be a pain for us in easybuild but also for any scientific project using python it's going to take time for them to actually support Python 3 or 12 um that's a reality we have to deal with it is what it is um we're very aware of that we'll make sure that easy build is tested or or at least testing the the framework test Suite on all these python versions so if we do uh introduce any trouble there that will notice but if if you are in one of these splintered versions if you're using another python version and you're seeing a hard crash of easybuild or something that it's expecting to be there it's not there let us know please report it so we can fix it so we we definitely want to support essentially all the python three versions newer than 36 Easy buil should work on them um I'm not sure how long we will be stuck to 3.6 but probably years to come it looks like unless that's really going to unless there's a big incentive that people will start moving to real 9 which is 3.9 I think python 3.9 I think so yeah um but yeah it's clear that adoption of that is still quite low so we'll see so this is going to be a challenge uh going forward we'll have to make sure we keep testing on enough versions um how many people are fine with getting rid of 2.7 yeah most of them are of course it's now basically 95% 5% says it's problematic but it's only problematic probably because they have to install one or two extra packages um which is very easy to do um even on on S7 you can very easily install a python 3.6 and just tell Easy B to use that instead so it shouldn't really um cause any trouble for people in practice if they look into it for a couple of minutes um you've been getting deprecation warning since January of last year um hopefully the next release will actually drop support for Python 2.7 so then it's going to be a hard error and there will be absolutely no way to to still run easy build on 2.7 we kicked out code that's that's supporting it so it's not going to work anymore so be aware of that if you're still using 2.7 somewhere the size of systems that people people are um are managing this is not really specific to easybuild so we even if you're not using easybuild on these systems you could still answer this question um again this is a huge mix of PE of things no surprise some people are apparently mostly using it on their laptop because that's I think the only place where where you can get less than 100 cores these days um huge mix um some people are relatively small less than 10,000 cores uh that's about 40% some people are in the middle another 40% and then anything else over 100,000 course which is actually not that difficult to to be in these days if you have one decent side side system a couple of hundred of notes were very quickly in in this box um so yeah nothing really special here nothing I don't think that really makes a difference for easy build but it's good to know that at least people are using it on quite small systems as well for gpus this is actually remarkably stable in terms of numbers um of gpus now we know these things are expensive already and are only becoming more and more expensive so I don't expect to see an upwards Trend here um in general um there's still a small part that has no gpus at all and maybe wants to keep that up because it's it's additionally additional effort it's an additional ecosystem we have to figure out um most of the sites have pretty low numbers of gpus so 100 or less so it's not because you have only 100 gpus that you're um an OD duck let's say so especially well at least not in the easy bu community so that's that's just good to know all right and pretty much all of this is inidia uh that's coming up as well terms of types of processors Intel is still king um today it's going down at least until Skylake is going down Skylake is already actually relatively old but it's still by far the most commonly used um processor in the easybuild community at least what's interesting to me is that the older processors has well broad well even Ivy Bridge Sandy Bridge are sticking around so people are not really killing their old Hardware at least some of them are not um that's definitely related to prices and and being able to buy new hardware as well of course um Cascade Lake and Ice Lake are seeing pretty good adoption um and sappire Rapids is coming I don't have any experience with sappire Rapids myself but that's a bit of an an odd duck as well because you have these mix of cores I think in sappire Rapids e cor and PES I'm not sure if that's something we really need to do sorry some of them have just stand some of them only have one type of core so the more highend ones only have course then or yeah okay so I personally haven't looked into this as well in detail myself I'm not sure is there something we should do with easybuild on this anything special or is it sorry is it the same it's not no the the the efficiency cores don't support apx 512 I think yeah so I'm not sure that's maybe that's all shove to the compiler and the kernel for scheduling I don't really know but uh yeah that's uh that's why I ask is there something we need to do in easy build for this I don't think so but oh I'm sorry about that I mean I'm might be wrong but I think that for the sappire rapids The which I think is a server platform is super to be only PE course the mixture is usually there on the like on the desktop platforms where they mix the eour and peor because they I mean you do that mostly for power reasons right yeah uh but for the server platforms I mean when you want to have high performance and also I mean you have to run a lot of different workflows with different users it's a mess having this different things so I think that actually they don't mix them for the server platform but yeah I'm I have noten yeah just agree with that is that I do have them and I've seen nothing go wrong on the server platform Sapphire Rapids which would so I don't think there's an issue because I would hope somebody would have hit a problem not an issue for the server at least it'll be the desktop I mean but it might be those people who using desktop type machines in a small cluster might end up with both types and need to be aware and at that point I have no idea and then what what do you do do do you build the thing twice once for eor and PE course and schedular figures it out because you need different binaries or how does that work or or you build a binary that works on boat fat binary maybe there are options but it's not anything I've had to explore so I don't know maybe you just shove it to the to the compiler say but but you also hit if you've got auler there as well you also how do you schedule you know if you're say using slum how do you schedule E versus P yeah well it's May May boil down to pinning right where you pin on the right course and then but requesting it you know do you just request one or yeah I haven't seen a lot of a lot of news or a lot of noise about this so I expect it's largely an issue but yeah okay if somebody is aware of problems there are things we should be doing let no but then non Intel and just be aware of the scales here this goes up to 70% Sky leg here is at about 55 um looking at AMD we're hitting at most 40% so it's definitely lower than Intel but it is quite significant and what is interesting is that Milan so zen3 is definitely on the rise that's pretty clear um Zen two is maybe um Beyond it spe already I'm not sure again I don't want to over interpret this but uh but at least there's a good mix of of AMD and Intel going on with Genoa coming up um soon as well so it's about what is it 12% 133% and older core still around as well so it's clear that we're still in an x86 uh World currently at least for most people um but as Mel is also going to mention this afternoon that's probably going to change in the coming years there's no sign of risk fire yet in the easybuild community there's also no surprise there because the hardware really isn't there yet um but yeah it it's coming in the next couple of years at least what's definitely coming is arm um so Nvidia Grace is is uh yeah we're hearing a lot of noise about people adopting that also in the Euro HPC community so the bigger systems there's jualan coming up in Portugal which is Fujitsu a64 FX based so that's R based um and then there's Jupiter coming up um as well it should be there what sumwhere next year Alexander operational something like this yeah so Jupiter is is Grace Hopper so Grace CPUs Hoppers gpus and then with an ARA if all goes well ARA partition as well which is also art based so that's going to be a fully art based um system um so that's two very big systems there's probably going to be smaller systems that follow suit as well at least within VR Grace um and I think even aair also did an announcement of a new arm CPU um so yeah let's see so it's definitely coming it's something we should be aware of um thanks to easy we're actually um pretty well prepared for that already or at least we're actively working on it so we're testing lots of things on arm we're also hitting issues left and right but that we're figuring out together with um the maintainers of those open source packages we found a bug in open API for example um which seemed pretty serious it seemed serious enough for them to block their next release so why we were hitting it first that's not fully clear to me maybe we're doing something weird um and there's other projects as well left and right even in gromax I think we found an issue now as well on on Nvidia on neoverse V1 CPU so anything that has sves uh we found an issue there too so okay we're we're starting to poke projects left and right and again by running the test through the easy project not only doing the builds but being very uh paying a lot of attention to the test Suite we're finding smallish issues um on arm so it's definitely early days is maybe the wrong term but it's definitely interesting going into that uh that platform as well so we'll we'll probably see more of that Nvidia Grace is now is on the rise there so it's it looks like it's popping up out of nowhere it hasn't hit 5% yet but next year I expect it to be significantly higher uh let's see so the the easy days let's say of everything Intel is definitely over um in terms of gpus yeah it's pretty much all Nvidia nowadays a mix of different Generations um what is also interesting is that the older stuff uh like Pascal for example p00 um is still at over 25% and it looks like people are keeping that alive it's very difficult to get new gpus so people are trying to stretch the lifetime of the the stuff they have um and to make sure that all these AI people doing all all their llm trainings and Alexander um can still get their work done um aair seems to have Peak so a100 is at 65% and looks like it's staying there um hoer is is on the rise if you can get them and if your pockets are deep enough um you can buy some hoer gpus but it's pretty much still all Intel um at least in the easybuild community it is people that have AMD gpus are definitely a small minority it's currently at about 5% um it's not shown on the plot but from the from the survey we see about 5% of mdgs yeah so I was at a conference last week and one of the comments that came back from a large User Group there was p100 V100 for sort of some the computation chemistry work and some of that sort of side they saw no performance Improvement when they go to a100 because they don't use tensors so their comment was they're looking for GPU options that don't have that extra tensal in them because they hopefully be cheaper and they I know somebody else said a40s as an option for sort of similar reasons um because yeah the jumping price even you know to 100s and H 100s is um yeah the question is of course how long Nidia will keep supporting gpus I'm not sure for p100s if can you like update to the latest drivers and still have them operational yeah for now at least the cap is not supportive anymore yeah okay yeah so let's see where this goes I definitely expect more adoption of AMD gpus in the short term because the the prices that Nvidia is handling are just insane so they're essentially forcing people to look at AMD at least consider it I'm not sure they realize that but that's that's really what they're doing um I mean the the I guess the software ecosystem is definitely holding people back a bit things like pytorch I'm not sure Alexander how familiar you are with pytorch and AMD gpus but it should work right but uh software have been re so yeah I avoided so far looking into AMD support for pyto which would mean getting this rock M stuff to work yeah that's the first HL yeah it's already enough work getting the Cuda versions to work and that would be another dimension of things that need to be built and then actually run their test you with that and since not too many people have access to that and even less people are even running the test Suite you're you're to run into trouble there yes same as for example on power PC not many people using it it's not well tested so yeah less issues got reported less issues got fixed yeah yeah but they are also slower so it's yes even you need twice as much as the same price of an Nvidia one so there is very little reason to use it even the new ones the 300 series I have the 200 I don't know I think for 300 it's a different matter oh okay and and they also have more memory than the Nidia ones yeah but still it's I don't know but it yeah I expect it is something that we'll have to we'll have to deal with in maybe not next year already but it's it's coming at least yeah more and more people are going to pick them up the other comment I've been hearing is at least you can get them delivered or at least up to now you delivered now rather than the six or six or nine month lead time yeah to some extent that's forcing people well for yeah for the Nvidia stuff we've been any number of them it's just months and months so there's going to be impact there yeah I'm pretty sure we'll see where it goes now the if people are scared about roam some people are looking it into it already OA is one of them you've played a bit with the roam stuff in easybuild there's other sites as well but it's yeah there's not a lot of them there's not a lot of momentum let's say on this but people are looking into it and okay it's it's more messy let's say to get started large can um but it is coming and we are figuring it out and we can actually get some help from AMD for this as well so we have some contacts at AMD if we do need help we can we can reach out to them I was wondering does it make any sense to look into sickle more to avoid you know this specific we are going down Cuda we are going down Rockham so we have to redo it all again my understanding of sickle is you go down sickle and then the back end is a little bit more agnostic I might be completely wrong just to be clear here sickle is sitting on top of rock you need Rock first before you can get sickle I think so and and then you can only support the software that actually uses CLE like if it doesn't use CLE it doesn't really help you okay so it's not really up to us I would say whether we we put more effort in sickle than in in ROM directly if the software doesn't support CLE we're still stuck and we'll probably have to support both there may be advantages of using CLE over Rock directly even if both are supported so so not only is it becoming Messier in in Hardware Land CPUs gpus also software is going it's becoming more complex more things to look into yeah to some extent that's one of the motivations for the easy project as well like things are things are we knew three years ago that things were going to get messy and it's slowly happening right so yeah we need to plan ahead for that and try to factor it in okay I'm I'm sure we'll we'll get back to these discussions especially when we have the side talks when people explain what type of Hardware they currently have and maybe what they've been struggling with um looking at the documentation so this was a big effort we did in uh mostly in 20 22 um where Simon and James yeah sorry James were involved with quite a bit um so the old documentation the structure of that and the way it was being managed wasn't really good um so we did a revamp of that a lot of effort in porting that over I think it's a lot better now it's easier to contribute to as well it has a dark mode like it's showing there not only a light mode I'm a big fan of dark mode um um and we are ping in the survey how how happy people are with that um it looks like the um the new shiny uh look of the documentation has faded off a bit so a bit more people are critical over our current documentation than U than in the previous survey so that's grown from about 10% to 25% that says um I'm a bit disappointed or um I expected better um that's good that that's good feedback um we know there's gaps in the document we know there's stuff we can do better especially documenting new features is not something we're very good at um so we should try and do that better but overall I think it's still a significant Improvement to compared to what we had before and it looks like most people are also agreeing with that uh the tutorial which is still a bit disconnected of the documentation it's a separate site entirely um which is being updated essentially as we are organizing tutorials um what's good news is that more people are now at least aware that the tutorial is there that was a bit of an issue before now we're clearly linking that from the landing page in the documentation that probably helps we try to make noise when we uh when we do tutorials that probably helps as well maybe I I hope this this is also what people use when they're new to easy build and they're being introduced to it so what how many people here actually used the tutorial when they were learning about e builds okay least a handful of people did that's good um so there the focus is a lot different from compared to the documentation of course it's really step-by-step guides to to learning about EAS bus um have another slide on this yeah so the last time we did an official official easy build tutorial was at ISC 22 um a half day one we also had one at 21 uh we resubmitted for ISC 23 but it didn't get accepted so there was a lot of competition there and we didn't make the cut um for ic24 we we changed tactics a bit and we focused a bit more on the easy project as a more of indirect way of um of getting a tutorial for easy build we actually shot ourselves in the foot there because we submitted three tutorials one on easy as a more user focused one one on cvmfs as a more backend thing together with the cvmfs developers and with easy as an example um and then one on Magic Castle which is a tool to spin up slurm clusters in the cloud where we used easy again as a running example of how to get software on a fresh slurm cluster um and some of the reviewers noticed that and like scored gave a lower score for one tutorial because they wanted the other and then someone else did it the other way around and it basically killed both submissions so yeah okay I guess that was our own fault for being a bit too too aggressive um we were we were close to getting the easy tutorial accepted but it didn't make the cut um in the end so yeah we'll see going forward we're we're we've resubmitted the only the easy tutorial for SE in the US we'll see if it makes the cut there apparently there's a bit more room for tutorials there so if we already got a good score for ISC it seems likely that we may get selected now of course the US is not really our main Community but and to be honest these tutorials are important mainly in terms of getting the word out and and Optics but in terms of ual attendance you're talking maybe 20 people and you're not able to record those tutorials either so it's a one-time thing and then you lose all that effort to some extent so it is important to be there or to try to be there um but it's not the best way of getting lots of people to follow the tutorial the the last time we did an actual tutorial and in this case it was combined with easy with the easy project as well was in London last year right after the easybuild user meeting where we also had about 20 people I think attending live but what was really good there is that we recorded the whole thing and it's available on the YouTube channel as well so that lives on and that's something people can really get back to um very easily and and for free because the prices for attending the tutorials that ISC and SE are also quite quite High uh okay so that's about a year ago maybe it's time for another tutorial I don't want to give myself more work or uh or try and get other people to to do that work but I think it could make sense especially since we have an influx of lots of new people um to do another easy bu tutorial if we do I would make it an online thing free to attend maybe split up into small parts like three four hour sessions or maybe two of three are those spread over um the same amount of weeks maybe um just to make sure that that people can uh can schedule that in quite well um and it we could get very good attendance for when we did an online tutorial during the pandemic because we did couldn't do it at ISC we had over 100 registrations so that's more worth the effort than having 20 people on site at at I if we do that and there's been some demand for that as well um is we could make an an easy build tutorial that goes beyond the basics at least in the last part that talks about implementing hooks um talks about how easy build is being used in the easy project and all the bells and whistles that that come with that of course we'll need effort and time to um to put those tutorial contents together all the basic stuff is already there so we can just rehash and update that a bit um but anything beyond that is is probably going to be a bit of work now this could make sense once we have easyb 5 actually available because then we can make sure that all of that material is updated for easy bolt 5 um and maybe we can cover some of the new stuff that easy bolt 5 can do in in that tutorial as well yeah so for now it's just an idea if anyone's jumping around to help out with making that happen let me know um but we'll see all right and jumping ahead supported software this this graph keeps scaring me um because this line is just straight going up we're now very close to three and a half thousand supported software projects in easy builds without extensions so anything we we install as an extension on top of something else um so all these python packages all these R liaries all these per modules are not really counted in this three and a half thousand you can easily add 2,000 maybe more to that number to get a total number of software projects we support it's pretty insane right and it's not it looks like it's not slowing down we're adding every release we're adding about 100 projects new projects that we support um to some extent I think this is the fault of the AI boom uh where people pick people create new tools people people pick up on new tools that they're are not familiar with um yeah let's see what what happens here there's going to be a bit of a drop here for easy build 5 um because we're we're archiving old easy conflicts that are not using recent tool chains anymore and I forgot the number Simon do you know from the top of your head how many projects we would lose it was in the several hundreds uh we can probably look it up easily but fine but there's so we had a small drop here when we released easybuild 4 that's also going to be archived a bunch of old stuff that nobody seemed to be interested anymore the drop for easybuild 5 is definitely going to be bigger so there's going to be a couple of hundred projects that that are not supported anymore at least out of the box for easy build we can easily bring them back if there's interest for it but we only see easy configs with very old tool chains and nobody's really updating yeah some like would it be possible to add some kind of trays there like okay um we have software for that but it's archived so people know there is some Easy confix some old ones okay but they can use that to start an updated version when would you add that Trace when people do do say search or or you or just in the in the file tree just and you mean keep the directories around but just add a read me that says this has been archived yeah yeah that makes sense yeah we can autogenerate that as well so it's not a huge effort to to do there's a question there's a chat question on which tool chains are being dicated oh that's coming so um Simon will talk about that this afternoon yeah I won't spoil that fun yet okay so yeah let's see where this goes there so expect a drop here and I mean anything that's actively being contributed and updated is going to stay we're not kicking out stuff for for the fun of it the AR there yeah yeah the archive easy confli are still there so you can still if you want to revive them you have something to start from you don't need to start from source of course and it's all in history as well that's also something Simon will talk about a little bit this afternoon all right going on with the survey jumping topics a bit the easy build maining list that's a traditional one um it's still active people are still using it a lot less than before but okay um so maybe this is more the old-timers who um who want who prefer sending emails rather than chatting in slack I'm not sure but it it's an important thing to keep alive in the community many people rely on it um I'm probably the top poster there sending reminders for easy bu com calls and releases but fine but it's important enough to keep it around at least um what is interesting that um um a big amount of people like the yellow line here about 25% says I'm not on the mailing list yet but I should join maybe they're not aware it's not very active and I'm not sure why they just don't join but okay um yeah it's we're going to keep it around it's not the main Channel today but it is interesting enough to keep it around um so the the volume in terms of post has definitely gone down um in 2017 we had well over a thousand emails coming into the mailing list in a year then we started uh with the slack Channel and it dropped like a stone has slowly been going down last year it actually picked up speed a bit again I don't expect it to come up again fully but yeah so some people sent more emails last year than the year before I don't know um in terms of uh subscribers we still have over 300 subscribers every now and then people unsubscribe as well um so this this fluctuates a bit but it's it's still going up compared to last year then in slack this is the the main Hub let's say where you want where you can integrate or when you can get in touch with the easybuild community um many people are on there um so everything green and blue is actively is in the easybuild slack but it's not maybe first of all paying attention to it or participating in discussions so people just read don't uh don't ever reply or uh or mention something there which is fine you're not forced to say anything you can just follow along and try to keep up with stuff being posted there um but we should be aware that there's a big part of the community which where which we don't hear at all not in slack not in the mailing list probably even not in user meetings and golf calls and so on so some people are just silent which is fine but yeah it's good to be aware of that uh this is the activity in terms of what is this message no weekly active members so it's uh let's say about a 100 people are active every week and about uh what is it 20 no 40 people are are posting messages every week so it's not a huge vocal Community which is maybe good there's already enough to catch up on um but fine what is important to know is that there's 875 people last time I checked in the easy bu slack it's a very good place to ask questions to try and get help with something you usually get an answer relatively quickly uh we try to keep it organized as well so if someone posts a question we try to make that a thread so only people interested in that can actually follow along so you're not bombarded with with messages so yeah we do what we can to to keep this manageable um as well and we see about 150 people joining the slack every year so it is steadily growing uh it looks like maybe by the time of the next user meeting we'll we'll have over a thousand people there so it is important to make sure everybody behaves and that we we can sort of keep this organized a little bit um we I think it was I did mention this on the slide but last year was the first time we had to kick someone out of the slack because they were doing some spam type of messages uh so we we removed the message we kicked them out he actually joined again later with a different handle but he wasn't active anymore so fine if you're behaving you can stay if you're being stupid we'll kick you out pretty soon and there's multip people that can that can manage this slack it's definitely not only me okay then switching topics again looking at the environment modules tool so the only next to python the only true dependency that easybuild has you need some environment modules tool that easybuild can talk to and that can it can use to generate module files for no surprise here elmot is still uh the King of the Hill here um what's also good is that it's elmot 8 um for three qus of the easybuild community is using elmot 8 about 15% is still using elmot 7 now that is going down quite quickly that that's good elmot 7 is several years old it hasn't been updated in a long time so there's absolutely no reason to still use it um be aware that we will we will drop support for Elm mod 6 and seven and easybuild 5 so if you're still using Elm mod 7 um make sure you're not and you're making the jump to lmot 8 or you're going to be stuck to the latest easy bu for release um at least for elmot 6 people have been getting deprecation warning so if you ignore yeah you're you're shooting yourself in the foot at least if you make the jump to easy um if you want to talk about elmot and at least in in Ubuntu getting recent versions of it you should talk to Alexander from y sudak on on GitHub he's he's maintaining an elmot package for for Dean based systems and then there's the the competitor for elmot the tickle based environment mod the original implementation of environment modules is still actively maintained um but there's only a small minority using that about 7% the tile based one yeah sure there's no real we're actually getting active contributions by the environment modules maintainer into easybuild framework to improve that support so we don't see a reason to drop support for that well there is a couple at least some versions will be deprecated yeah um and if you're using version four it might be you need not an ancient version four yeah you have to keep up with versions there so EAS does a version check as well and make sure that you have a sufficiently recent one that's the only thing we're going to be a bit more strict on an E I do have it in my slides I can't remember exactly off hand but yeah especially since we're getting contributions from the maintainer there's no reason to drop support for this all right then um other tools I won't spend too much time on this because it's a big mix of things but some things I noticed is that um gc3 pi as a back end for letting easy bumit jobs is almost non-existent I also think it's not actively maintained anymore um so we haven't done that yet but we'll we'll deprecate the support for gc3 pi is a job back end I will make the slur back end the default one which it's not today so that's a pretty small change but we'll we'll push that in for easyb five as well um many people are using obtainer to run container images alongside easy build I would say I would expect that mostly images they get from anywhere like from Invidia or what else I'm not sure if that's actually images they're also creating with easy bu I would expect not but maybe some people are are doing that so how these things are being used is is not really clear um what is I should want to say a surprise or well maybe it should be a surprise is that many people are also using gon next to easybuild it's actually growing it was 25% before it's now 30% uh like Simon mentioned this um this morning I think at breakfast as well that they're using it in Birmingham for example um yeah many other sites are are doing so as well there are some several caveats there I don't want to be overly critical on on cond but uh in terms of performance that's definitely something you have to be careful with what kind of installations you get there there's also legal stuff going on now with anakonda so anakonda is becoming very aggressive with hunting sides down that are not fully following the terms and conditions I'm not sure why they're doing that they're are they short of cash and they want to milk some sites or um or do they have too many lawyers sitting on their hands not having enough work I don't know but just be aware of that if you're using cona make sure you're following the full um the full legal stuff as well um St V MFS I would expect that's mostly people using easy but I'm not sure is a is a small minority maybe that will grow as well start making more and more noise about that in the easybuild community and in the easy Community for sure um there's still a small minority using both easy build and spack I think that's something Luca is going to touch on in his side talk so it's it's 10% and it's a pretty stable 10% so we see see this popping up in the survey over and over again um to me there for different use cases I'm not sure if everybody agrees with that but that's also fine some people are just cherry picking whatever works if it's a container then we'll use the container if it's an easy build we install it with easy build if it if it's better supported it's back then we use spec instead so I guess to some extent that makes sense um and some other things which are which were only added to the survey recently is open on demand so for creating a web portal um to let your users access your infrastructure through a standard browser more or more people are using that I think that makes sense it's a very good way of drawing and more people um rich is a tool that easybuild can use to give you more colors in your terminal and and progress bars and stuff if you're not using that yet do a pip install Rich then try easy build again and it will look a bit different in some cases um so just visually that's a useful tool more people are using reframe and easy is starting to get some better adoption as well so we hope we hope to see that grow over the years too uh yeah then a little bit more on easy there was an question on easy have you heard about the easy project um and there's definitely an increase in people that have at least heard about it or even have played with it um for sure so about let's say a third now has hands-on experience with easy to some extent at least maybe not really actively making it available on their site yet but at least looking at the project and maybe uh considering adopting it so again easy is now a funded effort since the start of 23 through the multiscale center of excellence um and there's definitely a lot of stuff coming back from e into easy build as well since this the main tool to install software in that project I will focus on that on Thursday as well this um a question here how many people have easy available on the main system you use today so let's say uh seven seven six people okay let's see if that changes next time but sorry e yeah the easy project not yet okay so yeah 56 then okay um in terms of easy buil releases so how happy are people with that that's overly positive um for now so we've been having a steady pace of uh about seven releases every year it seems that something people are quite happy with now releases are not a small effort to do so it's not pushing a button and the release comes out it's like like at least a day of effort maybe two um so usually I try to ask Miguel and or Sebastian for some help with preparing release notes and we do a regression test um it's not a full regression test anymore these days but we try to really do our homework and make sure there's no breaking changes in in the easyb releases so it is a bit of effort and that's why we don't want to do it too frequently but it seems like the community is pretty happy with how frequently we've been pushing our releases so far now for for this year I expect this to drop um so we did the 491 release in April which is already four months into the year that's quite late so usually we're we have a shorter time between releases we're working on easy build 5 which we hope to get out the door in the com weeks weeks is fake enough um we'll see how soon it it may happen that after 5 comes out and people actually play with it and give feedback that will do bug fix releases of that a bit more quickly than usual so maybe we we'll catch up again but it's quite likely that this will be a lower number in in 24 all right then looking at at contributors I can't I always put this in every year I'm sorry so contributions to easybuild we get lots of those we're happy with that but it's a lot of effort to keep up as well and to process everything the way we we want it to and the way they should should be processed um so looking into framework so the core of easybuild uh last year we got about 200 pool requests um now the vast majority of that is people outside of of gent University which is very good that used to be very different if you look at the history here so everything blue is is is me essentially um then the the red parts are the rest of the HPC again team so most of the team is now no longer involved in in active development of easu I guess Lara there is now the biggest exception since May of last year um so she's she's definitely actively contributing to easy build but the rest of the team isn't really involved in this anymore there's also no need because we get a lot of poll requests from outside of the of get University and even outside of of the current easybuild maintainers so about a third of all framework pool requests are from non- maintainers which is which is good to me it means that uh it's relatively or it's at least doable to to get into framework and make changes there make contributions healthy sign I think looking at easy blocks it's uh okay it's easier to contribute to easy blocks than to framework it's less daunting it's more focused it's essentially one python script that you have to deal with so that's more manageable um also there um we see a good amount of contributions outside of gent and even by non- maintainers um there's less activity here compared to previous years that's the same as it was in 22 so that's about uh let's say 170 180 pull requests a year um I think I didn't check this but I think part of the reason for that is that there's a a shift towards using generic easy blocks a bit more which typically need less maintenance as well because they're more generic so you don't need to update them for for particular software releases and and so on so we we've introduced additional generic easy blocks like Meson ninja and these type of things um that maybe why we're seeing less BRS in in Easy blocks than before and that's not necessarily a bad thing there's just less maintenance work in Easy blocks maybe that's good for yeah for 2024 the number is going to be a lot bigger because of eible 5 one of the things I've noticed is quite a lot of there's a fairly decent chunk of projects some of them have done it via moving say configure mate to cmake but some of them where the you just see the Upstream project has cleaned up there in and has done significant work and suddenly you jump from needing a easy block to just going oh default cake will actually allow you to do this because you don't need to yeah that's so the the project themselves has actually enabled us to switch to generic easy blocks and as a result of that yeah we've actually we're removing some easy blocks as well in easybuild 5 because there there's really no need for them anymore either the software is not relevant anymore or the easy conf has been switched to to generic easy block so that's definitely a help all right so that's let's say 200 hprs for framework 1880 hprs for easy blocks now the numbers become a lot bigger we're looking at easy configs um we used to we got very close to two and a half thousand poll requests in 2021 thank you co many people had time to open poll requests um it's a little bit lower again in 22 and 23 but it's still high enough that it's a significant effort to keep up um so thank you again to all the easybuild maintainers the active easy bu maintainers not all of them are active anymore um who help out with that we we try to plan a weekly role which we call maintainer of the week where one person is uh putting themselves up for trying to spend at least one hour per day looking at poll requests um that seems to be working quite well um in practice that usually works out relatively well it doesn't mean it's the only person looking at pull requests other people pitch in whenever they can as well um we're not really keeping up with incoming contributions I would say but given the numbers we're we're getting here over uh 2,20 pull requests a year we're not doing that bad I think so we're actually doing quite okay um let's see what else yeah of course lots of contributors outside of gent we still do a fair amount of easy configs we maybe focus a bit more on the tool chains and everything needed for that and common dependencies and maybe less in the more uh uh the more exotic Parts yeah so we're still actively contributing ourselves for sure but it is at least my part is definitely going down so the blue part is going down I'm spending less and less time myself on on easy conflicts and it looks like the community is just making up for that don't tell me I'm not doing enough as some um so that's opening PRS and where they come from then looking at merging PRS um so here on the left oh and they should have moved that one the legend there fine um I'm still merging quite a bit of uh of PR myself but it's a couple of hundred per year it's not insane it's like let's say one a day which is manageable all the other maintainers are are tackling the the bulk of that work that took some time as you can see we we started getting other maintainers in 2017 it took some time for them to be uh maybe confident enough to to tackle that work themselves but again thanks thanks again for all the help there without the other maintainers easy build would not be what it is today so if we there's absolutely no way that we were going to keep up with the over 1500 PRS we were getting in 2016 doing that by myself yeah I was I didn't want to say it was burning out but I was definitely heading towards that so asking help from maintainers there was a very good move um or it wouldn't have been so positive as it is today the graph on the right is showing how people are opening po request so we have easybuild we have GitHub integration and easybuild where you can open PRS from the command line the vast majority of the PRS are opened like that that's a good thing as well they're easier to review there's more structure there um so that's a good thing it's still not 100% it probably doesn't need to be um that's fine what's not so good is that we still have a a big number of pool requests open mainly from 22 and 23 so we have cleaned up quite a bit of stuff there's old PR still open I think that's what my next slide is on yeah so this is an a historical overview of how many pool requests easy cic pull request we have open at some point we were hitting over 800 open PRS which was way too high that's when we sounded the alarm Bell to some extent um and that's where we put in some dedicated effort to to clean that up a bit that's also the main reason we have introduced a tool chain support policy which is something that Simon is going to talk about um this afternoon a bit that has helped us we're now back to uh 650 is maybe closer to 700 today um but definitely once easy build 5 is released we're going to chase this down a bit more aggressively and get those numbers down a bit especially the PRS from over two years ago are unlikely to still be relevant so those can probably just be closed without too much effort but this is an ongoing fight that's pretty clear and it's a fight that we yeah we will have to keep uh keep fighting in in the years to come um yeah so it's definitely a challenge keeping up with things now if you keep in mind that we have two and a half thousand PRS per year and we currently have let's say 700 open that's not too bad so things are piling up a bit but not insanely uh so we are we are trying to make sure that we keep up to this to some extent maybe we should also be a bit more aggressive in closing PRS maybe we're sometimes a bit too friendly I don't know and that's maybe a situation that has grown over the years as well uh so if we do if we close something we'll add a good motivation to it it won't be applied to close um but yeah it's something we'll have to do to try and keep up and and make sure we don't drown in open PRS then another traditional graph this is looking at the number of unique contributors over time um so we we get about we got about 50 new contributors in 2023 so 50 people who never opened the pool request to the easy conflix repository before um we now have almost five 450 in total so that's 450 unique people that have at least contributed one easy config file that's quite good um that's again something we definitely didn't expect in 2012 when we open source easy builds we were not expecting to get that that much help so that's very good um we keep getting new people that are opening pull requests every now and then of course for easy confix this is skyro skyrocketing quite quickly but also when easy blocks and framework there's an influx of people who are new to that part of easy build and are actively contributing sometimes these people come to totally out of the blue they open a poll request and then they dis they have no idea who they are where they come from why they made the contribution and then they just wander off to someone else that's fine that's what open source is um but it's also yeah sometimes a challenge that we don't have a direct line to these people to discuss something um that can be challenging this is um okay so this was in total unique contributors over time this is unique contributors per year um so here you see for frame we're talking about 20 people that open a p request to framework per year so who are confident enough to find their way for easy blocks that was at around 30 for a while in 2023 that dropped a bit even below framework so we had less people only 17 who open a pull request to easy blocks I don't think that's an alarming thing um it can just be coincidence or there was just less activity in Easy blocks we saw less PRS as well so it's connected to that um but okay yeah it is what it is I already peaked ahead a bit so in the the first quarter of 24 we already have more than 25 people already so it's yeah it's like a temporary thing probably more of a coinci than than anything else um for easy configs we're dealing with close to 130 unique people in 2023 who open the pquest that's good that's a lot of people who know their way around enough um have uh enough confidence to open PRS again it's a challenge for maintainers the more people we have to talk to or the more people were were processing contributions from the more of a challenge it is but okay we shouldn't complain here we should be happy that we're getting these contributions and then I think this was the final question in the survey how how happy are you with easy build today how would you rate the overall quality of easy build that's overly positive again here there's a bias here people who hate easy build for probably not sit down for half an hour and fill out a survey but fine it is quite positive and we still haven't had anyone who ever answered would be bad so to hasn't filled out the survey ever I don't know maybe he's yeah maybe he likes it a lot that's possible yeah so overall people are pretty happy with easy of course that's where in the community that's why they're in the community as well okay then a couple of other slides to wrap up doing quite well on time all right excellent um so looking into goals and challenges for the coming months let's say first so the short term U the top one there definitely is is finishing up the efforts on easy build 5 we've been working on that for about a year now uh I checked my slides of last year and last year we said we hope to release e 5 in 2023 that didn't happen um then I think my next goal was make it the first release of 2024 that was taking a bit too long as well so we pushed out 491 um as well I think we're relatively close to it now we have a a good plan ahead um so Simon is going to talk about this this afternoon as well we'll get it out the door when it's ready essentially so there's no real deadline there for us um I'm not sure that someone is really waiting on EAS B 5 there's not that either but it is a bit of a a problem that we have these two lines the four branch and the five Branch side to side we have to sync these all the time some people are already mostly in in the five branch and then they have to make sure they don't cause trouble um so yeah it is we're losing some time by um by having both side by side so it's in our best interest to to get this finished uh pretty soon um we are it's now an effort of let's say about five people Simon is that right who are actively contributing to easy bu five development I think it's it's you me Jasper Bard Mikel is yeah a little bit as well uh Adam is there in meetings to support us it hasn't been doing Hands-On stuff a lot so it's it's down to like five people that's part of the reason it's taking a while of course there's a lot of things we want to do um and not yeah basically none of us have a lot of time to uh to work on that so it's taking some time but we are getting there Alex yeah Alex is another one break you're trying to break stuff yeah to make it yeah actually you've been helping as well you're not a maintainer but you've been actively helping um with some some maybe more yeah maybe some more controversial stuff or some things you really want to see changed and Now's the Time yeah but okay yeah uh you you'll get some time hand hands on this afternoon as well if all goes well um to play with easyb 5 um so that's when you can start discovering it and letting us know uh how much you like it up until now um another challenge and that's maybe a bit personal or specific to gent and actually I'm not sure how specific it is so that's a question I have for for the people here and also online um is keeping up up with um software installation requests we get from users so the the graph there on the right shows how many installations how many installation requests we get in HBC team at G University per year so that used to be around 200 per year and all of a sudden in 2022 that almost doubled to about 400 per year um and that's a that's quite a challenge to um to keep up with that we were hoping that was a temporary thing but 2023 was pretty much the same thing and for this year we're already well over a 100 request as well so it's not really slowing down it's maybe even speeding up um a bit so that's a challenge going going forward I think to some extent that's us shooting ourselves in the foot we've been screaming to people like if you need software just let us know and we'll install it for you and also like if they do a module Avail on something that doesn't exist it tells them open a software installation request here okay it's part of the service I think we offer next to keeping the systems running um but it is a big Challenge and I also feel that the stuff that people are coming up with is more and more difficult it's not like one thing anymore that's missing a dependency it's one thing that's missing five dependencies which in turn are all missing three dependencies as well so the stack is becoming deeper and deeper and you're only seeing the top of the iceberg when you get the request that that's my question yeah is that requests or request including the dependen that's only the requests okay yeah so the numbers that's like if you factor in weekends and and leave days that's about two per day that you get that you have to install there's just no way you can you can keep up with that we're we don't like saying no to these requests but every now and then we have to say look this is no this is insane um and what we're also doing and we're we need to check those numbers again um we see that a big amount of the installations we do are never used which is even worse you spend all that effort into it and then they never even bother using it at least not yeah we only check okay yes that's true but we're looking at what we have installed some not not everything e has in the earlier slide you know you talked about and you mentioned me foronda and Apper one of the bits I'm putting in for at least some of the software we get requested is understanding what the use of wants to do with it and a self-installed cond environment for a single node bit of software that they're only going to use a couple of times is a perfectly valid y way of talking them through how to do it and giving them a better understanding for at least the ones that can follow instructions yeah do you have a structured way of doing that so is there a structured way of people asking I need the software installed and they have to indicate like I expected to once or so we don't structure the request it's a large junk of it is I know some of the repeat people who ask for software asking and over again some of them I'll go we'll do this method now and re revisit in three months time if you're still wanting to use it again and again then we'll move it more centering it often Depends for the bits of software where the only Ed install method is cond y um that's one of the times I often start looking at this because it's sorry it's K and I open their environment cond environment file and it's like 500 items in there and I'm going and I'm looking at the thing going I think my user doesn't actually want this more than once so we have a summer intern that actually did some scripting to try to tie the knot a bit more between cona and easy build like if there's something that's supported in con already it will just do a bunch of autogenerated stuff that you can then do that installation with easy build as well we should maybe try and get that out there as well and see if it's something that's useful for the broad Community yeah there a question in the back here not only so much of a question but also comment we see similar things that users are requesting a piece of software and I look into it and I think oh oh yeah that's that's easy and then I go down that rabbit hole and I go down a different rabbit hole and all of that and I was wondering if we see something does it make any sense to say okay I just put it on the on in slack and say I want to install arrow for example which came up the other day and is anybody seeing a similar ticket and shall we just join an effort so instead of me doing 10 easy config files very easily and spending you know two weeks on it it is only me doing say five and somebody else is doing another two and somebody else is picking up the the other try and coordinate the efforts yeah so but is is through slack the best the best channel for that what we have in in the VC in the Flemish supercomputing Center we have a Central Public GitHub repo where we manage our requests and it's mostly gent and Brussels doing that Len to some extent anwp a little bit as well so it's not as well adopted across all the sites but this is how we communicate the work to the inits um consultants for example we open issues in the GitHub repo and they pick up on those and even push their draft easy confy fast to a branch to a whip Branch where we can easily then help them out if they need help or where they can even reach out to the easy build community and ask help because they're the draft work is already um available publicly maybe that's something we could or should shift centrally into easybuild um and set that up so like a more sandbox environment where you can share your stuff with with the community yeah if you have concrete suggestions there York maybe we can we can follow up on that through uh and Thomas is saying maybe easy is the solution yeah yeah that's that's that's my next bullet point yeah um yeah so easy is definitely part of the motivation for this uh for the increase and uh and requests um another challenge L goal for the coming months years even if you take into account multiscale which runs until the end of 2026 is whatever we need to do um in easy to help in easy build rather to help the easy project sometimes we need to implement additional small features and framework to to help us out a lot the the from commit thing the new the new feature that was added in Easy 491 where you can pull easy config files very easily from a particular commit not just from a pull request that's one of the things that was motivated by uh by easy so sometimes there's a little bit of work being done there that's maybe most most ly relevant in easy and not really beyond that which takes away some some time from uh from other development work or maintenance work um again maybe a bit personal but I expect other people to hit this quite soon as well is support for real n um now real N9 is not that scary we've already built hundreds of things with easy build on on the real n system so it mostly works at least if you stick to sufficiently recent software versions if you go too far back in tool chains even 22b and older it's becoming very hairy for real 9 so take that into account if you really want those old installations you may have to do quite a bit of work uh to get that working um the main issue there is open isl3 so and R 8 you have open SSL one one something in r 9 you have open SSL 3 which is an easy build we mostly use open ssl11 as a dependency so you need to introduce a hook to translate that to an open SSL 3 because you don't really want you don't really want to install op ssl11 from source with easybuild it will happenly do that and then just flat out going to ignore your system openl which is being updated for security issues so you don't really want to do that um we have a hook in in Gent now pretty recently which we should probably share with the wider um Community to uh to do that I'll add it to to the framework on trib and maybe even send an email on that so people are aware uh what we're doing there exactly but that's yeah that's going that's going to be a bit of a challenge and don't actually expect a huge amount of effort there are some software projects which only fail on rail 9 because of a newer gilp C or because of open L3 or other other things that are a little bit different there so i' I've run into a couple of those uh where some of them I still need to fix but so there's new problems there new operating system um and then related to both easy and r n um and we've actually been seeing this for for a while already we should really look into it is um to revisit the environment or environments in which we're testing mostly easy easy config PRS um C7 was already mentioned maybe it's still relevant enough that we should be actively testing for that um the only test environments that we consistently use today are the two test Bots we have which is generoso running at cscs that's a rocky 8 um virtual cluster and the JSC zen3 system in Yul which is a virtual Rocky n cluster both of both of those are used quite well but we're using them be so we're not running stuff in containers there or really controlling the operating system environment we should probably be doing that um so we can more flexibly also test for C7 or um easily test for Rocky 9 on on both these um we should also be uh actively testing stuff already in Easy confli PRS in the easy build environment which is a bit special it has arpot linking which is again going to be very relevant very soon for easy vol 5 but there's some other quirks in there that sometimes we run into installation problems too late and easy that could have actually already been easily fixed and easy build through that original pool request um we probably don't want to make that blocking thing so if it doesn't work easy that's probably not a good enough reason to block the PO request from being merged but then at least we're aware that there's an issue there we can easily get back to it uh later when we want to add this to easy as well um and then looking for specific features like the module only and the sanity check only command line options um those actually work pretty well but they don't work consistently and the only reason to make the only way to make them work consistently is by actively checking both these options because they're a little bit different um also when we're uh testing easy cic PRS now they're both quick they don't do a reinstall or anything they just regenerate the module and rerun the city check but sometimes that's broken and if it's broken it's usually a bug in the easy block not in the easy conf but then at least we know that there's there work to be done there yeah I put my foot in now um IP version six how much do you test that on I version six not at all because some of the cloud providers are I think it is Amazon they are only using IP version 6 now we internally in our cluster are using IP version 6 now completely so there is no IP version 4 yeah and how to put that politely I think the way forward is we have to test it like it or not because it is actually now happening a bit more than the last 20 years even internally in clusters internally all our compute nodes have own an IP version six yeah IP address and how often do you run into trouble with that like installations or tests not working installations it is um open MPI we need to have a newer um uh what is it ucx ucx yeah 112 or when I updated to easybuild 491 I actually had to switch to ucx 113 for whatever stupid reason I couldn't work out but then it was working so here particular with the easy project we should really bear that in mind that it also works yeah is also relevant for you okay or or you just see it as important enough that we should pay attention to it yeah yeah well this so easy and the operating system we test in the types of CPUs we test for as well it's now x86 only we probably need to start paying Clos to attention to arm there as well and then let's say the special factors or special aspects like IPv6 we canot test in a h environments right so we need to somehow have a like a yeah a good selection of um of configurations that that Mak sense now for me and not everybody agrees there for me it makes sense if we if we test easy if we test in the easy build environment that we do that on an arm system because okay we're combining two different things there but if any pops up there then we know it's either arm or easy that is causing the problem if it's working well there it's pretty likely it will also work on x86 because we testing it in other places and finding a good combination of these things is maybe the best uh the best approach I'm not sure where we would add IPv6 in that mix but we we need a system that's configured for this as well yeah yeah we're heading into lunch break so I'll fin I have two more slides the spoiler slide which I don't want to spend a lot of time on on easy Bo 5 sure many people have questions there what's going to happen there and when um all I can say for now is that we have a lot of we have done a lot of effort on this already we started April last year so about a year into this we've merged uh a couple hundred PRS there across framework easy blocks and easy confli so we've made some very good progress um there's some main breaking changes there python 27 will no longer be supported you'll need at least python 36 and at least LM mod 8 if you're using El mod um internally we've changed some stuff Simon is going to um talk about all the details the defaults out of the box will be a little bit different so easy will behave a little bit different but you can reconfigure it to go back to U what easybuild 4 was doing by default if you prefer that um we're working on improving the consistency of naming of e config parameters and configuration options but we're going to do that in a non-breaking way at least for easy Bol 5 so the impact of that will be pretty limited um early on and that that impact is only going to come later and the tool chain support policy which I already mentioned EAS Simon is going to talk about that this afternoon what that means um so what kind of uh policy we're adopting there from Easy Bol 5 onwards essentially so more on that this afternoon and later this week as well uh or if people have already some concerns or questions feel free to mention them uh either online or over lunch and then longer term that's my last slide um keeping up with incoming contributions that's a clear challenge that we have not for the short term but for the long term hopefully we'll keep we'll keep getting as many contributions as we do now and hopefully we figure out better ways of uh of keeping up with them maybe becoming a bit more aggressive with that is is one one approach so uh PRS that are say older older than one year maybe deserve a bit more uh uh yeah let's say a bit less flexibility to keep them open because they're unlikely to be still be relevant um support for arm and support for AMD gpus and rock is definitely something we'll have to look into not only because it's coming but also because it's one of the things we promised in the multiscale project so we have a task there on what we call emerging system architectures to me both of these are still emerging the R1 is definitely an ongoing um ongoing work um and we've discovered some issues there already probably not the last ones but uh it's ongoing um AMD gpus will will have to get into in the coming months at least in the scope of multiscale um I didn't mention risk five here yet but that's one we're also actively looking into um already at least in the last couple of weeks uh we've done some groundwork for that um so that's coming too and yeah things are going to get more diverse and we'll we'll have to figure out a way to keep up with that and do the necessary development work to make that supported um and there's probably other goals and challenges that you think we should have um if that's the case let us know this week or open issues talk about it um so what's important to you or what are you really waiting for what you think is really should happen hopefully also what do you actively want to help out with um yeah this week is all about that so let us know if that's the case yeah and that's all I have for now