3.4 General Q&A and day 3 and workshop wrap-up - HPC/SciComp Kickstart summer 2025

memory. So maybe Richard [Music] um again now you can hear me. Uh yeah, I'll switch to the notes. Mhm. And so as usual at the very bottom there's the feedback of the day and not just the day. I guess this should be for the whole course. Mhm. Do we have two feedbacks? one for the day and one for the course. If someone wants to duplicate it twice, you can. But usually it's just one. Oh, let's go. Um, at least for this part, but then now for general Q&A. So, now's the time when we can all be here and we'll just reflect on the course itself. Um, yeah. So, who remembers at the beginning when we said that it was impossible to teach everything that was needed in a um what's it called? In a 10our course. Yeah. So, I still think that's true. And I mean here in this course only like and we've can only had time to give you a summary have do some examples and so on but by no means do we think that this course is the end of what you need and all it's just a starting point. It basically hopefully gets you started lets you realize what's possible and and have had the initial experience with stuff and now um you will know when and how to go read for what comes next. What do the other instructors think? Was it Yeah, like um I I I think like I I don't remember like when was the last time first time I I was doing this course back in like 2016 or something and like the ecosystem has changed dramatically from those times like the like if like and don't be afraid of like like thinking that okay this is a lot of stuff happening because it's a lot of stuff happening for everybody. uh like like back back then it was basically like we wouldn't have even the like let's say the GPU day at all like basically because the GPUs went like they were being utilized but not in this sort of a level and and many of the concepts like AI and whatever they went like things that are happening constantly. So uh there's a lot of stuff currently happening in in the space like the computational space is getting uh very complex but at the same time it's it's I would say that it's good to differentiate like it's like like if you have certain things that you need and you you want it's good to like know that okay I don't need to necessarily know everything about everything. uh like you can you can focus on on learning those things that you need for your work and then learning the rest later. Yeah. Yeah. So some of these questions the first question is about console and how to get started. What would you all think? um depends on your time limitations. Uh I would suggest uh coming in on one of the console focus days that we have in garage because then there are plenty of experts here uh that have a lot of knowledge with console and and also a lot of detailed knowledge on how console exactly works. While if you come in on another day, um we do have a few people uh that do know about console, but you might be unlucky and uh well not get the right person on that day. Um but it's it doesn't hurt to show up any other day and you're welcome to, but uh it's just possible that we might not be able to help you on the spot. And and like I would say that console is one of these kinds of programs that like once you get it sorted out and it works, it works like it's the kind of a program that like because it it has like this like you can submit it straight from the like the graphical user interface and then it just runs stuff on the background and you don't need to think about it. But to get it working, that's the problem. And and that's usually like um yeah, like it's it's good idea to ask for help because like it's not fun uh doing it by yourself based on um just just looking at the manuals. Yeah. Yeah. And it's the kind of thing that like you'll always find people that know about the cluster itself but might not know all the buttons in console. So if you can get it most of the way there and show us an error message that comes from the cluster, you can probably get help almost any day. But if not that then next question is also pretty good. So um a custom library which is not written for Triton, not available and the default instructions don't work on Triton. So um yeah, what would people do there? I guess that is a very hard what what's the name of the library? Like is there a public link or something? Is it a common library? But yeah, so I would say in in in general I would say yeah come to garage and let us have a look at it uh together. Um I would say it really depends on what kind of library it is. Um if this is something that is for example pip in so or is a python library then you can normally just do it yourself if you know enough about uh python so that you can just run the code from uh from a customly installed library if you don't also come to garage we can try to help you there um and if it if there are no really strange things happening normally we get the things running Um like but yeah yeah like quite often like especially for scientific coding and scientific programs uh like like for the whole ecosystem like the Linux ecosystem and stuff like that like the new ecosystem and and um everything related to that there's like standards that that are very like engineer focused but but they follow them very well. So they have like usually like if you have like code you have certain like configurations or configure flags or whatever and you can like spot the pattern and use that. But the unfortunate thing is that because it's so engineering focused is that like I remember one CMake manual like tutorial from from those people was like 500 pages. So it was like insane insane insanely long like all of the tutorials but it's very strict and very like regimented and everything is is where it's supposed to be. But when it comes to scientific programs like nobody has time to read those 500 pages. So you might end up that the scientific program it doesn't follow those standards like completely. So it becomes kind of like a translation problem where you have like okay this is what the the creators of the program uh like intended to be done but they use like different names for the flags or whatever and and now suddenly like you need to do this kind of like translation. Okay, what did they like this XY problem kind of situation? What what did they want to do and what what things the installation thing need and and like it becomes this kind of like guessing game. you need to go through the source usually and and check okay what what it actually tries to do and and uh sometimes getting like outside feedback on this it can be really helpful like somebody else who has done enough of the pattern pattern recognition uh to to to see that okay like hey they're probably trying to do this kind of thing and they mean these and these things and and sometimes that can happen with many libraries. Yeah. Or um even just knowing uh where in the error message you actually have to look for what the actual error message is because you get like three pages of outputs and um if you don't or if you haven't done this before, you okay this this is so much stuff. I don't even know where to look for things. And uh there's a thousand different warnings, errors, and stuff. And yeah and we we kind of have seen these things and we know which things are important and need to be considered and which ones are yeah this is standard output that it always prints but I would also say that I wouldn't discourage you to try like checking out the pattern recognition yourself like like but the problem like the important thing to realize that the problem often necessarily isn't in you. It might be that like somebody misheard something 5 years ago and now it's in the code. Like now it's like like there's a this game of telephone that happens with like programs and like sometimes somebody like doesn't follow a standard or something and then it gets baked into the code and everybody uses the code already and now it's like a standard in the code but you need to know about it and it's like hidden in a readme five pages deep and it's like like it's not necessarily your fault that you didn't recognize the pattern or you didn't know about this or you didn't know about the thing but but you can try like at least I do is that like if I if I see something I'm I'm I'll usually go back to the source and check like like okay how is this supposed to be written like let's say like how are they expected u these flags to be given and why aren't they given like they are and what what flags they want to given and that sort of stuff but it it becomes this kind of like uh like it's very hard to say how do you recognize these things like what Thomas said about the error messages. It's very hard to say how you spot the right line in the error message and and it it only comes through time, but it's not the problem that like you're you're wrong because you didn't understand it. It's it's the message is always can be bad. Like the error message can always be bad, but you just like plow through it at certain point. You just know to look for a certain place after you encountered it more more and more times. So it's like unfortunately it's sometimes like that but so I I would re encourage everybody to just also try it out like like just try reading it and try googling as well uh because after a certain time you get accustomed to those seeing those patterns. Yeah. And uh just where you mentioned uh this has become standard in the code or there is something that um is happening. Uh it might also be good if this is a project that is hosted on GitHub to just look for something similar uh or something where you think this could be related to uh in the issues because often enough there is an issue from 5 years back that um why the heck is this the default? Uh this is completely unintuitive and not documented anywhere and uh could this place be changed? Uh and yeah, it just never was. Yeah. And I have a personal example for that from a Microsoft library. Um yeah, and my comment on this question, when I see there's an install.py pi file. That's not the standard name of a package project thing. So if it was setup py or pi projectl I'd think okay most likely I can make a virtual environment or content environment and run a command and it will install it and mostly do all the stuff automatically. I might need to tune some but not but install.py makes me think someone has written a custom script that tries to install it. who knows where it installs and if I need to do it myself, it's probably going to be a lot of manual work and figuring out how they did it. So, of course, like if you're using this, there's nothing you can do now, but this is a lesson for the future. So, when you're releasing something, it's worth taking a little bit of time to understand how the project, how Python stuff is supposed to be packaged and so on, and then do that if you want people to use it. We actually have other courses in the Python for scientific computing course that's usually in the autumns. there's a session where we very briefly talk about the packaging standards and it's not that hard and um to I want to add on that uh to also to consider what should be happening in an in an installation script in Python and what should not like things where uh Python assumes that underlying system libraries are installed. So um your installation script should not try to install system libraries because that is always operating system dependent and Python Mhm. says no um I I I don't want to do this. This this is something that the user has to take care of in different way. Cond for example does a lot of wrapping around that and adds some uh things that are system libraries in the environments. Um, but don't try to do this in a custom installation script. You will just most likely uh make it extremely complicated for someone else to actually install it. Yeah. Yeah. Yeah. But like like again like Yeah. I I will quickly mention that the like again like but it's often like also high burden like to like think that okay this needs to be perfect and whatever like like like first time like like if you want to do something and you want to publish it don't worry if it's like if it's bad the first time it's like it's the continuous process of improving improvement that matters more so like I have published code that is really bad and I have I have like made repos that I still look back and and think that okay like these are bad but they were bad because I didn't know better but at that time so so like you will go through the process and and like that that will happen for everybody like everybody will publish code that is not using the standards it like it's missing something that you realize later that hey I should have used this and that and like don't put too high of a like also like a burden on yourself but in the next iteration know that okay like maybe I shouldn't like use the previous one maybe I should try improve in the in the next iteration yeah I I yeah I sort of say something similar so you don't have to be perfect because perfect is always too far away just try to do a little bit better each time you do something and then once you like if something if you do something often you'll get better at it and reach the level you need. So what um I mean okay so for the feedback parts here it looks pretty similar to other days. Um yeah so what what what should we recommend people do next? So what we covered was in the schedule. There's a lot more written material to review. Yeah. Like like at least I would recommend just trying out whatever you are planning on working on. Like it's like if you have a bunch of cookbooks in your bookshelf, but you never go to the kitchen, you will never learn how to do like cook food. Like you really need to go to the go to the place and start start cooking and tasting the food for yourself. And then of course use the cookbooks as this kind of like reference point that okay, it should look like this, but why does it what I why my food doesn't look like this? and then you're like you can do the improvement procedure but uh like I would highly recommend that you you pick a program or thing that you want to do in the cluster and and like first let's say you have a program that you you run currently on a on your laptop and you it let's say your heats up your laptop and you you can't have it in your lab anymore like it's it's too hot because it's constantly running some program like can you move that to the cluster and what does it require from you to to do that and and try out like can would it be possible that it heats up the computer in a machine room somewhere so that your uh computer isn't isn't overheating. So so try try like some program move it into the cluster and get accustomed to the workflow that you have you like basically put something running somewhere else and you don't have to worry about it anymore. Yeah, that's a good way to do it. What courses would you recommend people to follow up next? follow up next with well on the on the autumn we have the Python for scientific computing coming up again like that's our course and there's also the code refinery course coming uh at some point they are good ones of course yeah I think code refinery is mid September now. So I guess you'll get links to this in your email if you've registered. Python for and code refinery and Python for ski comp are a lot of the same kind of instructors and the same teaching style where it's live stream with the notes to ask stuff and lots of exercises and so on. The focus is more on the on the workflows and and how to use Git and that sort of stuff and then improving your productivity basically. Yeah. We discuss any feedback. Yeah, I think the feedback is is uh good and honest and and like yeah, it's always complicated with this course and especially with the time frame like how how to get everything into the into the Yeah. And there's like this thing we we struggle with for every course. So if you make it like advanced enough to be useful, then it excludes most of the people that could benefit from it. And it's basically saying do all these other basic more basic courses before you can get to what you need. But with today's academics and so on, like the academic system and the time pressures people have, people just don't have time to do a ton of prerequisite courses before they start getting stuff done. And that's why we emphasize the big courses that sort of set the basic level ways for people to study what they need themselves. and then also our help so we can help you with the specific parts that you need help for and you know sort of like one-on-one working together. So times up is are there any final