Pursuing KDE Neon Core

Everybody take their seats please. Okay. So the next talk is by Kevin and it's on pursuing KDE neon core. >> All right. Thank you. Thank you and welcome. Uh so this will be an update on the KD neon core project from uh last year. So for those who don't know me, so I'm Kevin. Uh I've been cont I've been using the KD software at the end of the 20th century and I started contributing at the very beginning of the 21st century. Um and uh well obviously I fell in love with the community which is why I'm here. Um and I've been doing things here and there uh nowadays so I work for Inyoka couture um where we are doing so we're consultancy we're doing development trainings uh coaching all of that uh and that's how we actually bumped into this project uh that I will talk about and I'm a remote worker um living in Tulus uh so first uh I will have a quick recap uh from where we were uh last year. So on the previous episode on that show um we talked about Ubuntu core uh which is as they say on basically their website. Uh it's a minimal secure and strictly confined operating system. Uh it's been mainly designed for embedded systems. That's basically the immutable version of Ubuntu. Uh supposed to be secure by design, containerized and so on. All the components are uh provided as snaps though uh even the kernel the snap demon uh are provided as snaps uh and so you get overtheair updates with rollback for any components uh in there. Um two years ago they uh announced uh what they call to core desktop uh which is the fully containerized uh desktop. Um and that's basically a full desktop session on top of uh Ubuntu core. Uh the idea is to have similar benefits but for userf facing uh with better security uh stability because the updates you can always roll back and you don't have the system in an unstable state. Um it's easier also to audit a system as long as you know uh the packages which are uh in there. Uh and there's no inconsistency from system to system. Uh obviously it comes with extra challenges compared to the full embedded uh use case. Uh it's harder to draw the boundaries between uh the desktop components uh themselves. Uh and it tends to use more um storage kind of due to that. uh on our side. So we have uh KD Neon um which is built on top of the latest uh LTS release uh with the issues which come with that that um Arold mentioned last year. Uh the idea being to showcase KD exactly um as we KD developers intended it. Um and so no patches uh no changes on the default settings. uh so rolling ID software on top of a stable Ubuntu base um and you get very up-to-date experience um so the idea uh with the KD neon core project was to follow similar um idea so have the same benefit but on top of Ubuntu core instead of regular Ubuntu um so it has a similar architecture to the Ubuntu core desktop uh as um planned by the Canon code folds. Um you get the plasma based user experience uh latest uh versions that the idea and we provide the building blocks to uh package uh KD applications uh as a snap for use inside of Ubuntu core uh which leads us to where we were uh last year. So uh the when I presented last year that's pretty much the architecture uh we had. So I'm a bit too short to point to blocks. So I hope you will be able to follow along. Um but basically so we covered as part of the project uh a few applications all of those using what they call a content snap um to um to have the uh KFC execute runtime. Um and then for the session uh you would have the plasma desktop session snap which is a regular application snap from the uh perspective of snapd uh and this one is basically empty getting all its content for the base snap which was uh just under so the plasma core 22 desktop one um and so the applications and the runtime on the top right corner they are built from source but the content the sorry the base snap here was built from the packages coming from neons okay so we're reusing the packages uh as is um and so we ran the full session uh confined uh we had sdddm which was not confined though I will come back to that um and the other thing we were providing as I mentioned that was the building blocks for the apps Okay, I won't focus much on this part this year because that's basically that stayed as it was uh since last year. Uh we basically updated uh everything which is mark core 22 there has been updated to core 24 but that's uh the most of the changes which happened uh since last year. Uh we also had some challenges last year uh with quite uh some of the tooling uh which was unfriendly to rootless containers um to say the least. Uh so we had a VM on which we had SSH access. So in a way uh that was coordinated with the KD CIS admins. So in a way we helped to have the blueprint uh for the uh VMs that we have no uh which have been uh deployed by the KDC admins. Uh this year uh we also add GU tests which I mentioned as a whip up there um on the CI path to validate the applications. Uh so that's something we completed but I will come back to that particular part of the story uh a bit later. And also so we talked about uh how the snap confinement works uh including how the interfaces are implemented with sec and apparmo. I won't get back into this. Uh one thing I will get back um into is how we integrated the systemd units which are necessary for the session uh into all of that. Uh that part changed quite a bit. Um and also uh we've seen some of the problems which were created by talking to systemd at startup. Again I will come back to this uh because we had a specific interface for that inside of a snap dfork that's still there. Um and I also had plenty of stories of how to or broke queen and get black screens all the time basically. Um and we had some patches to uh snapd and so black screens a lot of them. All right. So what's new uh last year? So I presented some of the challenges which I uh mentioned there of building all of that. Uh there were quite a few uh but I also gave ideas on how you can debug them. Uh and where the important in integration points were. uh today I will f focus mostly on what changed during the past year and the new challenges opportunities coming with that. So the obvious big change that the reworked architecture um this was the obvious big topic uh and I closed my talk last year on that uh and we had a different target architecture uh in mind. So let's have a refresher. So that's the one you've just seen. Okay. Uh that's what we delivered uh last year. It was not ideal. Um, and well, we had to go with that because of various constraints uh that we didn't control. But where we wanted to go, it's uh this one. So, um, we wanted to move everything out of the uh base snap basically. Okay. so that we could reuse the uh regular core uh base map. That's uh the intent because we wanted to do this. Um that meant that the plasma test session right if you look at before it was depending getting all it stuff all the binaries from the base snap. Okay, if we reuse the regular base snap we cannot do that anymore. So we need to have a content snap uh just for that. um to provide those binaries so that the other snap can do its job. Um so we wanted to decouple from the base and we wanted to be able to slice things up uh a bit more because now we got the SDM application snap which appears in there. All right, so that's the target. That's uh where we got in 2025. So it's a bit of the seventh difference game. Did you notice that? I do it again. You see them? There are two main differences. Uh so the content snap actually uh actually is there. So we really do that. Uh but if you look so we didn't get SDM and there's a reason for that. I will get to it. And if you look the core 24 we're not quite using it, right? So we are using a core 24 that we are not making okay which is made by the canonical people but that the core 24 desktop one okay so that's the one with GDM and all of gnome and so on right which is in there um the reason why it's because we don't have sdddm yet we need to have a login manager okay and so well if you remove SDM of the picture then you ends up with having GDM because that one they already provided for free for us Right. So that's kind of an intermediate step. Um and there's another reason which I will explain further which is uh on the fact that we uh can't do this bit yet um for SDM because there are missing facilities on snapd at the moment uh that we um which which prevent us from having a proper uh SDM package yet. I would get to it. Um all right. So the impacts uh well that just doing this though even though we are not at the end uh at the target that we had in mind uh that brought some changes and impacts uh throughout the product. So we will talk about those now. Um one thing is the relationship to KD neon right because as I mentioned last year we were basically using the um web packages getting from the archive for everything. Uh but now if I get back to this you notice that we use the content snap. Okay. And that content snap uses the uh the KF6 runtime. And that one last year we were building and we're still building it from scratch, right? So by doing this we end up kind of having to build the um content snap for the plasma binaries as well, right? Uh that reduces quite a few risks uh doing this. Um and so that means we kind of uh parted ways with the KD packages in the Neon archive. Okay, we're are building the uh KD stuff uh on top. Uh that's more convenient for several things because uh as I mentioned potentially different API because we're using the uh build KF6 uh and also we are changing the prefix and the de packages they kind of assume that they are in / usr. Well, so we could get more control uh this way. We're still using the KD neon archive though but for the dependencies, right? because those dependencies were some of them uh that's for um updated they're updated faster. Uh so we could almost cut the ties but um it all dependencies outside of Q10 KD they bumped quite quickly in a way right uh in a matter of month since the release of Noble u we sometimes require new APIs in those dependencies okay uh this doesn't make it easy on a stable base with a limited set of official blackports it's not quite a surprise that's exactly what AR was complaining about about with KD neon in the first place. So no surprise there but that just shows up uh in a different way here. Uh so the question obviously is could we manage to bump uh dependencies less aggressively? Yes. No. Right. That that's thing uh we could think about right um like for instance queen already depends on a very different uh deep display info. Is that something which we could have been more diligent in like supporting both APIs for a while before breaking this or do we want to move faster? So that's something we have to figure out I guess uh as a community. Um if not I mean if we want to keep the doing it like this then it's kind of convenient to still have uh such an uh such an archive uh and I would lean toward keeping it for this kind of extended back ports because they don't exist in the official uh U backports uh but probably trying to keep it uh minimal at least from the Kadin neon core perspective. All right. And so the problems we got uh well plenty of course we ch that's a dramatic change right we uh did three things at the same time we bumped to a new base distro uh we move to a different prefix and we started to compile everything uh ourselves right for the session uh so that's quite a few moving uh pieces at the same time um so the first one is that obviously I cannot find my binaries um And well some stuff like the Q10KD stuff you just set a few environment variables say there there it's not in USR that's fine it's there and it works right there there no big surprises there. Uh some of the dependencies of all of that are a bit less nicer in that regard. Um and so for some of them if you dig in the code you might find some environment variable but which is supposed really to be used only for development purposes. So you can't really rely on that. Um or you can use an extra feature of uh snap uh of the snaps that we didn't quite use last year uh which is the layout where you basically create bind mounts from the perspective of the confinement or sim links which exist only in the confinement to present the right path for uh those dependencies. The other one was well we moved our systemd services again so now I cannot find my systemd services anymore. Um so we needed a way for systemd to see them. Um so to bootstrap the session there's active wrapper script which is provided inside of the core 24 desktop. So we're basically doing that very ugly uh that very ugly trick where we basically expose them uh in some way. we see link to the right place righting one of the folders which is not used and then we uh restart uh systemd now it goes a bit further because um as I mentioned last year um that was less of a problem um because of the packages were providing enough information but we don't quite control the systemd services um because they are generated by snapd Okay. Uh we so the content the the names okay uh we control inside of our content snap. Um but they don't map to anything else uh on the snapd side. Okay. Because they're all generated with snapd some ID and so on. Right? So we can predict those but uh we cannot um we cannot provide them because they will be created at runtime by snapd um and that's convenient for us to still use the name or names right the name of or services right so inside of the content snap we're basically killing all the services uh files which have been uh created by or build and we replace them with sim links to places where we know that snapd will create them under another name right so all our services file disappear and instead we have aliases against the snapd ones now the problem is that snapd when it generates them uh it does a very limited job so it doesn't give more advanced uh relationships like the one we use like for all services we're saying they are part of the um graphic uh the graphic t graphic section target and that's gone right so then when you boot and especially when you exit that's very interesting because you have services which lie around uh so next to the sim links I just mentioned we also have over override files to be able to compensate uh compensate for that and inject basically what uh what we are missing that's all to try to bypass uh limitations uh on the system side uh we also had surprise with the desktop portal. Um, and we had from the beginning uh most problems uh with it um and well it's kind of sensitive to startup conditions. So we had to have some extra care for that. um especially because we're in situation there where XDG desktop portal is not confined but XG desktop portal KD is confined right so that can that comes with interesting changes um and so it started after start plasma pushed its environment to systemd via the interface I mentioned last year uh and so it tries to find libraries uh and config in the wrong place, right? Because Exop portal is started outside the confinement. So it doesn't see the layout changes we did. It doesn't see like the build we have right with the libraries in a different prefix and so on. But because start plus map pushed the environment there, it's trying to look there and it's going to well that folder doesn't exist, right? Uh so we again had to trick uh trick it uh a bit uh injecting an override uh file just for that uh and to be able to start uh the XDG desktop portal while visiting the environment just for this one. Also we ship inside of the session a couple of applications. So think system settings, think K info center. Okay. So for those we have desktop files. Um and those desktop files uh where they we build in a prefix right but then inside um because of the snaps you can have several versions of them. Uh you don't exactly know the path you cannot outcut the path inside of the desktop file like we do during our build right you cannot do that. So instead you have to say well I need the exact line to not be a fixed path but to depend on the environment so that it actually picked at the right place at runtime. Uh so we have that's why we have that bit of script there to patch all the desktop files which are provided with the session to say look uh it's not uh where you think it is. It starts from the um uh snap environment variable and it's somewhere in there because that one is tied to the actual snap version which is running. Next to that uh we also had to have some uh improvement to snapd itself. So I mentioned already the patches that we did last year which were a bit standing outstanding right now there apart from the uh systemd user control one they are all merged into uh snap snapd upstream uh so that's uh that's uh a good thing but we still have that crucial one uh to be able to uh talk to snapd uh there's been also other changes which were more driven by uh canonical um because they h there's plenty of services inside of the core 24 desktop one they wanted to move out for instance pipewire to get it out of that base snap. Uh but if you do that and you start to confine pipe wire then you need the session to be able to talk to pipewire for the screen sharing. Uh so they changed snapd so that there's a specific interface now for desktop sessions to be able to talk to pipe wire. So that's kind of where they are going. So they are trying to get udisk 2 out of the base nap as well and so on. So they are removing more and more stuff from their uh base and we have more interfaces to support as a result. Uh the systemd uh user control didn't make it upstream uh and there are more changes which would be uh which would be necessary um to complete the transition. Okay, that's what I mentioned in the beginning where we are not at the target yet. That's mostly because of those uh those things uh like we would like to be able to properly describe the service services dependencies so that we don't have to do the ugly hacks that we are doing throughout right uh and we there's no way to do that. Um, we need uh a way also for a snap to be able to start another one because that's exactly what SDM does, right? If you confine SDM, then you're inside of a snap. The whole point of a snap is that you cannot trigger something else, right? Because then that would be a problem, right? And SDDDM is exactly in that situation, right? Or plasma login manager or whatever. Uh, so there's a need for some mechanism to do that. And that's something which is uh still on the drawing board that they uh didn't implement uh yet. Uh there's also uh some issues potentially with the PAL handling which again SDM needs right uh and you need to be able to escape just for that and we're missing interfaces there. So that's those two things which kind of prevent us to get all the way to where we wanted to be. Uh so what else happened? Um while there has been some coordination uh through the snap store right because we submit to the snap store that coordination is generally uh a bit slow uh it adds quite a bit of delay uh to things because they need to check which new interface you use and as I mentioned we had to use new interfaces that they introduced and then they're very picky but why why did you introduce that but you introduced it first for us now I just want to make use of it right uh now to be fair. Uh it's kind of their job to make sure that nothing bad gets in, right? And tries to get more privileges than it should. Uh so it's normal that they actually check that. But sometimes uh it can be a bit frustrating because that makes a longer feedback loop. Um one other thing which has been a bit uh of a problem uh for us that the selenium web driver spy uh bridge it's a great idea on paper. Uh it has been a constant headache in practice. Uh the Kate people probably remember that um because we submitted some APM tests for them and it was breaking so often on the CI that we had to remove them. Um so the thing is because I'm doing other stuff in other parts of the industry. uh it's kind of a constant uh we see on other projects I mean professionally outside of KD that the sel selenium test they tend to be very very flaky right um and that's not new right um and it's no wonder why on the web space play is actually way more popular than selenium at that point uh well there's nice tooling around it and so on but the core sell point of is that they are not flaky. Okay. Uh and that that's uh that's a big change. Uh it's way more reliable. Um we also add so more ways to engage with the uh Ubuntu community and the canonical forks. So I could advertise what we've been doing because uh we talked about it at academy that I could advertise it at the Ubuntu summit last year. Uh and I also ended up in some um architecture body around snapd uh with uh with the canonical folks to flesh out some of what do we need from a session um from a sessions perspective and especially from a plasma session perspective because the needs are not exactly the same than the gnome one. So I brought that extra perspective to poke uh hopefully in the right direction. Um, we got all the way to having a basic installer. We have ISOs and so on which we didn't quite have uh last year. Uh, I'm actually happy that we helped a bit the KDC admin to inform uh on the CI changes uh for the transition. Um, and I'm coming also with a with bad news. Uh, it's currently the project has been at least funding wise the project has been posed. Um and that's mainly because Canonical changed priorities uh internally. Um and why that impacts the funding because we are basically uh funded by Canonical for that. Um and that means that the snapd changes I mentioned they are not going to appear very very fast now right um now I think it happened. So that's my personal opinion on that. Um maybe the conical folks have a different perspective but I feel it happened in part because the needs of the desktop obsession be it the Gnome one that they were working on their side or the uh plasma one that we've been working on is actually very demanding on snapd right that creates as I mentioned right uh having the right separations between the components that's there's way more of those as soon as you're making a desktop and there what not necessarily That's a lot of interfaces to introduce and things to design and so on, right? And so I think we got to the point where the snapd team was like saturated, right? That's like drinking from the fire hose with ah all those demands, right? Uh so I think that's in part why they need that pose to swallow some of that, right? Because they also have other demands for the embedded space, right? Which are different and so on. Um so no now what right so I don't want to end this on a bad note like this uh so the inoca economical joint project is posed but everything we did is basically available uh so what could we do with it right where could we go from there as a community so I think there's a path forward um I think we can we're advanced enough that with a few compromis izes uh we can actually get to a full product which is not that far from um from what we wanted. The obvious sacrifice as you can see coming is the login manager. Okay. Um and we could get to something where we have something in between the 2024 and the 2025 architecture. Okay. So instead of using the core 24 desktop base, we could get back to our own base, but we could make it way smaller uh and we would have to accept to not have a confined uh login manager, but then the rest uh would be fairly stable and safe, right? So the blueprint blueprint for that would be so to get back uh the on the gadget and the bay snap that we have uh but to change them especially the base snap and to make that benap extremely lean. Okay so just the necessary to bootstrap the login manager not even containing the login manager but just having a bunch of scripts on top of core 24 to bootstrap that without the confinement and then you can still ship it as a snap. Okay, but you just don't confine it. Uh I have a couple of essential services uh like the ones for which they didn't introduce uh interfaces yet and especially XDG desktop portal and then we can produce a snap for plasma login manager for instance when it stabilizes and run just this one unconfined. So, it's a nice prospect, but there is a but um it's only worth it if we can get Canonical to accommodate us a bit uh for some things. Um because if we do that, that cannot work with the upstream snapd, right? If you remember last year, if you remember now a bit earlier, I mentioned that we're running with a snapd fork. Okay? Because for instance the systemd user control interface needs to be there and this one didn't make it to upstream. There's a couple more patches that the can people did on top of snapd for desktop specific fork that is used by the nom session that we uh depend on uh as well. Uh and so that means that this fork needs to keep living right and it cannot fall behind up uh upstream snapd. So we need to get those patches rebased once every I don't know few months or something like this. Okay. Um and the thing is well they posed right so that fork is dormant as well so it risks to fall behind. Uh so we would need to have an agreement between KD and Canonical to allow us to actually do just that maintenance of rebasing the stuff from time to time which is not necessarily a very big and horrible job but that's still some work that needs to be done from time to time. Um I think that would be a good way for KD to be proactive there in the relationship uh and to show some goodwill uh on the community side. Now would can accept that? I don't know right we have to ask and see right um for me there are good reasons to try this and p pursue such an effort uh because it opens the door to parts of the industry we have troubles reaching today I think um and um basically any Ubuntu core device uh out there with a screen could benefit uh from uh from something like I'm proposing we could do. Okay. Um and I've been surprised when I got to the Ubuntu summit that actually some of that happening um and we could get something usable anywhere plasma could be usable so that TVs point of sales digital signage uh some industry specific devices and so on. Especially for the embedded industry case. Uh that's something where it used to be like very simple ones screen stuff and they shoving more features into this kind of stuff where they start to figure that yeah I would like to have a window manager and stuff like that. Um we're already seeing some of those benefits actually even if that's uh small admittedly. Uh but there's like on the S SDK sides of things uh there's uh and some of the apps there's actually some interest. So I checked on um less for the apps but more for the SDK uh I checked for the use um so not just the runtime which any application would put but really the SDK that the developers would need to uh to track that stuff. There's actually more deployment because we got a dashboard on the snap store. There's actually more deployments of that than than I expected even for the ARM 64 ones that uh Carlos made last month I think. Um so that would be interesting to see uh what we can uh get out of it. Uh I actually have been in touch with uh so one company which is uh interested in in that um and well I'm trying to see if we can get them as sponsors and so on. So I will end on a few lessons learned uh there. Uh so one thing is as I mentioned why we might want to reevaluate our GU testing story. Uh so at that point I'm to the point where I completely doubt the viability of selenium uh for use unfortunately um the even though we didn't get to the target the architecture we have this year feels way more robust than the one uh we had last year uh so well I had surprises to move from one to the other but that that's a net improvement um and of course we depend on canonical to make progress in the Ubuntu core face duh surprise right uh but that's uh kind of of how it is um and I think there are opportunities there if we want to seize them um if you want to contribute uh so the code is at the same place than last year uh so the first one is for producing the images second one that's uh a group and under that there's all the snap packaging uh happening there's some documentation on the wiki and of course you can come uh talk to us. Um I also uh would like to uh thank again the uh team on the Enoka side and Scarlet uh and and Carlos who did good work uh with us as contractors. Uh there's above uh which uh Arold mentioned and I will mention it as well. Uh we are doing a both about everything which is immutable DRO uh related be it KD Linux uh Fedora Kinwat Kinwat I'm not sure how that's pronounced I just read it usually and KD neon core so that will be Monday uh 2 pm uh in room one. Thanks for your attention. I think we might have time for maybe a question or two. [Applause] Who has a question? How come all your systemd stuff isn't an already solved problem for your Gnome desktop? >> Your systemd issues that you had. >> How come they're not solved problems already for the Gnome desktop? >> Very simple. They don't use systemd to boot up their session. >> No, there. So I think I I'm two steps removed from the Gnome side of things, right? Uh so I'm not sure if that's because that's a version that they still use in their uh base snap which is before uh the move on the Gnome side. Uh but all the calls I had with them that was like uh yeah that's a problem we are going to have as well, right? We don't have it yet but we are going to have it as well, right? So they know it's coming. Uh, it's just that to get their base map with the version they were targeting, that was a non-issue back then, right? Any more questions? Got time for like one or two more. Nope. All right, then. Thank you very much.