The State of (the) Union - Arjen Hiemstra

Hello. Can everybody shift across in their rows for PE so there's space for late comers to get in? >> Always happens. Thank you. >> Okay. Now the next talk will be Arjan with the state of the union. >> Yes. >> [applause] >> Um hello uh my name is Arian Himstraka. Um I have been a KD contributor for a while now. Um and have been working on a project called Union for also a while now. [laughter] Um and I will be giving an update on how that's going. So to start off with um [clears throat] let's talk about what is this union thing again because I'm sure most of you here know it's not like the workers union thing. It's it's a programming thing. Um several years back um I along with several other people concluded that we have a pretty nasty problem within KDE when it comes to our styling which is that we right now have four different implementations of what we call breeze. Um, we have the widgets implementation, we have a version implemented for plasma, we have a cutequick implementation, and we have a different cutequick implementation because of course, why not? Um, this isn't mostly a problem because yeah. Oh, wow. Okay, [laughter] that was that was the wrong button apparently. Um, >> what kind of language is this keyboard? [laughter] >> Yeah. I mean, let's I guess we're just skipping to the beginning. [laughter] Spoilers. Yes, I'm sorry. >> Yeah, where I took a beginning and again so I can take it back here. So, as I was saying, uh the problems with this being uh we if you want to change anything right now with regards to breeze, you are stuck doing four different kinds of implementations for that change. And I'm pretty sure everyone here will agree that you know doing the same change four times feels like a lot of unnecessary work. Um the other problem is some of these implementations require a lot of developer knowledge. Uh especially the cute widgets one. It is painful even for me to make changes there because it's not not just that it's cute widgets code. I don't like writing cute widgets code. I will admit uh but it's also that it's very specific type of cute widgets code that really requires you to know how uh the style the class called Qstyle works. So yeah, it's it's painful and uh if you have a designer coming in with, hey uh I have this amazing idea, they quickly get turned off because us as developers say yeah cool but sorry this requires four implementations and uh we need someone who knows how this Q style thing works for that. Uh so a lot of this gets bogged down in technical details. Um and one of the other bits about this is uh so we have one of our styles doing using cute widgets for rendering cutequick stuff and that's not ideal from the cutequick perspective because it doesn't actually make use of a lot of the features that cutequick has for its scene graph. which is actually how the other cutequick implementation came about, but that's a different story. All right, so this was identified like five years ago already. Um, but it's a very hard problem to solve. I mean, we got into this state for a reason. Uh, getting out of that state is a lot of work as well. Um, so at some point I did some thinking and came up with this idea about, hey, how about we define or or we we pull back a little and rather than having either a cute quick style that makes use of cute widgets or a cute widget style that makes of uh makes use of cutequick which was suggested at some point. uh how about rather than that try to define a base a core data structure that both implementations can use. So that's how I came up with this idea of having three layers. an input layer that generates a data structure and then a data structure that can then be used by output layers um to to uh style things based on whatever that specific part of the platform uses. Uh this I did some proof of concepts for that. Um it seemed to be working. it felt like a right idea. So, uh, last year somewhere we started doing a more, uh, production ready implementation for this. But that did mean we needed a plan because trying to build this entire thing where we go, yeah, we have four implementations and we throw all of that out and we have this new thing here which will be amazing immediately. Yeah, that's resite for disaster. So let's see if we can come up with some smaller steps that we can take so that we slowly can go into this state where yeah we have this implementation that's now become obsolete. Oh and then we have that implementation that's become obsolete etc. So uh our main f my main focus has been on cutequick as output building a cutequick style for uh again I much prefer writing cute code over cute widgets so it's partly personal preference it's also uh I feel a bit easier to some extent and we have uh a bunch of implementations on the cutequick side that we can make obsolete if this works out. Um, and one of the things I decided initially was to use plasma SVG theming as an input format. Um, because this was something that we already have and something that we know how it needs to look. So it meant that we simply needed to consume that and send it through this intermediate layer and then create a cute quick thing that outputs that uh both of those sides being known. So it meant that we could focus on that data layer. Um yeah so focus on the data layer, focus on that intermediate layer and uh later on worry about hey is this actually the right choice of input or output. So that's gets us to a year later about a year I don't recall the precise date of academy last year but uh a year later where are we now? Well, I'll start with some images. This is discover. This is discover running with union. And I think most of you will look at this and think that just looks the same, which is the intention. Don't worry because this is also discover. And it look I know this looks horrible. uh but this was a testing style I did to make sure uh to to give me myself an idea of how much control do I actually have over discover from the style at this point. So, uh, for those of you familiar with breeze and everything, this is not just a color scheme the because there's some details here that are different like a lot of things are missing rounded corners. Uh, cards have a very different styling both without and with hover and everything like that. So basically this screenshot is is proof of I have control over how discover looks from the style. And uh also I will note I did not touch a single line of discover to make these changes. So current state cute quick controls. Um we're working on building that style out. Um there's a bunch of stuff that's already working. Uh like most of the the the basing controls that you would use in an application. Uh there's some bits and pieces where we still have work to do. uh most notably table controls and there's uh some calendar related stuff that was added in cute 6 that still needs styling. Funnily enough, uh as far as I know, none of our other cutequick implementations have styling for those calendar controls yet. So, if we can get that into Union, Union will actually be the first to provide a brief style for those things. Um a important thing to note here Kurigami uh we have the I wrote uh the usual Kurigarami intergration points for it but also I created a bunch of things. So the custom controls from Kurigami are actually under control of Union which is something that if you ever seen uh a Kurigami application running on a different platform with a different cute quick style you'll notice that the cards for example will usually still look like breeze. It that's one of the specific things I wanted to fix with Union to make sure that we the the style controls the entire application. Uh, Kiraami is uh there are still some issues with some of the controls. Uh, I'm basically doing something with Kirigami that was designed but was never fully tested. So there's a bunch of controls that we run into where the uh the the template basically doesn't do enough and the actual control does a whole lot of stuff which means that if you're then reimplementing that control you also need to do all of that. Um it's fine or it's it will be fixed eventually hopefully. Um some notes on the CuteQuick side uh cutequick API details. Um included a snippet here of what uh one of our style implementations looks like. Um on the cute quick side, most of this is controlled through attached properties. Um we there is an implementation of a custom re renderer uh for rectangles because there's a bunch of features that I needed there. Um I'll talk a bit more about why that is later. Um and I implemented a custom layouting thing so that the style is actually in control of how certain sub elements are placed. Um this was to deal with um spin boxes. For example, spin boxes on different platforms and with different styles will have different positions of where the up and down or plus or minus buttons are placed. And I really wanted to be able to give that decision to the style. Um that's all working now. Um and as I was saying we have an in initial implementation using plus SVG um that required a lot of extra information because plasma SVG turns out that SVGs only in in um only contain graphical data and not really information on hey this element should be used for this border and this element should be used there. So we had a whole bunch of extra stuff around this uh to basically encode what uh in the current implementation of plasmas theming the cute quick codes uh encodes um it it al it worked uh or it works it's not ideal. So somewhere in the start at the start of this year I made a decision that this plasma input format was not going to be future proof. It's if I I would not be comfortable releasing a version of Union using plasma SVG as input because it would not be future proof. it would basically be saying, "Yeah, here's this cool new thing, but you can't do anything with it because yeah, you can't do anything with the input layer." So, instead, uh, I've went and spend a bunch of time on something that I've originally already wanted with this, which is to use CSS. And why CSS? Well, I had some requirements for the input layer. Um, and very specifically, one of the explicit requirements here is that there is no logic in the input layer. Having logic there would data flow extremely complex and uh would make it very hard to cache anything from that input layer. So basically at the design phase I went okay no logic in the input layer. Um also I want something that is easy to change for non-developers. So it basically should be a text file that you can add it to get different results and it needs to be something that is extensible because as I already mentioned like Kurigami controls I need to be able to style those eventually plasma plasma has a whole bunch of thing different things we probably want to use this at some point for plasma so those need to be able to be supported etc. So how who of who here has never seen any CSS? [laughter] Yeah. So CSS is a very well-known language. Um it's very actively developed a bit too actively sometimes. Um, but it's also designed in such a way that it abstracts uh it's it's an abstract description of of a style because that's how HTML uses it. That's all nice and such, but there's also some other sides to this because of how it's used. A lot of CSS is geared towards web usage. And more importantly, that also means that most of the implementations of CSS are tied to web browsers and web web engines with a few exceptions. If you look around like for C++ uh CSS partial libraries, you'll get a few hits. Most of those are hits of libraries that are no longer maintained and implement some version of CSS2 whereas right now I think we're at CSS 4ish already. So this is a problem and this is why I chose to initially work with plasma SVG rather than going for CSS immediately. Well, luckily there is one project that kind of stands out from the rest which is called Servo. Servo is a rustbased web engine created by Mozilla to I think originally to replace uh Firefox's rendering engine that never happened but still the project is there and one of the advantages of servo is that it's actually designed in a fairly modular way. Most importantly for our case there is one rust crate which is called CSS parser which ostensibly uh implements a CSS parser using Rust or that's at least what the name suggests. Um it's part of servo. It's used by servo to uh for their CSS parsing. Um, it does support a whole lot of uh modern CSS, but it's actually uh turns out that it's quite a low-level library and it's it's a parser, yes, but it's a parser that basically gets you a bunch of function calls where you go, okay, yeah, yeah, now you need to do something with this data. that which you know I guess it works fine if you if you're servo but if you want something where you're like okay I I need to get some sort of abstract description out of this CSS file that I can then transform into something that my style can use. It's it's not that great. Um there's also a bunch of of CSS modern CSS features that it just goes yeah um you have to implement this yourself. Luckily there is there are some bits and pieces uh extra crates that allow that that help. Uh most importantly there is a selector parser library uh or create that's also done by servo which uh takes care of the entire parsing of selectors that saves a lot of headache but there's other bits where I basically ended up writing still sort of an my own parser simply because that's what the create the crate didn't implement that. So there's also the other side of this u it's great it's you know this thing is available but it's rust and union is a plug basically a set of plugins for cute code which kind of requires it to be C++. Luckily, there are is some stuff um that helps um specifically there's the CXX crate which in this case I decided to okay let's create my own library using CX with a very thin layer uh for a very thin CSS uh or C++ API uh so that I can use Rust for most of the actual CSS parsfing tasks and just apps kind of output this abstract uh description of what the CSS file uh produces. So that's what I eventually called by by the very creative name uh CX Rust CSS parser. Um it's a library that basically allows you to uh to take a CSS file uh par have the Rust code take care of the parsing and then get an abstract repres representation of the contents of that CSS file out of it so that you can then do stuff with that like you know build a styling engine. So great, we have that thing, we have CSS. How do we make use of CSS for styling our applications? Basically, how do I transform uh a bunch of things like uh this kind of gobbledegook? How do I transform that into something that the application can do stuff with? So uh one very important concept in CSS is selectors. Uh selectors basically say hey um this is a query to your application. If something matches this you uh do something with that element. Um, me personally, I knew from because I kind of was hoping for using CSS for Union. Anyway, I kind of stole this idea from CSS uh in Union because it's a concept that's just very well applicable to what Union is trying to do. So C union has the exact same concept which meant that you know doing this from CSS that's easy easy to map. So as an example again here's a bit of uh union quick API it basically says hey uh if you're a button then um you know I declare that this is a button and some states and then there's some of the CSF possible CSS selectors that allow you to target this specific element either very generically like hey every button needs this styling or uh specifically the press state of a primary button needs this styling. So the other thing is properties. [snorts] Uh properties tell your application union what to do with an element. How does it change when these proper when this rule applies? When the selector applies. Uh this is where things get a bit hairy with regards to web. Um, web CSS has a whole bunch of properties and a whole bunch of of those properties are not applicable to union or to an application in general because they say things about how certain layout stuff happens or things about uh overflows and such that just really don't map nicely. Uh, and this is actually the reason why I wanted uh a more of a low-level uh CSS parsing library for doing this because I wanted to be able to uh from union say these are the kind of properties that we support because again some of them just don't make sense. So we still need to know what those properties are. Um, and luckily, um, modern CSS, I think it's very modern. I don't even know if it's official standard yet, um, does have a syntax for declaring properties. And I realized that I could just use this same syntax to declare all the properties that union would need. It would me mean that we you need to load first one CSS file that just defines these are all the properties that we need and then with that it can then parse all the rest and that turned out to actually work really well. I think that works a lot more it's it's a lot more flexible than having to worry about declare designing an API for to define properties in the parser before loading a CSS file. Um so the other bits is for important in CSS is functions. Um this is one of the areas where the CSS parts are great really um uh it kind of disappointed me. Uh basically all of the function stuff in modern CSS if you use that crate you need to reimplement it manually. even v even varss which are basic more more of a part thing than really a function call even those you need to reimplement um and that was a lot of work so there or that is a lot of work uh which means that there's a lot of stuff that in CSS functions that has not yet been implemented um it does give us some flexibility because other stuff like having a simple linear inter interpolation function was something that we could add. So in conclusion, I switch we sw I switched to CSS for input for uh union. I am despite the issues with the crate, I am very much uh I think it's a good choice. It's a choice that long-term um will benefit us. It means that we have a language to work with where we can say, "Hey, we've have this thing. If you want to build a new style here, here's the the documentation on CSS. Go wild. See see what works, see what doesn't. Um, it will most like it it there will be changes, but it will not be a dead end. So with that um what's next? Well um obviously this needs to go to users or develop at the very least developers people who like testing this thing. So uh we need to make a first release. Uh I had hoped to be able to do this with in conjunction with plasma 605. Unfortunately, that's not going to be feasible because there's too much that still needs to be implemented. Hopefully, uh we can do a release somewhere between 6.5 and 6.6. We'll see. Uh it will contain a CSS version of Breeze, which means we'll now have our fifth implementation of Breeze. Hooray. [laughter] Um the thing is uh one of the things I hope uh is that this implementation can actually start uh replacing uh the cutequick to brief style already because it should basic it should do mostly the same as that style except with a different back end and in a way that is hopefully more scalable towards the future. So the other things are uh I don't think most many of you were not at Andy's talk earlier about plasma next but Andy's been working on this project for a while now and really wants to build a new application style using uh the design system they're developing and it's kind of stuck on Union getting to a state where they can start developing this can be start developing with because otherwise they would also need to do four implementations of that style and in finding some developers for all that it's a nightmare. So hopefully uh we can start building on this soon. Uh it will probably require extra development in union because there's several feature there's features that I know aren't yet fully supported. Um but we'll see how that goes. Um yeah so there's stuff that I have not yet implemented for the CSS part there. uh those will eventually need to be supported for it to be fully you know um to work like uh a lot of people would expect from uh CSS and especially more fanciful advanced styles will require features like gradients or textures uh like um and one very important thing that is not yet implemented is animations which is its own kind of worms and then there's other uh outputs formats uh most prominently cute widgets uh there is a merge request from Joshua Goins who started a very initial implementation of cute widget style which mean so I know it works uh but that still requires a lot of work Um and uh there are potential other targets within KDE. Uh we have decorations that are right now hardcoded or yeah do their own painting and everything. Um stuff like that might also be a good target to eventually unionize. And now we do get at the question slides in the right order. [laughter] So any questions? [applause] >> Questions come first. >> No [laughter] you. Yes. >> So [clears throat] thanks for a good presentation. Uh I didn't watch your presentation last year but I had an understanding that union was also a plug-in based architecture where you could choose different styling languages such as CML CSS and other and then the designer could basically choose which one to use. Is that something that is still relevant or did you move away from that? Thanks. Uh so yes at its core it's still uh a plug-in based architecture. The input plug input formats are plugins. Um right now while the plasma SVG input is still there it's not very well maintained uh because I've been making a lot of changes in favor of the CSS stuff. uh which means that the plasma SVG has kind of bit rotten. Um I've been trying to decide whether uh I want to revive that or you know get rid of it. Um Nate actually made a decent argument that uh eventually we may want to at least be able to support a bunch of plasma styles so that you can uh we can port plasma without needing to get rid of all the 900 something teams that are there. Uh [laughter] so but that but that's yeah uh so so in that case we would at least have two different input formats uh one CSS based one S plus M SVG based >> I'm being >> uh perhaps I'm overthinking this but um have you made any performance considerations in terms of resource use use for example uh when all of the style engines now include a CSS parser. >> Yes. Uh I've actually done a bunch of performance work recently. Uh interesting side note on the CSS parsing thing. Um because the input uh input layer is basically immutable, we can parse the entire CSS file, get that result and then just discard the entire CSS parser again. So that that bit uh eventually my my goal there is to actually be able to cache that so that uh you basically don't even need to run the CSS parser for application startup. But that's a longterm goal. Sorry. >> It's not. >> No. No. >> Oh. Uh, you mean the desktop style? >> Yeah. >> Yeah. So the desktop style does uh basically it uh recreate it creates a Q style and then uses that Q style use with Q painter to render to an offstream texture and then render that offscreen texture as a cutequick object. Again that that's one of the things that I mentioned earlier where uh yeah that that implementation of of the cutequick style is not making good use of what cutequick offers. >> Yes that that that specific case I uh or for that case I hope that union will be better. Okay. [laughter] So, um I this is less of a question, more like a wish list, but I really hope somebody makes the fluffy bunnies theme with this. I really want to see that. That's all. Thanks. Anybody else more questions? I can get you from >> It's fine. I reach you. You mentioned something about rectangles because some API is missing especially for the cute folks in the room. Maybe you can drop some wishes for them. >> Yeah. So I I said I was going to in this to to mention that later. I forgot. Um so basically if you have CSS uh for a CSS background, you need um uh variable border width, you need variable uh border colors, you need variable corner radius. Um there there is a whole bunch of features uh that are implied by what CSS offers there. Um, so far I uh, so I I I have a custom renderer using a shader that basically supports a whole bunch of that. Uh, not yet variable border colors, but I have an idea how I can do that. Uh, so that's a feature that will be supported at some point. Uh, also things like gradients and all that kind of stuff. Um, yeah, there's there's a lot of features there from CSS that you need for Yeah, because you want to be able to offer to designers that, hey, if you want a button with weird different borders uh and colors and glowy things and everything, yeah, that should be possible eventually. Thanks. Uh do you think that uh kind of Q CSS parser written in C++ of course would be interesting or beneficial in some in long term in future? >> Um maybe I am not the one that's going to maintain it. [laughter] because I already have this thing to take care of. Uh so uh it's the the thing is with CSS it's one of those web standards that just keeps evolving. I think it's better to have it in a central uh like shared repository of or of some kind. Maybe eventually we we will replace the current implementation. We'll see how that goes. Awesome. And one more very quickly from online. >> Yeah. So the question is to what extent of theming will be allowed in union on a scale of color schemes changes to the widget style changes. >> Um so in that regard I showed a picture two pictures of discover one with brief style one using a a completely different style. Um I can maybe very quickly go back. Um yeah, so um this is the extent that it already allows now. So there's different border sizes, different uh border styles, uh different fonts, different colors. U it's it's a lot more extensive than just color schemes. Um eventually I hope that the entire look of the application will be under control of union. >> Okay, we've run out of time unfortunately. So if anyone has more questions for Arian, come find him afterwards. [applause]