PLASMA STUDIO DEVLOG #1

A couple of weeks ago, I told you that I was trying to do this weird application that can do video editing but hardware accelerated and that I can also do like vector animations and that can also do OBS and that can also do Audacity kind of stuff and maybe even more in the future. And I laid out my plan and now it's been 2 weeks and I would like to update you all on the development of this weird overambitious application. So, let me show you. Here is Plasma Plasma Studio pre-alpha. Like it's just a couple of weeks of development. What did you expect? But actually, there is a lot to to like, I think, already. Let me show you. Firstly, we've got this tabbed interface, which I really like with all of the tools that you have at your disposal. And this here, white, is um a node graph. So, you can actually drag and drop anything and drop it here and it's going to uh you know, create a node. And uh I dropped a video as an example and by default it loads a local video, but you can just assign it to anything. And you can drag it around, obviously. And if you double click it, you can actually preview the video the the node and you can play it. And this, as a reference, is a 6K. can see the resolution here, 10-bit HEVC video. So, it's as heavy as it gets on my workflow, which is pretty heavy, I think. And uh let's do some color grading to show you how this works. And of course, first things first, I apply a LUT. Well, technically, I should color grade first, but uh we don't do things correctly here. And we've got LUT. Now, obviously, this is upside down and I'm going to do a video about why it's upside down, I promise. And in position and geometry, we've got flip, so we can connect flip. And as you can see, you you can connect nodes, and you can do stuff, and you can preview each one, and you got a little preview of each, and you can still play. And well, now it's 25 FPS because I I'm not uh giving enough power to this computer and all that screaming, but normally, it's like 100 FPS for a couple of nodes on a 6K video, which is great. Um and in here, you've got a button to hide the nodes to actually see what you're doing. And um I don't know, let's do some more stuff, some I don't know. You You can see there's a bunch of uh color grading things. Of course, none of this is particularly detailed, but hey, at least it exists. You can even uh pick up waveform if you'd like to see what you're doing to to the stuff, and uh let's add some gamma, bring down the lift, something like this, you know. Um random color grading. I could also add a vignette to cuz this is already quite vignette, so I can Yeah, I'll show you. I can balance it out like this. And yeah, now it's much better. This is what we got. This is how we started. And all of this in a node-like interface which which is cool, objectively. And you can do even more complex stuff if it works, and it might not work. Again, a couple of weeks of development, but position and geometry, we can also move things around. We've got a lot of transforms. And um as an example, let's move this 6,000 pixels to the right. And of course, uh this is a new node, so it's going to get to a different branch. And we can merge this together to see like the before and after with a group element. And we've got one element here and one element here. And just like this, voilà, before and after. All in a node-like application. And you can play this. Of course, it's going to lag because now it's a 12K video. That's a lot. But yeah, how many 10 FPS? But I promise you actually get more FPS normally. And the cool thing is that well, after doing all of this, you could say, "Well, I don't actually need 12K of resolution." So, you can scale this down to something smaller. Actually, let's resize because this resize node has less bugs. And we can bring this down by a factor of like 10. Like this. Now it's much smaller resolution, obviously. And the cool thing is that each node actually tells the node before how much resolution is really needed to be displayed. And so, given that at the very end, we resize to a very small thing, well, each node now knows that it's not necessary to actually render a 6K texture because we don't need that. And the cool thing is that normally you would need this at the very start because otherwise everything would be 6K. But in here, you can just have it at the end and it's still retroactively going to work. And the cool thing is that all of the textures are still going to think they're in 6K. All the coordinates, all the transforms, you can still see that this is 6K and it it still thinks it's in 6K, but actually it's been rendered in you know, 600 pixels by 400. And this should improve performances. Yeah. As you can see, we get the same thing as before, but 100 FPS instead of 10. And so, I think that's cool. Isn't it cool? And you can save things. I can save this. And I'll just save it to the home as test.qml, which should exist. And you might might ask, "Okay, but what does the save file look like?" So, let's bring it up. Actually, let's open a new instance of Kate like this, and let's open this file. And so, what this file looks like is a normal QML file. Now, if you've never worked with QML, that's fine. It's a markup language, a bit like HTML. Um and so, you've got an item, which is just a container, and then we've got our video. And then there's the loot and the flip and uh flip flip the things. And each node has all of the relevant settings in it, which is exposed in a human-readable interface. And the cool thing is that I'm not going to do it right now, but QML files technically are applications, like you could run them as applications. And if you do run this QML file as an application, you're going to get this without the UI, just the result. Because it's an actual the content is actual content. And of course, we can also load the file. Um I can even attempt it. I don't guarantee that it's going to work because like I implemented the loading like a couple of hours ago, but let's see if it works. And does it get now? Yeah. It worked. Uh yeah, a bit bigger, yeah. That works. So, that's a bunch of cool things that I've shown you, I think. We You already have You could already do a lot of things. Um especially because let's quickly go through the the nodes that we have in two of them. Let's start it again. Okay. So, what do we have here? Uh let's close this stuff. So, I've shown you a video, but we can also do images, obviously, and we can also do raw images. And this is actually going to load the raw image as a 16 float bit depth file, so you can do all sort of editing into this. Let's bring up the exposure as an example. And yeah. And this is actually raw image image editing, which is cool. Uh what else? We can screen capture through the normal interface of Wayland, and that works. Uh you will also be able to um capture a window, not just the screen, but right now for some reason it's broken in KWin, not my fault. I'm going to fix it if I manage to. And you've got also camera capture, which I don't think Oh, yeah, it even works. This is my computer webcam, and yeah. That works. That's so cool. And of course, you can you can still do all the graphical effects to all of this, like I don't know, um I should have here lens distortion. And I can do this. Let's do like like this. And I can do this. Which one is cool. And and you got a lot of effects, and um you got to clip, which basically takes a a video. We don't have any video here, so let's bring up one. And basically, instead of having the entire video, you select a start and duration, and it's going to clip part of the video. Which means that you can have like multiple clips, and then in I guess position and geometry uh where was it? No, video elements. You've got concatenate, and you can concatenate different videos so that they play one after another. I'm not sure if this actually works. Yeah, it doesn't. But yeah, that's the idea. Eventually, this is going to work. Eventually means like this afternoon, because I'm disappointed that it didn't work. And um So, what else? Let's still bring our standard video as a reference. Uh we've got a bunch of graphical effects. These one are These are the most simple ones for now. Saturation, LUT, exposure, opacity, the contrast, white balance, vibrance, tone equalizer. I'm showing you a couple just as as an example. I can change the contrast. We can change the highlights. Yeah, this looks terrible. But yeah, you can do all sort of stuff, and then vignetting. Position and geometry is all of the transforms, obviously, flipping, and also grouping things together. And there's also compositing, which uh should probably be called blending. I still don't get the difference, but um let me show you what this can do. I mean, I think you know what this can do, but let me show you again, because it's cool. Um camera capture. So, this is my camera. Um we can check how big it is with the resize node, and it's 1080 by 1920. And so, actually, let's do that 1080 to 1920 to the video. So, they are the same size. And we can composite them together. Now, normal it's just this, but I've got darken, I've got multiply, I've got difference, color burn. These are all of the ones that Inkscape supports. I checked. And um yeah, they work as you would expect them to work. It's pretty cool, isn't it? Um and then, of course, you've got the stuff like perspective, uh lens lens distortion, and you can mask things. Let me show you. This is the background, and um let me bring a rectangle uh as as a mask. And we can do this, and this is masking the size of the rectangle, which is 100 by 100 on the top left. But, uh as I've shown you, we can move the rectangular around with a translate um node, like this. And we can also move our masking target around, like this. Yeah, it works. And maybe let's make it bigger so you Yeah. Yeah, I I didn't um put a vertical limit to the size of the nodes, okay? So, cropping. Uh we've got mask thing. Uh and this could be any shape. Let me show you uh with a more interesting shape. I've got polygons as well. Um 500 by 500, like this. And like this. And voila! It works. Uh yeah. So, that's cool. Hello. Go on. It's cool. And uh well, I've I've I've begun to show you this rectangle and polygon nodes. So, we've got vector graphic nodes. And I've started with a rectangle, circle, circle, circles, and polygons. And you've got also paths. You can enter manually a path right now cuz I as you can see this is entirely node based for now. There's no editors. But you can do stuff like difference and it's going to do what you expect it to do or maybe it's not. Why is it not? Exclusion maybe? Why didn't this work? Okay, this one works. I'll double check the other one. Don't know why. Never mind. But yeah, that works and you can like apply a gradient to it. And you've got your gradient. And you might say, "Okay, but like this is low resolution. This is 500 by 500." But the cool thing is that again, you can make this bigger. Let's resize this to 5,000 by 5,000. And now it's higher quality even though this is was lower quality because again, the same thing as before. These nodes tell these nodes and tell these nodes the actual size that we need. And so even though this polygon and this rectangle thinks they're 500 by 500, they are actually asked to render at 5,000. Retractively. So, all transforms my touchpad just died. Okay, no, it works. So, you know, all transforms are non-destructive. As an example, if you take an image, you resize it to like 10 by 10 pixels and then you resize it again to the correct size, you don't lose any quality. It's non-destructive. If you do want it to be destruct- destructive, I guess we can do that somehow, but it's still cool that you can do it non-destructively. And uh actually, let's stick with with rectangles because I've got another cool thing to show you, which is animations. Because yes, we've got animations as well for vector graphic elements or anything else, really. So, let's animate the rect width property. Right now, the number animation is this weird node that you attach to what you want to animate. Eventually, I actually want the UI to be directly within the node. But yeah, now we can change the direct width, and it's going to show us how it changes. Let's zoom on this. And and again, we we can still use this as anything, so as an example, let's use it as a mask over my webcam again, like this. Voilà. And now the the the mask is going to be animated over time. I can show you. It animates, and you can pick the easing curve. So, that works as well. Wee! So, that's cool. And I'll show you something more complex, actually, which is uh suppose that you do have two different items that are animated, like this. Blah blah blah blah blah blah Now, let's do the circle height as an example. And let's do it to 50, like this. Now, it's going to be animated. And of course, uh if nodes are not linked together, then they animate by themselves. They are not linked together. But if you group them together through, as an example, a union node, a path union node, like this, uh then yeah, the the rectangle is absorbing the the the circle, but uh now they are linked together obviously because they have a common parent. And so they animate together, which is cool. Let's do like this so you can actually see the Yeah, circle. And um You can even have a delay node that allows one animation to play after another like this. And let's put a delay of 2 and 1/2 seconds. Again, this is only milliseconds. And now, as you can see I'm in the latter part of this animation, but I'm in the beginning of this one. And there are they are still synced together because they are they are part of the same scene. And you can change the time here. You can change the time here. You can even change the time here, and it's going to apply to both. And this is the full timeline. And you can still save these things, load them Come on. Come on. Let me be proud here. And um This is actually already a lot of things. Let me show you plans for the future. So, Firefox, please. Uh this project is currently open source. Of course. And it's part of the of my personal repository on um kde.org. Now, this currently is not a KDE project for the simple reason that uh you know you don't get to decide what is a KDE project like this. There's actually an application process. Multiple developers need to be working on it. So, right now, as you can see, this is just on my personal page of the KDE GitLab. But of course, eventually, I would like this to become a hackaday project if it works out. Uh And um there's a work um items list and these are the elements that I want to implement. And uh is this would be the elements to implement for like the very first alpha. And there's uh 95 of them. And I'm currently at 46, which if you think about it, that's pretty good in a couple of weeks. Sure, they don't work great, but you know, that's something. And uh you can actually go through this list. It's public and you can tell me whether something is missing. You would like to have more effects, less effects. I I don't know. You tell me if I'm missing something. Now, obviously, there's uh a few things that are I need to work on. Firstly, there's a bit of debugging to be done uh on both animations, which uh worked now, but sometimes uh they don't. And there's a few bugs to be to be fixed. Uh so, I'm going to focus on that and also on the concatenate node, as you can see as you previously saw, that doesn't work. So, I'll fix that. And then there's some cool feature features. Firstly, uh I need a renderer, a streamer, and it's missing here, but also um a recorder. So, basically, um nodes that record or render what you give them. So, uh as you can understand, those are pretty important if you um you know, edit a video, you'd like to eventually render it. And uh a streamer and a recorder would be for more like OBS type of things, where you just record what's happening, not necessarily render it. Like, you can miss frames and that's fine. And the streamer would actually stream onto YouTube as an example to do live streams. So, those are uh a big focus for the next couple of weeks. Then, uh another thing that's currently happening is that this video that you see here is actually a Oh, by the way, I forgot to say but at this point I think you can understand it. All of this is hardware accelerated or it wouldn't be running at all because again, 6K video, but all of this all of this hardware hardware accelerated. This video right now is a QML element QML video element which works uh as you've just seen but has some little issues uh such as the fact that it uses a lot of RAM and it shouldn't. So, what I was previously doing and what I will I wanted to be back at doing is embedding MPV into this. Um the reason I'm not doing that right now is that even though it works better in almost every way, it's slower and I haven't understood why. It's probably my fault for how I am embedding I'm embedding libmpv. So, I'll work on that but yeah, it's cool that you can actually play 6K videos at usually it's 120 FPS. Uh and by the way, the video has 25 FPS so you you wouldn't notice but still. Uh what else? Um Yeah, audio. The all of the audio effects are missing and all of the audio nodes such as audio capture or audio are missing and that's because I haven't figured out figured out how to do audio yet. I I don't know. I just don't know. I'm not an audio person either. Uh suggestions welcome. So, that's another things that I'm going to focus on. Uh I want to finish up all of the graphical effects nodes because those are useful and maybe also like transitions. Uh, I haven't showed you but you can also like change the playback speed of any animation, of any video playback. And same goes with invert, which should be called reverse. I got the name wrong, but you can like invert an animation. You know, it's all the things that you you would expect. And um Yeah, and and then in the next couple weeks I'm also going to try to use this on different GPUs to see how the support is. Right now, all of this is running on my integrated Intel GPU from 4 years ago, which is not good. So, it's actually impressive that I'm able to get 120 FPS from playback of a 6K video and still be at 100 FPS after applying two or three passes of like different color grading nodes, especially because again, each node actually creates its own texture, so you can preview it. Maybe in the future that can be optimized. And uh yeah, probably if I were running this on my Nvidia it would be even faster. Maybe. Currently, there is no code actually that's specific to Intel GPUs. It's all handled by cute, interestingly enough. If I do implement libmpv and if I do implement recording and rendering then yeah, probably those are going to require some custom codes to handle different GPU, I guess. So, I'm not looking forward to that. And by the way, if you would like to help with this project, final part of the video. Firstly, uh yes, uh if you know how to do recording and rendering of QML video QML elements, uh send me a message. Let's work on this together. I really don't want to do this. So, I really don't want to reduce myself. That is. So, a help would be needed there. And you can also donate to the channel. You donate on my website, which is the the libra.news. I hope you know about it already. You can subscribe here and you get some cool stuff. Um but yeah, I'm going to work this regardless of the donations or not. So, I'm just very in love with the project as it's going. I think I've shown you everything. 25 minutes minutes of video. I mean, what else? Like Yeah. I think I've shown you the important bits. But yeah, this is like very cool, isn't it? Come on. Come on. Tell me it's cool in the comments, please, because I've I've spent so much time developing this and so much time thinking of the proper way to handle time management because you both have to do a kitty life type of things where you move things around, but then you also need to do an OBS type of things where there are continuously updating elements. Uh it took a lot of time to figure this out and now it works. Actually, let me show you this. Um you even have a playback controller, which seems stupid, but actually it's a very smart node. This node takes a a a video, such as this animation, and as you can see here, down, you actually have the the playback of the animation. The break for some Yeah. As I told you, there are still some things to figure out, but we'll get to that. But the thing of the playback controller is it takes um a a a video animation which you can see the application recognizes as a video and allows you to change the time and play it and such. And then it give it manages that. So, it allows you to play it and pause it and move it around just like the application, but this element My computer just died. No. My touch pad died again. I don't know why. Okay. And this element is not uh considered a video. You can manually play and pause things, but actually this is a considered a static element. So, you can like embed it together with my basically the equivalent of adding a video to OBS. It's not going to render it because it's it's it's OBS. It's OBS, but it's going to give you the option to play it, maybe loop it. I don't know. This is cool. Why did it break though? Yeah. That's all folks for this week's devlog and follow this channel if you want to see more and learn more about this weird project of mine. I would argue I'm doing a good job. Let's stop this.