Submind YouTube summaries
Thumbnail for AI Editorial Assistant for Fedora: Demo

AI Editorial Assistant for Fedora: Demo

Watch on YouTube

Video summary

The video presents a demonstration of an AI Editorial Assistant designed specifically for the Fedora community, aimed at helping users evaluate draft articles against established publication standards. The tool is capable of operating through multiple interfaces, including a graphical user interface (GUI), a command-line interface (CLI), and Jupyter notebooks, offering flexibility for different user preferences. Users can select between two specific publication types, such as the Fedora Magazine or the community blog, and choose from various large language models, with GMA 4 E4D recommended for current use. The system is built to be reproducible without requiring complex builds, allowing users to simply pull container images directly from Quay to run the analysis locally on their machines. The core functionality of the assistant involves analyzing submitted text to ensure compliance with specific guidelines regarding tone, structure, and content quality. During the demo, the tool successfully identifies when a draft meets standards but also highlights missing features, such as featured images or proper header tags, which are crucial for publication readiness. If an article is deemed too brief or lacking in certain views, the AI provides detailed feedback that can be exported as a text file, allowing writers to make necessary revisions before resubmission. This process acts as a preliminary filter, ensuring that only high-quality drafts proceed to the final human review stage by the editorial team. To support these evaluations, the project includes a comprehensive repository structure containing raw data, quality reports in JSON format, and scripts for fetching articles via the WordPress API. The demonstration also explores performance differences between CPU and GPU environments, noting that GPU provisioning significantly speeds up the analysis of faithfulness and accuracy. Furthermore, the tool allows for benchmarking and visualization of results, enabling users to compare good versus bad examples and understand specific issues within a draft. Documentation within the repository guides users through prerequisites, deployment on platforms like Local OpenShift, and step-by-step instructions for integrating these tools into their workflow without needing to build everything from scratch. In conclusion, this AI Editorial Assistant serves as a robust foundation for improving content quality within the Fedora ecosystem while feeding into larger RPM packaging projects related to Retrieval-Augmented Generation (RAG). The presenter emphasizes that while the tool provides automated checks and recommendations, it is intended to support rather than replace human judgment, ensuring that the final published material meets the high standards of the community. By offering clear feedback on structural elements like images and headers, the system empowers contributors to refine their work efficiently. The video ends with acknowledgments to mentors and the broader Fedora community for making this internship opportunity possible, highlighting the collaborative nature of developing such open-source tools.
Read the full video transcript
So we will do a demo uh seeing the tool in action like on real drafts. Um so at first we will just try to launch the we would paste in a draft expect to get like a grounded review. So the first demo is going to be about a good article. So uh the tool in action should load a draft that meets like uh sorry we will try to load a draft that actually meets standards. We have examples of that and then the model probably will have to confirm it's compliant and then it sites um so we have multiple paths right you can actually do it using the streaml app or even doing it like using the terminal path uh which I showed earlier in in in an earlier video like we I'm also going to show it here uh we can actually try to switch publications like it can either be like the magazine or like the com blog and then it's actually are reproducible. So you don't need to like to build anything. You can just pull like the the OA images from Kore directly and run. Now there is an honest move uh that we would like to make here is that be the final person to sort of confirm it but at least the tool will flag some missing features that you need in order for your draft article to meet the guideline and then you'll submit something that's best for publication. Uh so the next phase will be uh like because this project was the foundation on which we built or like we are sort of feeding the larger RPM packaging uh rag project which is the main project for my internship. Um and of course for that I would like to thank my mentors uh the feder community uh at large internship program for this outstanding opportunity. So now let's get into the real work. when you download or like clone a copy of I mean of the repository and then you will see this app.py pipe which is the file that we have here and when you launch it it will open um the GUI like the graphical user interface you have here um you have the publication type it can either be like are you trying to test like a draft article against uh the Fed community blog like guidelines or is it against the Feder magazine model that actually exists right in in this case for now we only have these two models and the recommended one is like GMA 4 E4D and then you can also choose the review mode. Do you want to review like entirely like the full editor review or do you just want to review stuff before this one um actually ends? when we uh come back to um I said I mean three possible ways of using this which is the recommended one for everybody uh like CLI like the command line interface and then run like run that is already there I give you the model right wait so now it will be waiting for a prompt here that it will use and then it analyzed and then it gave the pillars of the tone, what you have to avoid and so on and so forth until the end. So you could do it using this one if you prefer and if you have the knowledge of it rather than using the GUI, right? You could take an article, paste it like possibility. Uh you could also launch a notebook, a Jupyter notebook uh that shows you everything on local host locally on your machine. And then here if you run this you will see uh here set up an imports try to run the quality report analysis to know how how many of those articles that you f either in like the account blog or like magazine uh worth or like keeping uh for your lag enhancement and then it will give you that output. Then after that okay there is drinking strategy experiment because like I said before we for like memory outage and then after that you can now do the right query testing doing this one benchmark results visualization. So here you can even like sort of visualize the results reference articles you would find like some good articles and then like two bad articles and then you can now try to test them out and see what it what what it like. Let's go back to the other one uh and see hopefully if it's over. Okay, it's still running. So when you go back to the repository here you tells uh in this docs folder about first of all this guy that MD give like what are the things that you have to set up like the pre requisites and so on and then what is the contribution like you have like a quick uh a quick start and what are documentation files that actually help you understand this project as a whole and sort reproduce it. Tool was happy. We gave it this um uh we gave it this uh to be a good one by my mentors and then of course the tool actually told us that okay this is ready to it wasn't able to detect the feature image. Why? because we copied the text content rather than taking the source file or like uh maybe because if you do a the image tag like the HTML image like you know tag and then it will okay there was an image there was like a featured image and stuff like that. So the reason why it was not able to find the headers is passed. Okay, talking about some some uh tags and all like let's see now what's the overall uh recommendation. So it says it's ready to publish or need some revision. So the article is too brief and like some views and then you can go ahead and try to download the review as a txt file and see the recommendations and try to adjust your draft article like that before submitting submitting it again. So, what I'm what I'm going to do is uh I will simply copy this one. Go back here. Copy this and go back to the tool. Just get rid of this. Paste it and run this one again. Finish. It's going to be uh running the second one here. Reviewing with JMA. So, in the meantime, I can go ahead like I said download to actually improve. So, which is um a good way of having a feedback on like a draft article that you actually submitted. Um review this article. Okay, maybe I mistakenly stopped it. So, we just have to Okay, it's still running here. You can stop it or you can just let it be until it's over. So here uh select reviewing and it would take some time. Um yeah okay we spoke about this exploration uh so we are going to go ahead and try to shut it down. Yeah shut down this Jupyter server. Yes. All right it's down. So you see that we can use like the Jupyter server like the notebook or we can use it on like the CLI environment to do it or like the most preferred way the most recommended way for anybody for anyone in the community uh is the like the gooey like the app finishes so running. Um there's a repo. So maybe um here is what we actually got when we um okay fetching the article and then um like thinking of deploying it using local open shift or CRC internal to like Fedora and Red Hat. Um so we were sort of preparing for that. And then like I said we also have the uh GPU environment that was provisioned which is very much better uh as opposed to running like the tool or like doing everything in a CPU environment work uh and checks it against like faithfulness and accuracy and other like uh criteria. So for the repository structure maybe we will just get back to this before and you have the data folder in which you have like a child folder or job directory of raw data and then the guidelines right like for magazine articles too uh and that is also like the reports directory uh in which you find like the quality report JSON uh because when you fetch all of those articles some of them. So you need to run a quality report to get to know those and then you have uh the review that's saved model review outputs. good versus bad examples. And then you have also the script directory in which you have the script helps you fetch articles using the WordPress API and then you can uh the the stage corpus uh what helps like sort of clean the data set like the quality report generator. Then the model benchmarking review a good example and the model directory in which we do have the Ramadana configuration like directory in which we have the details about that and then the notebook the quick store um that helps you sort of use the tool core images no pipeline no build or like set up from and then the guide that is a complete step-by-step like instructions uh guide for you to the local open shift like deployment guide and we also talk about like community shift at some point I just check this out oh okay I'll be right um and then like the step by step also like bastion login and then anible like we were just sort of exploring platforms like these tools for for the system that we're building um so yeah you'll see all of these and then the requirement before instead of like installing them one by one. Yeah, it's actually timed out after 300 seconds. That's because I'm running it on my laptop. It's CPU. It's on GPU. And then because I run the first one that was successful and this one just output this one a bad article and then we know the like the issues in that B article and then we expect to return like to flag the same problems or the specific problems and then name or give recommendations on um like how to fix it to meet all of the guidelines. Thank you so much for listening, watching