Video summary
The Flock 2026 Fedora Engineering Steering Committee (FESCo) Q&A session marked a significant milestone by welcoming newly elected members Neil Ga, Fabio Valentini, Maxwell G., Simon, and Michelle Lind to the committee. During their introductions, these panelists highlighted their diverse backgrounds in packaging, security, tooling, and various language-specific Special Interest Groups such as Go, Python, and Rust. A central theme of the discussion was the overwhelming surge in Common Vulnerabilities and Exposures (CVE) filings, which some have termed a "CVE apocalypse." To address this challenge, FESCo is actively working on enhancing metadata standards for languages like Python and Rust, aiming to help product security teams accurately map upstream issues to Fedora packages. The committee acknowledged that while many of these filed CVEs are low-priority or already resolved, the current state of bug trackers often lags behind reality due to poor ticket quality, a situation that necessitates automation rather than manual intervention.
The conversation also explored the unique value of Fedora's RPM packaging ecosystem, which provides consistent metadata, versioning, and integration capabilities that other ecosystems like Go or Node.js struggle to match uniformly. While panelists recognized the complexities associated with dependency trees in certain languages, they emphasized that Fedora's tooling, particularly its support for version ranges in RPMs, allows new language stacks to be integrated without compromising quality. However, significant concerns were raised regarding bottlenecks in the package review process, where human reviewers are currently overwhelmed, making it difficult to process AI-generated submissions quickly. The group cautioned against relying solely on generic AI models trained on outdated data but suggested that artificial intelligence can be highly effective when guided by up-to-date documentation and policies, potentially through mechanisms like Model Context Protocol (MCP) servers or specific contextual inputs.
Looking toward the future of Fedora and responsible technology adoption, panelists advocated for the use of open-source, transparent AI models that run locally on consumer hardware rather than depending on expensive cloud-based solutions. They encouraged close collaboration with the AI SIG to integrate these tools responsibly while maintaining a "human-in-the-loop" approach for critical decision-making processes. The session also touched upon strategic initiatives such as reviving inactive Special Interest Groups like Security and leveraging projects like Hummingbird and the Critical Risk Assessment (CRA) to improve long-term stewardship of the ecosystem. Furthermore, the panelists addressed persistent documentation challenges, noting that existing guides are often too voluminous for new contributors, and directed attendees to an upcoming talk dedicated to improving the accessibility and usability of Fedora's documentation resources.
Read the full video transcript
Good morning everyone. Welcome to the
Fedora Engineering Steering Committee
Panel or FESCO as most folks know. Um
joining on stage is some but not all
members of the uh FESCO the Fedor
Engineering Steering Committee. I would
first like to wish a congratulations to
our newly elected members. We have Neil
Ga, we have Fabio Valentini, we have
Maxwell G. We also have Simon and I
can't I'm not going to pronounce his
surname because it's Yeah, I can't do
it. Simon, well done. Um, and I believe
that could be it. No, there's five. I'm
missing a fifth.
One moment. Technical issues.
The technical issue is my brain memory.
We have a fifth, but congratulations to
the the current and the the newly
elected members.
I'm
>> okay. Uh
>> yeah,
great. So there is technical issues.
>> Yeah. [laughter]
>> Please stand by.
[laughter]
>> None of us like curling. Jeff,
[laughter]
>> I don't know if that's controversial.
repeating.
>> Thank you.
>> Michelle.
>> Okay.
>> Excellent. Okay. We are live on the
stream. Jeff wasn't even in the room for
my comment, which is great. And also
Michelle Lind is our fifth elect for the
Fedora Engineering Steering Committee.
So, congratulations to Michelle as well.
Thank you.
Okay, without further ado, I'm going to
pass it over to our FESCO panelists.
They're going to introduce themselves
and a short description of who they are,
what they do around Fedora, and then
we're opening it up to you.
>> Hi, I'm Neil. I've been doing this for a
while. Uh, and I sit on Fesco to
hopefully do things that help people
make a better uh project.
>> Hi, I'm Kevin. I've been around for
forever. Um, and uh, likewise I try and
uh, help people out. I try unblock
things. I think I have a good amount of
history that that I bring to Fesco uh,
uh, just from, you know, ages past.
>> Hello everyone. Uh, I'm the new one,
Maxwell. Um,
so I have been contributing to Fedora
for probably like five years now. Um, I
do packaging. I am a sponsor. And then
I'm also very involved in the Go
language SIG and the Python language SIG
where I work a lot on macros,
automation, various other fun stuff.
There's recently a big change to the Go
packaging ecosystem, which I think went
pretty well. So, I'm happy about that.
Um, and then I also am part of the
Fedora packaging committee, um, where I
help with the guidelines, help review
stuff, help fix stuff in the packaging
guidelines, which I think are very
important, and now I'm here to help
steer things at a higher level. So,
thank you for voting for me.
Okay. Uh, hello again. I'm introducing
myself again. [laughter]
Uh, still Fabio Teeth is my username.
Uh, yeah, I I've been on Fesco for like
this is the sixth time I've been
elected, I think. And um, yeah, I've I'm
interim uh, council representative for
FESCO and for Fedora. I mostly do rust
packaging these days working on packager
tooling and uh I'm also a member of the
packaging committee
>> and my name is Bishek. I have been on
fesco for I think eight years. Um I work
on systemd and I also work on various um
tooling uh parts of the packaging story.
So uh RPM autospec uh most recently
um add determinism which makes package
builds
reproducible.
Um
uh and I would like to
also work more in the future on various
other parts of the packaging story to to
and the packaging workflow to make it
smoother.
I'm still double.
I'm also pretending to be Timothy Ravier
as well as if there are any questions
for Timothy. He's here on chat in the
matrix room. So feel free to message
him. Am I not loud enough? [laughter]
>> Internet people, Timothy, I'm going to
pretend to be you for a little while,
but don't worry, you can tell me exactly
what to say on the phone. Uh, so
questions from the audience for our
fellow panelists.
Don't be shy. They're very nice.
>> What is the current state of CVS in
Fedora? Is there anything happening? Did
we realize that something is happening?
Is there any coordination like any
anything we are trying to do or might
do? Any help that that would need? Yeah.
And what was the state actually?
>> Uh so there is a a ticket this was
actually something that we started
looking at like last year. Um and so the
whole process uh red hat actually the uh
production uh security folks actually
file tickets for CVS for Fedora packages
helpfully. Unfortunately, they also, you
know, sometimes get it wrong or it's the
not the right package or there's issues
or you etc. Um, so we've actually opened
a dialogue about like trying to fix that
and make uh those bugs more useful. And
of course, you know, as you know, the
the CVE apocalypse uh has hit really
hard this year. Um, there's been a lot
of kernel ones, there's been a lot of
browser ones, there's been a lot around
the project as a whole. Um
I I don't know. I think this we're in a
kind of an uncertain time right here
because is this something
[clears throat] where they are there's a
lot of CVS being found because the
tooling has gotten so much better the AI
stuff uh and we're going to get to a
point where we're back to an equilibrium
again or is it just going to continue to
be this way for you know the foreseeable
future and it's it's really hard to
predict that but I think we definitely
can improve the process a lot and and
we're working on that. Um I think that
yeah there could could be a lot more
help that maintainers could get for
handling stuff like that. Um so yeah
it's it's a it's a hard problem. Um
>> uh yeah I mean the latest thing which uh
Fabio proposed is we're working on
adding additional metadata to every
package b or not every package but
within language systems like for all
python packages for all rust packages
etc. Um, so we're adding the
standardized metadata to these packages
and the goal is that product security is
going to be able to consume that
metadata to basically be able to
properly identify which Fedora package
maps to which upstream package because
historically that has been a huge
problem where like there's some NodeJS P
no there's a CVE and a Node.js JS
library that gets like filed against a
Python package and then it's like we
have to look at it and like this makes
no sense and we've started an effort to
track those issues using like uh
Bugzilla tracker bug. So we've kind of
that's helped us identify it and we're
really hoping that this new proposal is
one not going to create any extra work
for packagers because we already have
generators that create metadata for
Python packages and cargo packages and
that all happens automatically based on
the upstream metadata. So all all we're
doing is adjusting those existing things
to just add another one that's
standardized across every language
ecosystem.
>> Yeah,
>> thanks. Oh, sorry.
>> Yeah. Uh I just wanted to add some small
thing because I think we are actually as
a project not that bad at pulling in
fixes for security issues. Just the
state of a bug tracker doesn't reflect
this. So there are a lot of bugs for
CVEes that are still open despite the
bugs having actually been fixed already
just because people to some degree
stopped bothering to look at them
because the quality of them is so bad.
So we're trying to fix that to to have
the state of backtracker also reflect
that things aren't actually as bad as
they might look.
to add this to this. I think that u with
the sea apocalypse happening, this is
something that we cannot handle manually
and the current processes that were
designed for manual work and we need to
automate this and I really hope that
once we move to the new forge with this
it we can I don't know figure some more
uh
less human resource intensive ways to to
deal with those things. Uh, one one
final thing on the CV stuff. If you just
look at Bugzilla, uh, at all the CVS,
you you go, "Whoa, look at all those
CVs." But there's a whole lot of stuff
that's very low priority, is not
exploitable,
is not even arguably a CVE. And so I
think a lot of those things accumulate
and I think we need to discuss as a
project whether we want to what we want
to do with those. Do we want to track
them and just close them? Do we want to
continue to track them and keep them up
until they that we know they're fixed?
That kind of thing. So that that's the
discussion we're got to have.
>> Yeah. Thanks. My goal for why I was
asking is that like within a packet team
we are working on a like automated
solution on central stream side. So I I
was curious like if there is a de demand
and people interested in having
something like that on Federa. So we can
probably like discuss later if anyone is
interested.
>> Yeah. Yeah, I mean if we could I know
that packet has the ability to
automatically do new upstream release. I
know to do automatic upstream releases
in Fedora. So if we could update that to
also look at the CVE metadata and
automatically add the you know the
closes RHBZ whatever the security bug is
when it knows that that new upstream
version is the version that fixes that
CVE or some other tool that does that. I
think that could be very helpful.
>> Yeah. Hi, Lucas from BC here. Sorry for
all the bad CVs. Uh yeah, me and all
other folks we are working with Federra
for improving those and thanks Fabia for
the PL change proposal. Uh actually I
wanted to ask is there someone from the
Federa security sig here because we are
trying to find those people here and
there is going to be a talk about the
CRA today. If you can come there, we can
have a chat after that because we are
doing the gap analysis on the security
side on Federra and we are trying to
improve like everything
and yeah we want to meet you guys and
have a chat.
Uh, no. I just um for data on the CV
flood in case it's interesting for
people, not speaking to any of the
points the council made about how many
of these are real bugs, whether they've
actually been fixed, but between April 1
and, uh, June 15 last year, there were
122 CVE issues filed. In the same period
this year, uh, 1919. So, there is
definitely a CV flood.
So, I was wondering if um sorry, I had
to switch gears there for just a second.
The the uh I wondered if there was a um
a response to an earlier question that
happened in the council. So I feel like
this is a more appropriate place to ask
that question around the encapsulated
binary uh and curl and just uh um and
the like how how we see that as a as
from an engineering perspective. And
then secondly,
um I wanted to ask about uh the
changes in the way that Jeff was talking
about how we have multiple vendors at
this point. Um uh how we can be flexible
on the way that our packaging models
could be could incorporate uh some of
flex some of the flexibility around
those vendors.
>> Um I can answer the second one. Does
anyone want to answer the first one? So
um in the in the discussion there were
two points raised I mean in the answers
I think it was Alexandra and mirror but
I'm not sure anymore. So one is that we
the packaging introduces a basic level
of quality. Uh and the second one was
that the packaging allows for
integration and and those two things
well you can do do neither of those with
car uh pipe bash. Uh and I think that
um
so I think that like as was said before
we we should not sacrifice on quality
but we should figure out what parts of
the process and once what parts of the
packaging story are not relevant anymore
what we can automate what we can remove
and so on. And this should must be an
ongoing process and we must make
decisions all the time not following the
the latest trend but what we think is
technically reasonable. And uh to start
on the second question.
>> Oh yeah for the first one I thought I
did not realize you said kernel bash. I
thought you said kernel and I was like
when was kernel mentioned in the last
thing but
>> but yes about the the issue with
packaging in Fedora. I do agree with
what was said. our packaging work and
our integration provides a lot of value
to our upstreams. Like I was just
talking to um Carol from the Python uh
team who works on the Python rebuilds
and like everyone like people in the the
AI desktop whatever were like saying
like oh it's so annoying Fedora has this
new Python and it doesn't work with
anything and like the point of doing
those Pythons early is because we test
them early we fix things. we push those
things upstream and we help those
upstreams get ready for the next um
Python version. And that's like
something that we do that is not only
helping us but it's also helping the
community at large. And
I think that's valuable. And as for like
the issue another issue that's been
brought up with packaging is the issue
of managing complex dependency trees.
And I think like in Python and in cargo
upstreams provide proper metadata for
that for the most part. Like there's
version ranges and we have tooling that
consumes that metadata and translates it
into RPM metadata. So just really making
sure that everything is integrated and
that the versions are compatible and
everything's installable. But in other
ecosystems like forgo and no.js, we
realize that doing that is just
completely infeasible. and we've
switched to a different approach that
doesn't
sacrifice what we do in Fedora like we
still work with upstreams. We still
follow the rules about making sure that
the bundled metadata is there and that
the licensing is there. So I think for
certain ecosystems that is something
that we need to evaluate but I think the
tooling that we have especially for the
langu other than go the other language
ecosystems that I do work in I think is
working for us for the most part.
Um to add on to Maxwell's point, we have
extended uh the RPM ecosystem tooling
specifically to incorporate features and
expectations from new language
management stacks, new uh new features
and things like that. For example,
version ranges was a feature that was
added to RPM specifically to support
Rust and Python's emergent use of of
this stuff. But I also want to add
another bit to this to kind of overlay a
a an u a a point here.
One of the
hugely underrated qualities that we have
within Fedora, it's a side effect of our
integration process, but I think it's a
valuable point in itself is you can
understand regardless of where a piece
of software came from, what it is, how
important it is or how unimportant it
is, how broken it is. Uh and from a from
a quality perspective, from a version
perspective, how out ofdate, how new,
what what is included in it because we
rationalize all that into a common set
of terms and a common set of queries
that you can use that makes it so you
can understand coherently at scale. And
this is a problem that you know when you
go outside of the Linux distribution
space that people are really struggling
with. there's you know the package URL
stuff came out of this idea that like
there is no coherent way to understand
this because if you don't have a
unifying delivery mechanism you don't
have a way to have rationalized metadata
and when you go beyond that into even
like how do I understand what a version
means in in different ecosystems and in
different stacks they have different
meanings. Go has an absolutely insane
construct of how versioning works
>> in which versions don't mean anything
>> or they could just be arbitrary commits.
It's ter that's why we changed it. But
in other ecosystems we don't have that
problem,
>> right? But like it I I want I'm
underscoring this point that like one of
the most underrated things that we have
in Fedora is that as a side effect of
our quality and our integration
work,
it is easy for someone to go in and say,
"Hey, I know that this thing is bad and
this thing is good and and this thing is
new and this thing is old and this thing
is probably a little wibbly and this one
is great because there is a consistent
fabric that we have to understand each
and every component within within
whatever deliverable we have and
you know there's a lot of people on the
internet that like to say that you know
this stuff is outmoded and we can just
go in all the different ways but I think
when you think about at larger scales
this this becomes a a valuable property
that is really not talked about that
much. Um, so we we have RPMs and RPMs
are just tarballs with a some metadata.
Uh, and they are quite flexible and I
think that we should uh keep using them
because they are nice. They they work
well for this purpose. But we should
also uh have in mind the fact that most
likely people will be using immutable or
um atomic type installations much more
uh like as as the primary uh
installation method in the future and
that is good but it does not mean that
the the packaging work that is happening
stops being useful and to to answer to
Neil's point about knowing I think that
uh knowing about the quality well we do
have a problem with this because yes you
you know the version you know who did it
and when but we don't know uh how often
a package is installed and and where and
I think we should start gathering um
uh installation data for packages and
have uh like this has been talked for
many years people are afraid of privacy
uh issues and privacy and and blowback
from people who don't
uh
data to be gathered but as I think that
as a project we need to do this because
to a large extent we are flying blind
because we don't know if a given package
is installed one or 100 or 100 thousand
thousand times
I'm following up with that um I'm
working on packaging um more complex
projects and making them possible for
example on atomic desktops like I spend
a lot of time improving the Nvidia
stuff. Um all the curl um install
scripts um I come around with um that's
what I'm improving. Um and up until now
I always struggled in finding resources.
You've talked about those tools you have
internally in Fedora for um making
things easier um making things faster.
Where do I find docs for that? Um
because especially for me it's usually
starts off with I have a um operating
system I install something then do a
operating operating system diff to see
which files changed and move on from
that.
Yeah, I mean as for documentation,
anything about packaging, the first
place to start would probably be the
packaging guidelines that the FPC Fedora
packaging committee, which I think is
technically a FESCO subcommittee, but no
one really publicizes that. But anyways,
go there. Uh, generally like like or at
least for the new guidelines and the
ones that I've written or the ones that
I've reviewed, we're kind of trying to
focus on making the guidelines
prescriptive.
um and trying to not like be a tutorial
but kind of always they provide an
example spec file at the bottom and that
is generally always the best place to
start and sometimes there's additional
examples or additional tutorials or for
go what we've done is the guidelines are
prescriptive I mean they're written so
people who package go can understand
them but they're relatively prescriptive
and concise but they explain things that
need to be explained and then There's a
spec file at the bottom and we maintain
our own documentation for the packaging
tooling that actually does provide a
tutorial and says like if I need to do
this, how do I need how do I do that? If
I need to do that, how do I do that?
>> Is there a list of all tools or scripts
or utilities that exist for making
things easier?
>> I mean generally a lot of stuff is done
on like a per ecosystem basis. So like
there's stuff for Python, there's stuff
for Go. I know for like the QT or cute
and um KDE stuff, they have some of
their own macros. So gen generally it
kind of depends on what you're trying to
package. But there are stuff like RPM
lint, Fedora review, RPM inspect which
kind of are a general tool for checking
the quality of RPM packaging and uh also
like if it's not in the docs then it's
completely okay to go to federal develop
channel uh or the mailing list or
discourse. Oh, but actually the the
matrix is the best option.
Speaking of tools and making um like the
packaging experience easier like what's
your point of view on like use cases for
using AI tooling?
>> Okay. So just in my personal experience,
submissions that used that tooling that
I got to review were pretty bad
just uh based on outdated information or
on old versions of the uh packaging
standards. So I don't know if that's
just because those systems were trained
on old data or if that's some somewhat
uh old things being more
amount of old things being available for
training. So I don't think that the
current systems are very useful just
because they don't use the new standard
stuff.
Uh so I think that AI is okay but you as
a person submitting it must take
responsibility for every line that is
submitted. Um and I think this will
happen and it I mean it is happening and
it's it's okay. It's just that we don't
want the quality to decrease. But uh
also packaging is about having a like a
scriptable workflow that works for
hundreds of packages and then for tens
of updates for each package every year.
and AI is a fairly expensive way to to
achieve that. But yeah,
Alexander wanted to make a comment.
>> Yeah. So regarding that um I find out
that the um contemporary
AI tools actually pretty good if you
point them to the right um source data.
So for example, Fedora packaging
policies are available. They they can be
downloaded. I mean you do get check out
of that content of the policy. Then you
point the agent to look into the policy,
look into the RPM to Rust to RPM and it
will produce you uh something that after
one or two iterations
faster than the the human will do that
will produce the actual results that are
within the policy. The problem we have
is not that the problem we have is a
bottleneck as a human because if you
submit new packages and for example I
have two applications each of them is
using 400 crates
roughly 60 crates need to be packaged.
um if I'm submitting those crates
simply because they used as the um
source code dependencies they never used
as endpoint like executables or
something. It's a source code. Uh
getting anyone to even review them takes
months. This is
taking more time than I get time to
polish these in copper bits.
This is this is big problem. Um finding
out whether a particular spec file or a
packaging is against the policy is
trivial because believe it or not most
of the tooling like code and and codecs
and so on they can read Rust RPM Python
code they can infer all stuff you coded
there in in the um Python scripts that
handle it and they infer and apply these
changes much better than a human does.
So that's that's the the the problem. We
we don't rely uh nowadays if we do this
work, we don't rely on where these
models were trained on because they
always lag behind it. If you point them
to the material that exists today, maybe
the material itself is not good. if if
the end result is rejected. But I'm more
concerned about the fact that if I
submit a new package request,
it's not being even looked at for
months. That's our problem and that's
what we really need to look into.
Okay. Uh just
Come on.
>> Thank you. Uh, so
if you know what you are doing,
am I getting an echo or not? Yeah, just
got to sorry.
>> Yeah, I I will not go upstage. I'm not
on fesco anymore. Uh so if you know what
you are doing, you know where to point
the agents to and you know how the
result is supposed to look, you get
pretty good results. If you are a new
person and you just open a chat and say
I want a spec file for this, you will
likely get AI slope or Yeah. So as I
told previously, writing spec file is
easy and even if the AI can make it
easier, I don't see it as a big deal
when the bottleneck is the review. And
we can even have automated tools that I
want this new thing, the new Ras app in
Fedora. Oh no, it has 3,000
dependencies,
3,000 spec files like that. We don't
even need AI for that because the
tooling that creates the spec file, it's
almost perfect, I would say. Uh
the problem is how the hell am I
supposed to get 10,000 spec files
reviewed? And this is something that
need to be fixed
both on technical level and on human
level. And on technical level I imagine
a world where I send all the spec files
as a pull request somewhere
and they are all built together or CI
tested together or checked even by AI if
we want to against our guidelines. They
are checked by RPM lend, RPM inspect,
whatever installability checks and then
a human reviewer still in the loop
somehow they can approve this in a bulk.
They can read the results of the CI
test, point out that there is a weird
spelling of color in one of the spec
files or something that important and
then they can provide feedback on the
individual lines of the spec file in a
system that is built in this century.
Uploading spec files and source RPMs to
somewhere and then linking them in
Bugzilla is horrible. And I think we
need to fix that.
>> Yeah.
um to but to to just add a like a well
maybe a short answer. uh you actually
need two people to because you cannot do
this alone but with two people one
person submits uh 40 spec files another
one does 40 and they cross check it and
then it's okay and
uh so it's actually not I mean I I agree
that the current process is clunky but
it's it can be it can be done
So I have a statement and a question. I
start with a statement. What worked for
us in in hummingbird is really making
docs available to the agents. The agents
by themselves just like any human they
they don't have the magical knowledge.
At some point they will guess if they
don't know. Um how we solved it is just
by having a rest api available with all
the container images that we built with
all the texts with all the CVE data and
including documentation. So this way the
agents are now perfectly capable of
navigating the space building container
files, Docker files, knowing what to
look at. So um just making such an
experience available in Fedora in case
there's an appetite for it. Just want to
say that this has worked out pretty
pretty well for Hummingbird.
The question that I'm interested in is
I feel like the the answer or this
conversation immediately circle circled
into something focusing on the community
itself. But I think there's a unique
opportunity right now for any
distribution to play a bigger role in
the AI world in the agents. So I'd like
to extend the question to how you I'm
sorry Neil but
the agents will be your new users for
some time. So the question directed to
Neil here maybe
how do you think Fedora could play a
more important role going forward in
this new time. you know that for
instance agents would default to using
Fedora containers instead of Alpine or
use uh Fedora host instead of Ubuntu.
>> Oh my god. I mean he called me out. I
don't have a choice.
I guess my mixed feelings aside about
the the the value proposition or even
the long-term viability of of agentic
workflows and stuff like that. Um
I think a lot of the stuff that the a
IML SIG is doing is probably going to
allow us to be in that space more. Um
the bigger problem that we have right
now is a lot of the this ecosystem is
operating at a pace where I my head
spins like every 30 days on this stuff.
Um, I kind of peripherally track it and
like I've also experimented at least
once or twice with like Fabio has with
seeing how it works when you use, you
know, LLM tools and chat bots and
whatever to interact with this stuff and
the experience ain't great. But um I
think as we look towards having um our a
IML SIG folks work on building out the
tools, building out workflows and
looking at thoughtful integrations of
that tooling like for example one of the
things that I've been collaborating with
them as Fedora KDE is talking to them
about improving access our accessibility
stack experience for in in
non-traditional ways leveraging um the
hardware acceleration support that comes
from some of the libraries and and and
modules that come traditionally from the
LLM stack people. And so that sort of
thing is going to also lend itself
towards a direction of where people will
want to do run these tools, these agents
or chat bots or models and things like
that. Hopefully more in Fedora
containers and in Fedora environments
rather than some other type of
environment. But beyond that, I don't
know what else.
Yeah,
>> I want to jump on it and give an answer
uh for it. Uh because I'm uh involved
also in AI projects and everything else
and you cannot trust AI today.
Therefore, you need the human in the
loop. That's really important
unimportant whether you are using
agents, whether you are using chat bots
or anything else. erh if you want to
verify that all is running correctly,
you need a review or something like
that. And therefore, if you want to
develop AI in the community and provide
AI containers with Fedora Foundation,
uh you need human in the loop uh for a
specific time until all is trained
correctly and that's a real hot
requirement.
I just wanted to add something to what
Neil said earlier. Um I think one thing
we can do is to focus on making things
better for everyone. Write better
documentation, make our tooling better
and then it to some degree doesn't
really matter who is actually in ing
inesting that information whether that's
an actual person or an AI agent. So I
would rather us focus on things that
actually benefit everyone and not
specifically on things that are only
geared towards
uh systems like that.
>> Okay. So uh first I want to uh thank you
munchuk for mentioning that yeah the
processes needs to update and second
thing uh you said basically that the AI
generated thing is not up to date or
might be updated or anything I would
rather than fighting against that or
saying okay it's bad you shouldn't use
that go with path let's try to uh guide
the AI engines. Let's try to provide a
skill, provide an MCP server, anything.
There's like the skill is super easy to
like super super cheap thing to point it
to documentation, explain the explain
the situation and provide that visibly
on the Federa packaging guidelines or
anywhere like related and the models
will learn later and will try to like
even suggest maybe to to the user. I can
imagine that we will see. But I also
want to point out one thing which I see
missed here. Uh it's not just I will
open like uh open AI or whatever and
type write me for this project the spec
file but the great benefit in it is uh
the gu it is able to guide the user to
understand the processes to understand
why it's this way. If you' open the
documentation for the packaging
guidelines it's huge. It's really hard
to find anything there. the AI has a g
huge benefit in I can spend like 10
minutes on learning something which in
documentation take me half an hour to
understand.
>> Yeah. Uh one of the things I was going
to add and this ties in with your uh
question uh you know when the rise of
search engines and Google we had search
engine optimization are we going to have
agent optimization? Yes. So yeah,
exactly. It's it's so I I don't know
that there's not an answer here, but
everyone should think, okay, so if
somebody types, you know, make me a
container that does this and this and
this and this, how does the agent decide
what container? If you didn't say
anything, it just generically picks
something, right? There's a reason
behind that or a commonality behind that
or a uh something in its training data
or whatnot. So,
you know, in order to to capitalize on
Fedora being used for agents or uh
things like that or being used in the
way that we want them to be used, we're
going to have to kind of figure out how
do we do that? How do we convey that
information? How do we
convince the ecosystem to do that that
sort of thing? Um, and I think the the
point about the docs is a good one. Um,
and I think that's something that maybe
we could explore as a project.
not a general uh you know somebody
asking a question from a general LLM but
having you know something on the doc
site that says ask a question about docs
packaging docs or something like that
which is trained specifically on that as
a local model etc etc so
>> um one thing to add to that if you read
documentation and you find that it's
hard to figure out something we should
consider that a bug and please report
issues for that so that we can actually
improve the documentation
and not just have everyone who looks at
it be confused and ask OpenAI instead
that doesn't scale.
to follow up on that uh to Alexandra's
point about having it being hard to
package. Um let's also try to sit down
maybe with Fabio and we can figure out
if we can move this process along and
the same applies to everybody else who
has a problem. Uh I think that flock is
an excellent opportunity to to to sit
down and push things along.
Yeah, honestly, usually in open source
like bumping things is kind of
considered rude, but with package
reviews, there's just
>> that's kind of what you do and
everyone's fine with it. Ask for a
review, offer to trade reviews and
that's it.
>> It's also expected that you do that.
It's actually written in our um new
packager things that yes, you should
just go ask people, just go poke people
with a stick. Anyway, uh the other point
I wanted to mention is, you know, with
the suggestion of, oh, we're going to
run an LLM to to give people the ability
to do queries and stuff. Um, while in
theory that sounds nice and actually
does kind of sound nice even in
practice, I don't think we can
realistically afford to do it. Um, it
would be extremely expensive and Fedora
runs on a shoestring budget of paper
clips and uh glue. And so, uh, I'm sure
that the idea of having built-in GPUs to
run LLM agents even locally to do search
queries sounds like fun, but it will
also make us go even more broke than we
already are. So, I don't see that
realistically happening basically ever.
>> And honestly, maybe you can doubt that
some of us are pessimistic about AI
stuff, but if you want to make your
thing, like make your thing and people
will use it or people are not going to
use it. like we approve things that are
like fundamental changes in the
distribution or that like affect other
components, but like this is an open-
source project where as long as you're
not like violating existing rules, you
can just make things, you can just do
things. So, I invite anyone who's
interested in exploring how we can, you
know, integrate AI into whatever, like
maybe it's not something that I'm super
interested in, but as long as it's not
disrupting other people's work and
people are finding value in it or you
find it as a fun hobby project, you
know, go ahead.
>> And hey, if you happen to be a rich
person with a pile of servers with a ton
of GPUs and you want to run some kind of
massive supernatural language processing
index of all Fedora website stuff,
go for it, have fun and share it with
us.
>> And I guess I'll also add someone
mentioned making Fedora or you mentioned
also uh making Fedora container images
more useful. That actually is something
that I'm also interested in and I know
there's efforts to do that and I really
would like to see Fedora become the
operating system that people do use in
CI and that people do use for their
container images.
We have about six minutes left
>> because I've just seen Anchor go into a
second hand counting questions. So
>> I'm gonna go first. Right. Okay.
So as a proud user of Fedora, um
security is always my top priority and
concern and everything.
Also I'm super excited about the new
initiatives like hummingbird and etc.
which is amazing. Um also I have heard
about this little thing as the USA
resiliency act which may affect the
whole uh open source ecosystem but also
it's I've heard that it can bring some
of the really meaningful uh security
improvements uh to the projects.
Um but I'm less familiar with uh Fesco
um kind of official procedures and
governance and etc. And my question is,
but also I'm talking to people here at
the flock and a lot of them are
interested to actually contribute to the
um technical improvements to security to
quality. So the question is what is the
uh best way how fesco can help to run or
re or revive initiatives like security
sig and everything so that we can come
and collaborate on those improvements uh
together.
>> Yeah. That's a difficult thing. Uh, one
of the things that we've learned or at
least that I've learned over the years
is Fesco's power in in the organization
is to say no to something, right? And
that's it. We can't we we don't have a
team of engineers that we can say, "Hey,
you guys work on this thing or sometimes
we have fesco members. We like amongst
ourselves say this is important. Can
somebody really work on this?" Um, but
we can encourage people. We can try and
uh, you know, fix process that sort of
thing. Uh I think the security SIG has
seen an uptick of late. It uh was very
inactive for many years, but in in
recent months, there's actually folks
that are getting involved. There's
meetings uh that take place now. Um
they're actually discussing stuff.
They've made a a forge tracker, uh
things like that. So that's very
encouraging. Um, another thing that that
plays into the volunteer part of Fedora
is, you know, these SIGs or initiatives
show up and they say we're going to do
this thing and everything is great and
the project you're doing really great
work and then the people sort of drift
away and there's not as much kind of
goes inactive again and you know that's
that's just I think a common problem in
a volunteer organization. you can't
force somebody to go work on something.
And I think also, you know, people see
that thing as really active and they're
like, "Oh, they don't need my help. I'll
go work on something else." And, you
know, that's not necessarily the case.
So, yeah, there's a lot of things that
we can try and do to cajul things, etc.
But it's it's a difficult problem.
And to add to this very briefly, I think
the CRA is a great opportunity for us
because it's cut out for uh to to
advertise the work that distributions
are doing actually caring about
long-term stewardship.
But uh
a group needs to come forward and
proposes for Fedora people who
understand the CRA who understand the
distribution and can figure out how to
integrate this and encourage everybody
else to
support that.
>> I think we have time for one more
question maybe two if you're short. Just
while on this AI topic just one small
thing uh those are doing all the
integration work please also consider
the transparent truly transparent models
the work opensource AI sort of abused by
many of these big vendors uh they are
open weights only but there's also up
and cominging open source models federa
really I think should consider when I
say federa those are integrating into it
AI tools instead of just hooking into
open open AI or or closed AI or um
anthropic stuff. They're all good, but
for the long-term sustainability, you
don't know when they'll pull the plug as
as we all know. So, focusing on there's
this Allen Institute uh model called
Almo 3. It's not great, but it's
something they they what they advertise
is they they open up the entire model
flow what they call not just the
endpoint. So, things like this should be
explored a bit more. I don't see that
much but I know everybody is sort of
strained for bandwidth. Just wanted to
add that note.
>> So one last question.
>> Yeah, it's not a question really. I just
agree with what he said because
basically you have a broad choice of
small models that you can use that have
open weights. You can use them as
embedding models. You can do all kinds
of stuff with it. And there is a work
being done on the back end side where
you don't need expensive GPUs and you
can run the model locally. You don't
have to pay anyone. You can train it on
your own data. You can still update it.
It the model can be very current.
Probably what's Hummingbird is doing
with retrieve augmented generation. Make
the model a small model capable running
on consumer hardware. That's all.
>> All of those sound like great ideas.
Please go talk to the AI SIG about it.
That's that's their thing. They're
interested in that stuff. I've even
talked to them before. They're
interested. If you are excited about
that stuff, go talk to them and work
with them on it.
Uh I wanted to uh make a also a comment
that uh I mean we try as as FECO to do
the right thing but I know we often are
slow or fail or don't satisfy everybody
but if there is something that should
happen then please uh reach out uh
formally or informally and well we we
all want to improve Fedora so I I think
we can make things happen.
We have time for one more question. We
can squeeze one more in.
>> I don't know who was faster either.
>> And uh I want to make a comment about
the documentation. Uh I worked three
years at packaging for OpenStack and
RJO. So I needed to learn and to read
the documentation. But when I talked to
my friends and they wanted to start
contributing they they felt the
documentation it is too big and it's too
confusing and and we have some examples
there but um we don't have a quick
recipes to give them h for example I
teached some of my friends to just bump
a package and they they h they like it
it so you can start having a more simple
recipes because we want more
contributors, but it it's hard to to
start when you really read that big
document. So that's what how I felt as a
user and and about all the friends I
have around me.
>> Yeah, I I don't know if we have time,
but
>> um you know, there actually is a really
good documentation talk at 2:30 today.
That might be super to get your answers
from there. Not that Fesco are not
capable of answering, but we're really
edging into a new talk time, but 2:30
p.m. in the Opal room, I believe, for
docs.
>> Thank you everyone.
>> Thank Thank you to our FESCO panelists.
[applause]
Thank you to the audience. Thank you to
the matrix room people. We have two,
three minutes to swap over to our next
talk. If you do want to contact the
FESCO reps, they are um they're very
active on mailing lists and also in the
Fedora development fedora infrastructure
rooms.