Video summary
The August 27, 2026 meeting of the App Runtime Deployments Working Group was primarily focused on reviewing and managing various Pull Requests (PRs) amidst a period where most team members were on holiday. The discussion began with smaller requests, including one to add a new origin from configuration for the CFON client team and another involving a log aggregator feature that required specific tests running on local machines rather than in the cloud. A separate PR regarding the removal of Dora zip assets upon cancellation was approved, while a larger, critical PR concerning the cut was kept open due to a dependency requiring a specific version of CFCLI greater than 1231, preventing its immediate merge.
The conversation then shifted to operational issues and configuration updates, specifically addressing the transition where Novell became the default runtime environment instead of Ubuntu. This change necessitated adjustments to ensure that ads are not installed incorrectly during deployments. A significant portion of the meeting was dedicated to resolving a critical issue where the cron league deployer job had become stuck for over ten hours due to compatibility problems between an updated internal MySQL driver and self-signed certificates from an external GCP database. The team discovered that manual intervention, such as unlocking jobs and deleting VMs, was previously required, but a fix has since been merged by the copy team, ensuring that future triggers should automatically resolve the issue without manual cleanup.
In addition to these immediate fixes, the group acknowledged several ongoing improvements and backlog items aimed at enhancing system stability. These include implementing time-out mechanisms for stuck jobs to prevent indefinite hangs, releasing a new agent from Broadcom to fix failing Windows validation tests, and developing a more resilient smoke test framework. Furthermore, the team is working on a proposal to make the stiff Linux FS 5 stack the default configuration for safe deployments. With these items addressed or noted for future work, the meeting concluded with no high-priority blockers remaining, allowing the team to proceed with their respective holiday schedules and plan to reconvene in two weeks.
Read the full video transcript
Hi Dave.
>> Good evening.
And good [clears throat] morning from
our side.
>> Looks like we're uh
a lot of people on holiday at the
moment, I would assume.
>> Yes, it's on Europe's side. It's quite
um
I would say holiday season still.
Indeed, quite a lot of people are
vacation at least to enjoy the
uh glimpse of the summer that's going
away.
It was quite long.
>> Nice.
>> Cool. Um then I think so um let's start.
I mean um cuz I'm pretty sure that no
one is joining. Maybe Melina will join.
But beside that one, all other
colleagues are on
vacation. So maybe I will share my
screen.
>> Sure.
>> Um
>> [clears throat]
>> Can you see
my screen now? Now? Okay, cool. Awesome.
Um
So yeah, for today as an agenda, I think
so we almost 90% of the
the thing that we want to discuss is
related to the PRs. I mean um
or maybe we can start from top to
bottom.
First one are the cards PR. They are
small and one uh kind of a big PR that
we need to look for it. I mean there's a
I already
requested the reviewers on this one but
other one but it's a small one that we
need to approve it.
Um one is this one, the read origin from
config.
>> Okay.
>> specifically is almost 2 days ago it's
open and I think so there's a new
elevator of origin that has to be added
and yeah, this has to be reviewed.
I just checked it. There is nothing
specific except that the version they
have tested against.
Uh but now that is yeah, that is from
the CK the team that works on the CFON
client. Yeah, Panama.
>> Okay.
>> Uh they haven't joined it but yeah,
nevertheless this is something I see
that's being open and we need to review
it or maybe if they have any questions.
>> Mhm.
>> [clears throat]
>> The other one is from the log aggregator
colleagues. I think so they want to add
this new feature for this is log agent.
And for that one,
they want to have some tests um and
yeah, there is a PR.
And this is also something it's Sisler
rain filter.
Um and yeah, this is something I haven't
got really time. I need to look into
this one. This is not
as of soon, yeah.
I mean, a little bit I was curious about
that the test roughly 8 minutes, but
it's running from the local machine. So,
I mean, that's quite different than one
that running in the cloud.
So, maybe I will reach out to them if
they could possibly treat them like
this.
>> seems quite a long time for a small
number of tests.
>> Yeah, yeah.
But, it seems to be a local machine. So,
yeah.
Maybe we will try to look maybe if we
can add a comment onto this one. But,
yeah, this is also the one.
And a similar colleague also had a small
PR.
Which is I think so it's clear that they
want to remove this Dora zip, which is
this assets.
When someone cancel it, so it's into end
test. So, they have to do it manually
and they say that
yeah. So, I approved it. I just leave it
for the
for the colleagues if someone wants to
get it, but yeah. And the last PR for
the cut was the big one that's still
there, but I think so we cannot um
I think merge this one. There is a
reason why we cannot merge it. If I
remember correctly, there was a comment
from Johan
which points out that
it needs CFCLI greater than this
version. So, I think so that's not a
kind of only the 1231 we should look for
it, but this one is
>> Got it.
>> Yeah,
still we can keep it open. Yeah.
Cool.
Then, maybe we can move to the next one.
Um
Should I move it this up to the top?
Maybe we I can move it like this because
the reviews things are
uh kept together. Um and this is to the
C department. It is also very a small
request or they want to add as an
experiment.
And I think so it's a lot about this
experimental to just an Ubuntu. I think
so we can accept it and yeah.
>> Yeah, that makes sense.
>> Because obviously now Novell is almost
yeah, they are going
>> Yes, I'm surprised we missed that, but
yes, we should look for other
>> Uh indeed, that
>> Yeah, other um
I guess these are runtime configs that
we need to
>> adjust it accordingly.
>> Yeah.
>> Because Novell is now default, so
obviously we have to watch out for that
when these are the ads on and
won't be installed, yeah.
Cool. And these were the small um
related PRs, quite important. And the
last one I want to talk about is this
um maybe if you have seen it in the in
the selection I have posted about that
the the cron league was stuck.
The deployer break and it stuck was like
quite good amount of time. Maybe we
should also introduce some time outs. Um
uh for more than 10 hours. And the next
time it was more than Yeah, so we should
introduce some time outs.
Um by digging into a little bit, I
checked out that there are like a
scheduler VM and the um the API VM, they
were having some problems in terms
of um if you look at this new copy
release, it has this internal MySQL
driver which they updated it. And
because we are using external database
from uh from the GCP, it has a
self-signed certificates and this does
not support it. But in the end of the
day, it should handle it.
Uh it did
and it got stuck forever.
So what we uh what we have to done is
log into that, uh manually unlock the
stuck um job or the task, and then
manually delete a VM from the GCP
console, which is not a good option, but
that was the only thing which was left,
you know?
Um
did that once and re-triggered it, but
obviously the issue was not, and then we
reached out to the copy team, and now
the fix is there. I think so it's
merged.
Um if I checked in the morning
Yeah, so they disabled this one, they
merged this PR, and I think so the copy
is out, so by tomorrow
Yeah, this one should have Yeah, this is
there.
So with the tomorrow trigger, they
should able to pick up again. If not,
obviously we have to do the manual
cleanup and then trigger the job again,
so it will unlock
this whole pipeline.
Mm. Yeah.
Oh, that's an update.
I think so that's it as of um
what has to be what is on the plate. But
before that, we maybe we can check from
the previous one. Uh yeah, this one is
also done, so the Windows flip um
validation which was failing. So yeah,
thanks from Broadcom side, they have
released a new agent.
And the Windows deploy job was done.
This is still old.
Um still uh
we have some other priority stuff, but
this is something we already have a
backlog item to make um
to make the smoke test a little bit more
resilient, stable, and yeah.
Uh and also the last thing, this is also
something we are working on it to Yeah,
propose and bring the PR to make the
stiff Linux FS 5 stack as a default in
the safe deployment.
Yeah.
I think so that's all from my side.
And as of agenda, there is nothing that
is I would say with highest priority or
something that is blocking us, so I
think
>> Now we I I've heard the there was a PR
to address the UAA.
>> Ah, yeah, yeah. Okay, yeah, yeah, yeah,
yeah, you're right.
>> So, I I reviewed and merged that and
uh
>> I should ask you
>> the CI jobs. So, so they went they went
green. So, that that looks good.
>> Ah, yeah, okay, you mean this one that
was
>> Yeah, exactly.
>> Okay, okay, yeah, yeah, yeah, that's
totally makes sense. Uh
Thanks for reminding me. I will try to
add this one to agenda.
At least this was there and there was a
PR.
Thanks Dave for taking care of this one.
>> Yeah.
>> This one has to be added. Cool, yeah.
Exactly. Okay, cool. Then I think so I
can add this to the notes. Um
That's uh thanks so from my side. Do you
have anything else, Dave? Regarding
anything else that needs special
attention?
>> No, not at the moment. I'll try and find
time to take a look at some of those
cats PRs. Um
>> Okay, that would be great, yes.
>> Yeah, we can get get those merged in. Um
And then yeah, I think we're in good
shape.
>> Perfect.
Then I think so
I'm not expecting anyone to join us
almost like 10 10 minutes off.
Then we'll call the day off uh from my
side and from
for you I would say a very nice rest of
the day, good start and then we catch up
each other in next in the next 2 weeks.
>> Sounds good.
>> Perfect. Awesome. Yeah, thanks a lot
then, Dave. Have a nice rest of the day.
Bye-bye.
>> Bye-bye.