Foundational Infrastructure Working Group, August 27 2026

Hello peoples. Hello. Good evening. >> Evening to you too. Hello. Good morning everyone. should actually say good afternoon to everyone else. I'm sorry keep forgetting I'm not the only one who's just started drinking my coffee for the day. Not drinking coffee at all. just naturally this uh sharp [laughter] mod. But yeah, >> Aram, I was meaning to ask um are you planning to visit the CF summit this year? >> I am not this year unfortunately. Um I was hoping that I could make it but I have um I won't I won't be able to make it over for this year. I'm hoping soon I will be able to make the next one. I'm not sure where it's at. >> I think it will be uh it hasn't been uh discussed, but it's probably going to be Europe. I think they got rid of the North America one after it was there was no attendees or like really low attendees last time. >> I I showed up. There were two of us at least. >> Okay. [laughter] >> Yeah. And they instead made it a two-day event now in Europe. >> It's great. Yeah. I would have liked to have come to this one. I think it's near H Highleberg now. >> I think it it is in H Highleberg. No. >> Are you going to go Ruben? >> Yeah. Yeah, sure. Oh, we have a Wayne. Are you know do you know if any of the Broadcom people are going? Uh I mean probably sales but from engineering. >> Yeah, possibly sales. I don't know if any engineering folks are going. Um, Broadcom doesn't really provide conference budget. So, um, I think even if folks are speaking, they won't pay. But that's I haven't I haven't checked. Possibly some folks from the runtime like Amelia maybe. I'm assuming you do have some people in Europe. >> No, >> we don't. I don't think we have any more people. Reuben was one of them, but I don't know that we have any other engineering in Europe that is working on cloud fun. >> No, they got rid of most of that. >> No, >> there was Constantine, there was me. >> Um, There was a team some team in >> I mean there still is >> maybe yeah but they did other stuff uh that's more for the uh was it uh docker images I don't remember and there's one in London uh team >> harbor >> no no I don't know. Um, it's like they have a distribution of images, but they have like many images that they built. It's also a VMware products. >> That wasn't the Bitnami thing. Did they acquire or is >> Yeah, I think wasn't it Bami? >> Yeah, Bitnami was acquired by VMware before before Quum. >> Yeah. Yeah. Yeah. And I think that that's the Broadcom thing. And I believe that team sits in uh Seville in uh Spain. >> Yeah. >> Anyhow, who wants to run the meeting? >> Um I can drive unless you are excited about it. >> I think I did it last week. So, >> excellent. And there we go. All righty folks. Welcome to the foundation of structure working group meeting. And we you all can hopefully see the browser window. And away we go. Aram, you're bringing me back to the square CRT monitors. >> Yeah, should make this a little bit bigger. Um, spare you all having to look at my really messy desktop. All right. Um, Ned removing health monitor. Yeah, if you remember I mentioned we might be skipping the health monitor testing >> which is not the case but there was an ops file called remove health monitor which was not used. >> Yeah, this is in the bush repo, right? So this is not borch deployment. So >> okay. Yeah, we should this seems definitely a good thing to get away from. Thank you for tracking this down. >> And didn't sign. >> Yeah, >> the CLA. I mean, >> okay, this is >> a typo, but still. [clears throat] >> Yeah. Uh, >> I already asked. >> Okay, cool. And we're waiting for changes. Correct. All right. And did you ask on this one? Yeah, >> these I saw these earlier in the week. They're interesting oneline changes. It makes me a little suspicious that there aren't some sort of strange Unicode characters hiding in there. But >> yeah, so I don't know. We can have them sign the CLA first. That that should get rid of bots. [snorts] >> Yeah. >> Okay. I have some context on this. I worked a little bit with Coker on it. Um, [sighs] so this is about adding a Rosetta stage or a variant of the Bosch warden the Bosch OCI uh stem cell, the warden stem cell. Um, and what it does is it adds a certain number of ARM 64 binaries that are binaries that in x8664 don't work under Rosetta. Um, specifically some stuff to do with uh systemd and also we discovered uh the tar binary. Um, uh, Cooker's on vacation, so I'm sure he'll probably come pick these up or I can sweep through this. Um, basically the the the intention here is to make, uh, a variant of the stem cell that will run easily under Rosetta emulation on a Mac OS um, ARM machine. >> Nice. Um, so yeah, there's a quick check for uh an x86 uh an ARM 64 executable. And yeah, it looks for each of these packages, checks to see if they're downloaded. Um, and there is I don't know if it's in this one or one other one where there's a there's a slight change in whether something is being untarred outside of the chute by the host OS or being untarred inside of the chute by the host OS uh that we had to modify to work on a different side of that boundary because of the um because the tar executable didn't work under Rosetta and I don't remember which direction that flip-flopped. It's an ARM assertion packages. Okay, I'll fix these talks links or if somebody else feels like jumping in there. Yeah, I assigned I went over all the PRs that were open like 10 minutes before the call to assign the reviews of >> Okay. >> I find these co-pilot balance reviews uh they find a bit more. >> They find a bit more. Yeah, that's good. >> Um Okay, I'm going to approve and then um it Okay, one of us may feel free to push and whatever make this change if somebody feels like it. >> Um, close that one. Um, Ned, your magnum opus. >> Half of it. Um, so yeah, this gets us to a working life cycle and bats tests. That's amazing. Um the bump dependencies I think we can take care of after this. >> Mhm. >> And there was a create docker image task as well or job. >> Mhm. >> Um I feel like we need some credentials for it with permissions to to write but yeah. >> Aren't those already in the concourse? in the cred in the concourse in variables >> once I used um I think are returning unsufficient permissions. H >> I I would suggest maybe using the credentials for GitHub's um container registry GHCR um and just not not going to DockerHub if that's if that's an option. >> Yeah. Yeah. We moved more there, right? Like other places went there as well. >> All right. Yeah. I didn't look much into the the task itself. So yeah, this will be the next uh yeah next step. >> Cool. Shall we emerge? >> I mean um think it's good. It's tested. It's working. >> Awesome. Congratulations. >> Yeah. Awesome. >> Oh, um something I would like to add to it. [clears throat] There was previously discussion about deprecating Ruby >> and using Go instead. >> Um, Somia can correct me if I'm wrong. I think internally we're using the the Golang version of the the CPI. >> Yeah. Yeah, that's right. Right. >> Yeah. Yeah, I think there was some discussion right like on the slack from and some people from N9 they were they were surprised that the default ops file for open sack CPI in bush deployment still points at the Ruby one. I think that was their confusion. >> So I'm assuming we should look into moving to go as well. >> Yeah, I think so. Like first we need to make that yeah make that change in bush deployment and then >> Mhm. >> we can get rid of it. >> All right. Okay. >> Cool. >> Um yeah it's a huge step. Congratulations. Um this is some stuff that I worked on. Reuben I saw you approved and coer looked over this. It's uh when I was looking through the CLI to solve some other problem, I just ended up down a rabbit hole. It turns out that most of the mocks in the Bosch CLI are hand rolled um based on a slightly odd pattern and there was a large chunk of code that was using Gmach. Um so this is a rewrite of all of the fakes. Um, there is still one one package in in UI called fakes, but those are actually more like a test harness. Um, so I'm going to open a PR to to rename those once that's once this has landed. >> Um, yak shaving. >> Yeah, thankfully with some some help from Gemini being stupid. Um, so yeah, I'm going to go ahead and repaste this unless folks have concerns. All right, good. Away we go. All right, Foxmox Wayne, it's your time to shine. [laughter] >> So, I don't know where to start, but I think I got two PRs for this. I don't I don't I I'm not at my desk. still be there in five minutes. But >> yeah. Yeah, this is just adding it to the CPI the porch releases index. >> Okay. Yeah. So, we have an actual release that uh is now being used by others than ourselves. Uh it's on GitHub. We did the Apache 2 thing as well. And uh yeah, we'd like to get it into the Bosch IO releases. And uh also we we're working on we got it we got uh we got it passing the uh bats and uh is it called wash dash certification repo both of those um we have that running on our hardware. Don't know what that would look like moving in the future but yeah so we're it's we're open we're ready to contribute it and do whatever. Yeah, >> I think this one is good to go. Uh this one >> guy that I was working with in the community by the way for that other in the channel. Uh we we worked together and I took some of his ideas in into our CPI and then now now he's also using too. >> Yeah. Um Aram I think this one needs uh you need to run something right to create the pipelines. You're muted. Uh, sorry. I think that happens automatically, but I don't know how slash why. Um, because we merged a PR last week to rem to take out a number of uh releases that weren't syncing from the Boso pipelines >> and for about 5 days they didn't disappear and then early Tuesday they had gone. Um, I have I have no idea. Sounds more like a human has has finally done a thing because like 5 days waiting and then a thing happens or do we have like a thing that runs every week? >> I think there's a thing that runs every week. >> Okay. >> Um if you have any inclination to dig in. Um I'm the the Bosch.io sort of uh machinery is pretty hard to reason about. We have some we have like jobs that run workers that run code that >> yeah I know for maintainers after merging a new release remember to run the worker sync scripts to configure the pipelines that's what it says in the readme just need to read the read me I will I will look into it >> okay uh Azure security profile docs I think this is just ready to go some changes from shade. Um, think it's already approved. That can go in. All right. Uh, gem updates. And we've got a new gem vendored uh static interface configuration. Uh I think Wayne, I don't know if you're at a desk yet. to talk about this. >> I'm just getting in the house. Uh what's uh >> this is the static interface configuration grouping. Um I think you looked at this as well. >> So as part of the Proxmax CPI stuff basically >> um the way they deal with networking groupings and things like that. Um I was trying to do dual stack and this is kind of what I had to do to get it working. I'm happy to change it, but uh >> yeah, and I don't have a lot of context on this, but Ned, it seems like you looked it over. I'm guessing this is probably >> Yeah. Um I mean, it looks good. I think the reason we didn't hit this issue with AWS is because it provides different MAC addresses. So yeah, change makes sense. Um I also think it might address the same issue for Azure um which Sebastian recently um added of the tool stack feature as well. >> Oh, cool. Um okay. Yeah, let's let's merge it. Um maybe we want to I guess Sebastian will probably notice. Um, come in. All right, Wayne, you're good to go. Congratulations. Thanks for the Thanks for the visit. See, we have merge. >> Sorry, I had somebody come to the house. Um, thank you. >> No worries. Yeah, you're welcome. Thanks for Thanks for joining. >> Yay. I hope to do it more frequently. >> Yeah, that'd be great. >> All right. Rubocop errors and warnings. Uh >> yeah, just uh fixing some Rubocop errors. >> Oh, cool. Yeah, looks great. Funny thing is we in the PR template um have a point where it says yeah I have checked rubocop there's no errors after my change so someone was definitely [clears throat] going in there [laughter] >> yeah oops um maybe we should try to add that into like a a GitHub a GitHub action um at some point if it's not already there. >> It's already there, but I don't know how why it was m missed before. >> That's unfortunate. All right, I think this is Thank you again for that uh clean up. That's awesome. >> Thank you. >> Uh let's see. This. Okay. This is another part of that change from Matthew Coker. Oh, no. This is that same one. I just put it back in the wrong, not in the wrong thing, but I need to make these docs changes. Done with that. Um, this is the secure boot changes. I haven't seen any more folks chime in, so I think it seems fine to go ahead and merge. >> Sure. That's all stuff we looked at. This one, that one we looked at needs discussion. And then um seems like the Bosch boot loader maybe isn't um doing it updating its dependencies or something. Maybe ask um [clears throat] >> but did notice some of them were for OpenStack CPI. So these will should eventually be handled by the pipeline. >> Okay. Yeah. Yeah. I think those are those the OpenStack ones seem seem like they're probably safe to merge, but because the there's some failures in the Bosch GitHub actions, I didn't uh you didn't merge those. >> I prefer not so we can make sure that is working with them. Yeah. >> Yeah. Absolutely. Um all right. Uh Wayne, we got another one from you. um deployment key when SSH. >> Yeah. So, don't ask why, but I have literally like 30 SSH keys in my keychain and um this basically the idea is like just clear it to the one that Bosch is using >> is the idea. Great. Sounds like an opportunity for you to write a general SSH key manager uh utility. Why why limit this to Bosch CLI? [laughter] >> Because everybody is using Bosch CLI. >> Uh yeah, this seems this seems fine. It's just this is just a I tried to keep it small as possible, straightforward. >> Yeah, that's totally reasonable. I'm I'm good with this unless other folks have. >> Boy, this is the most contributions I've done in a while. >> Pretty much you'll be ready to apply for contributor here after this week. Oh, I don't know that that I'll slow that roll a little bit, but yeah, maybe maybe I do want to get more involved in this working group group for sure. >> Awesome. Um, next favorite one. >> Oh, it's the best one. Um, all right. Next up is your Proxmox VE. I don't know, Ruben, if you have any co-pilot. >> Oh, yeah. left. Do you want to add to this one? >> Uh, that's the bor >> got back to my desk. So, we have >> um where this is. We'll look real quick at this and see if Okay, we still don't have any that's waiting for changes. I'm going to walk away from that one. What's uh does that need more stuff or something or is it like >> um Let me take a quick scan through here. Oh, you got the CLA. So, that's done. I don't think we have >> Oh, yeah. I was wondering, but yeah, we want to wait for that CPI discussion, >> but that's the the gentleman in responding to the channel that will use this. >> Yeah. Yeah, I saw it. It's fine. Um, uh, I've assigned a a co-pilot review. >> No, I'm happy to change you want any more changes before I, you know, >> um, yeah, I don't I don't expect there's going to be a lot of stuff coming out of this. I just Code Rabbit had not recognized that you had addressed the um stuff. So, yeah, I think this is probably fine once we I'll go ahead and By the way, I think the pipelines so the >> Oh, nice. should soon be on the bar. >> Okay. Um I think we'll maybe we'll check back on this in the end of the meeting. Maybe uh co-pilot will have started. Um it's kind of good to find it's been pretty good at finding like sort of minor lints and stuff and we can reapprove if you want if there's something in there that seems worth fixing. Um let's see. A quick question on the pipelines that you mentioned, Ruben. Like how long before it shows releases? Is it like a nighttime sync or something like that or >> I think it's going to pick it up now. >> Yeah. But then it needs to I think it redeploys Bosch io changes. So you should see it show up in the uh releases index. >> Yeah. the uh the release shows up >> like >> yeah but there's a releases index repo should be pushed to there borch io releases something >> um there's a stem cell index and a releases index and those get >> p cf pushed no not cf pushed somehow I think they get it uses cfsh to do a git pull on those indexes on all the app instances it's a great system. Dimmitri really went uh all out on that one. >> Yeah. >> Doesn't do a blue green deploy. It just in SSH is using CFSH into other app containers to pull in the latest index. >> Okay. So, if I wait a little bit, it should theoretically show a version listed kind of thing. >> Yes. Do you do you have access are you member enough of the group to have access to the CI instance even in readonly mode? I guess maybe >> read only. I probably read only, but I don't think the the Bush io stuff is public enough. >> Okay. >> But it's borch.ci.cloudfoundary.org. Um main child Oh, it's XB.io. That's it. Releases. Um 520. Oh, it's paused. Interesting. Unpause. That should help. I think it's in because it's a new one. It was paused. This is the first one from 520. And they got split out by uh organization. Now, >> no builds, no triggers. Release index. Porsche io. Okay. Found a version. It's running now. >> Um, there were some look like just like spelling and like style nits in here. Wayne, if you want to take a look at those and either dismiss them or or >> Okay. >> address them and then feel free to ping me today in um in the CF Slack. I'll I can happy to approve and we can merge this. Sounds great. Thank you. >> Yeah, you're welcome. >> Cool. Close that. Um, I looked at this broken logo thing. This seems fine to me. Just a a minor path change. So, folks have concerns. I will merge that. Don't want to squash. I want to rebase. >> Oh, wow. We now have a a Borsch.io a person who cares about Borsch.io. >> Amazing. >> You did that whole upgrade, right? >> Yeah. >> Nice. >> Um, this is I want to put this in pending discussion. I'd like some folks to take a look. Um, this is some work to regenerate fakes and to use the sort of standard counterfeitter behavior rather than bespoke counterfeitter behavior, which is to put them in package name fakes. Um, this impacts uh blob store fakes, which are consumed outside of Bosch utils. Um, so that'll need to be updated. There's like I think the CLI and the agent both use these fakes. Um um once this lands up there they're the upgrades to the updating this package will fail until somebody fixes it which will probably me. >> [clears throat] >> If uh folks want to weigh in on this, I'll I this the change of the fakes didn't seem worth updating significantly updating the package um version. Um but I also realized when I was looking in here that the Bosch utils pipeline only ever updates the minor version and doesn't even have the capability of updating a patch or a major. So, um, that's maybe something we'll need to look at later. Put that in review. Uh, these docs ones we all looked over. Um, this is blocked. Waiting on some changes to the co-etcher that I have yet to figure out how to deploy. And let's see these are all dependabot. And then this one depending >> this is pending cla right. >> Al easy cla is one. >> Oh okay. That was another one. Okay. So this is >> Did I already assign a co-pilot here? >> Uh you have not. >> Okay. >> So I think this is just augmenting the pipeline. >> Oh, that's the backup restore one. >> Uh yeah, PBR SDK release. Yeah. I'm going to go ahead and remove the workflows from that. That was fast. Um, okay. I think we've basically done what we can do here. Uh, Wayne, feel free to ping me on the Roxmox thing once you've >> Sounds good. I'm working on it right now. Thank you. >> Cool. Uh, let's see. open issues. Uh, this was just a tracking thing that I opened. Bosch backup and restore is leaking PBL environments. This isn't the ones there. So, the Becca backup and restore CLI prefixes its bubble ends with BBR CLI. So, it's not the one that that is leaking a large amount. But this one of the failures like both both the setup and the and the insure tearown failed which left a a VPC lurking around. Um, just kind of leaving this as a breadcrumb for myself and looks like maybe this is addressed um suspens merge and X And um yeah, I think this is I'm going to assign Colin. Um >> is it prioritized? Is he going to work on it or is this >> uh I think that this is I'll leave it there. I think he's I think he is working on addressing it. It's it's possible to opt out in um in use to not use BPM or or Bosch workers. Um, this is a result of some work that Matthew Coker did to get all of the worker all of the Bosch jobs, including the Bosch workers running under BPM as part of the Resolute work. >> I see. >> So, I think I think Colin posted this mostly asformational content rather than a thing that needs action. Um, and I'll leave that there. Let's see if promotions. Well, nothing. Are we sure the automation is running? >> There's no automation for promotions. I think it's bay on labeling and bay on is on vacation. So in that sense the automation is disabled >> temporary. >> Yeah. Um I saw the thing I brought up last week about the um BPR acceptance tests not working. It seems like maybe the automation isn't picking something up. Um I don't know if you know enough about the automation to take a look at [clears throat] this ribbon. Um, I was able to like manually check a checkbox, but I'm not sure if that's been >> that the checkbox wasn't wasn't toggled by the automation when I PRD the change to use deploy keys. So maybe I'm maybe there's something some other special sauce in the automation missing or maybe there's just a problem. Um, >> when I update the PR is do I have to like do we have to like run another code rabbit thing or like how does that work? Um I think code rabbit will run automatically and when you push the um yeah I think code rabbit should automatically run if we have enough code ra okay >> and co-pilot is just depends the the person who assigns the if you uh like >> So, should I should I like do like the like slashcode rabbit or something like it? >> No, >> I think it will pick it up. >> I think it will pick it up. Uh let's see if it's if it thinks it's working. >> Sometimes it gets stuck from my experience and doesn't do a re-review. >> Yeah, I've definitely seen that. Let's see if that will trigger thing. Okay. Looks like some jacket. Okay. Code rabbit is churning. Um, yeah, I think once that once that I'm sure it will probably be fine, but it it's sometimes good at finding small knits, spelling errors, or as you saw earlier, docs references that are stale. Um, I think we have achieved the end of the meeting aside from me u punching punching merge on this later today. Wayne, is there anything else anybody has they want to bring up for the group? >> Yeah, sorry. I have a question. >> Oh, yeah, please. in the Bosch um Azure CPI repo there's um deprecation of unmanaged desks issue and uh I went to the Azure website and they have apparently um deprecated this already in 2025 and they said um in March of 2026 they will completely uh get rid of it and everything will be migrated automatically to manage discs. So, um yeah, I was looking at this and was wondering if we should keep the functionality of migrating from un unmanaged to manage discs or I just remove it completely. >> It seems fine to remove it since nobody could use it anymore. >> Mhm. >> Um >> yeah. >> Yeah. And I don't >> So they can use an older version of the CPI, right? >> Fair. Yeah. And yeah, that seems good. I think the only I don't think there used to be something we supported was like Azure private cloud or something, but I think that support was taken out years ago. I only mentioned that cuz it's possible that those private the Azure kind of an Azure version of OpenStack that you run on your own hardware was often very far behind the the like hosted Azure, but I don't think that's supported anywhere. So, >> what would you expect to do with it? Uh, Azure had something like like OpenStack where you could run an Azure instance on your own hardware. Um, >> or something. I don't know. >> Don't remember. >> Or was that a Google one? >> Yeah, I don't know. Um, but yes, aside from that, >> Azure Stack. >> Azure Stack. Yeah. Um, so yes, Sumia, please tear out the unused code. And I think as Ruben said it, there's it's pretty safe since somebody could always go back and use the older version if they had to for some reason. >> Okay, perfect. Thank you so much. >> Yeah, you're welcome. Thanks for being into it. All right. Uh I think I think we did it, folks. Um >> congratulations on making it this far through August. Hope you all have a good a good rest of your week. >> Just a few days before the new tokens roll in. >> Yes. [laughter] [gasps] >> Excellent. Take care. Take care. Bye. Bye. Bye.