Open Source Maintenance, 2026-07-18

Hey folks, welcome back to uh another Rust stream. Um this is going to be one of the open source maintenance streams. So um I've done a few of these in the past, although it's been a while since the last one. Um the intent behind these is well I maintained a bunch of different open source Rust crates. Um I maintain some non- Rust things too, although most of those sort of fallen by the wayside um at this point. But uh keeping up with the open source stuff takes a bunch of time. Um there there's a lot of like reviewing issues, reviewing PRs, fixing issues, doing releases, uh you know, updating dependencies are going to depend upon, fixing CI, like there's a bunch of that kind of maintenance that, um I think many people who are not used to being open source maintainers don't really see um they don't see it both in the sense of don't appreciate that it's happening and the maintainers have to spend time on it, but also they don't get to expose to it. So they don't know that those are things that if you are to release something in into open source you have to spend time on and it will consume some of the like it's not as though putting an open source library out there is just a matter of putting it out there. You also need to sort of sub subsist those projects over time. Um and this is an intent to sort of show a little bit of what that requires. Um and so in particular we're going to basically go through my GitHub notifications. I've already pruned a bunch of things of there. I mean, I'm obviously subscribed to a bunch of, you know, Rust issues and like tracking issues and stuff, things that are not that interesting to look through on stream. Um, so we're going to specifically look through for things that require like my maintainer attention. Uh, and then we'll just start walking through them and dealing with them as I would normally. Um, this is a task that, you know, I have to do once a week or ideally more often, but often I don't even have the time for once a week. So just somewhat regularly I have to sort of cycle through these and and uh keep up with PRs and issues and everything and you'll come along for the journey. Um now worth pointing out that one of the things that makes this uh challenging is that there are many different projects and so we're going to need to do quite a lot of context switching. Anytime we switch between repos there's going to be a bunch of context you don't have for that repository but that I have in my brain. So I'll do my best to try to carry you along here. Um, but just keep in mind that the the transitions might be a little jarring. Um, but I'll I'll do my best to to have that not happen. Um, and as always, like if if you have questions, put them in chat and then I'll I'll try to address them. If you're watching this after the fact, we'll put the questions I I'm in on a piece of paper and and make a paper airplane. I don't know. I don't know what to do about them. Um, but maybe you'll join the next stream and you can uh ask me live. Cool. So, um, as I pointed out, the one of the problems here is that the notification button on GitHub is behind the image of me on the screen, and I have a little danger box there so that I can see that things aren't visible to you. But that also means that I can't click on the notification thing. So, I need to go to notifications. Um, and then what I'm going to do is, and you can't see this offscreen either, but I'm going to sort from oldest to newest instead of newest to oldest because it tends to be the case that the oldest ones are the ones that most require my attention because well, I haven't addressed them in a while. Um, this also means that uh it's sort of a a slight punishment for people who keep bumping issues. Like if you are if you want me to reply to something, bumping it is not the way to do it because makes me less likely not more likely to look at the thing. Um, cool. So, let's uh group this roughly speaking by repository so that we don't end up switching back and forth a lot. Um, so you'll see there are a couple here from left to right. So, let's pull those out. Uh, and I think it's yeah, there's only four. So, for context here, left right I'll I'll um pull it up here first. So, left right is this uh concurrency primitive that I've given a a talk or two about this in the past. Um and it is a concurrency primitive that acts a little bit like a read or writer lock except it is a lot faster for reads but it is also slower and less memory efficient for writes. Um and so it sort of makes a slightly different tradeoffs in the in the in the concurrency performance space uh compared to something like a reader writer lock. Um, this one has a lot of like low-level um low-level like uh concurrency manipulation and atomics and stuff. Uh, but that's part of what makes it interesting. It's also part of what means that whenever I go back to look at this, there's always like issues or pull requests that are like very specific about some aspect of the the concurrency protocol. So, I always need to like bring it back into my brain in order to be able to address them. Um, so let's see. The first one is fix the data race on right handle drop. Um so if we pull up left right um you'll see that when you create a left right uh here for example uh you call left right new uh and then you give it a uh you give it two types. One is the type of the data structure you're protecting. The other is the type of the operational type over that value. So in this case um this is an like in left right unlike with a a reader writer lock for example you need to define what operations are possible to do on that data type so that they can be replayed later. Um so left right from a very base level it keeps two copies of your data structure a left and a right copy. uh and so the writer is always modifying one of them. The readers are always going to the other one. But that means that you all your rights have to be applied twice. Once to the left and once to the right map uh or data structure and so um you need to keep a log of all the rights you did so that after you've applied them to this, you now need to remember them so that you could apply them to this later. And so that's what this this operational type is. So it's generic over two things. Um, and when you create a new thing like this, you get back two handles, a write handle and a read handle. The read handle is clonable. So you can create as many as you want. Give each one to a reading thread, for example. The right handle is not clonable. Um, and so it is a thing that only you can only have one writer unless you put this in a mutex or something. Um, and so the right handle is where a lot of the logic lives for swapping between the maps, keeping track of the operational logs, uh, waiting until all the readers have moved from one map to the other so that it's safe to write to one. All of that logic lives in the in the right handle. As when they say here fix the data on the right handle drop, they're implying that when you drop a right handle, there's some kind of data race uh, presumably around the axis to the two maps. Um, let's see. that allows read handles to enter while the value is being dropped. The issue was observed by building with thread sanitizer in a test where one thread creates a right handle and shares a read handle factory for it with some other threads. A read handle factory is a a thing that can make new read handles but is not itself a read handle. Um the reason this can be useful is because the work that a writer has to do is proportional to the number of active read handles. uh read handle factory does not increase the performance penalty of a right handle. Um and so you might have a factory so that you can produce read handles if you need them but they are not themselves read handles. Uh T1 continuously creates and drops right handles sharing the factory accordingly while the other threads create new reles from the factory and attempt to enter. Uh so enter here is um if you look at read handle um on read handle you can call enter and that sort of means give me access to the underlying data structure. I want to do a read uh and you get back a read guard and the read guard is the thing that actually dreferences to the underlying data structure. The need the reason you would need to have this guard is the same the reason you need to have a a guard for something like a reader writer lock is because while you hold the guard uh the writer can't modify the map you're reading from. Um but the moment you release that guard you're sort of giving up your um uh your ability to have access. I don't want to say exclusive access because readers share it with each other but to have access to that and you will need to do more work to coordinate in order to enter again. Um, okay. Uh thread sanitizer here is a is a very common concurrency checking tool that essentially um you often run it in CI but you don't have to where it um it keeps track of all the memory accesses by all of the threads in a program and it checks that um none of them do atomic accesses in such a way that they could race like two threads uh concurrently writing to the same memory location without having done some kind um atomic for that right for example to ensure that they have not stepped on each other's toes without realizing um so Tzan reports a race where some threads manage to read a bool within V while T1 drops it uh co-authored with Claude surprisingly common these days um yeah so you'll see they posted this in April uh then they did a follow-up in May and I just haven't had a chance to look at it since because I've been moving and stuff. Uh this is sadly one of the realities of open source uh software as well. So let's see what's the change here. Okay. So this is in the um uh in the swap function. Yeah. Okay. So Oh, it's not in swap. It's in drop right so we have an impul drop from right handle for right handle um and what right handle will do is it will um uh take the inner and the reason this is a take is because I want to consume the the fields that are inside of right handle um whereas drop only gives you an exclusive reference to them uh and so if you actually looked at the definition of right handle which we can do over here. Um uh and then I will do let me uh see if I can move this so you can't see it and then I will mute myself so you can't hear me. and then let's try to pull again. Excellent. Um okay. So if we now look at left right here and we look at the right handle, you'll see the right handle um here. So if we go to the take inner Um, take inner is the thing that takes out the the backing data structure. Um, and the backing data structure here is if you have a right handle to a T, it basically gives you back the T so that it can be dropped. And this is a tricky operation, right? Because there are actually two T's, the left and the right. Um, and you need to only give it out of the take inner for like actual dropping. um once there are no readers in left or right because if there are any readers left in either of them then now um you're trying to drop the data structure while someone is reading it and so that's not okay. Um so if we go back to right handle you'll see that it holds um a W handle which is a non-null so a pointer to a T and it holds an R handle uh which is a read handle to a T and read handle inside is itself an atomic pointer to a T. Right? So there's two pointers to T's here. One is the right handle, one is the the read handle. Um, and so these are the ones that we need to get get the owned T out of because currently these are just raw pointers. So we need to get the owned T outs so that we can um let them actually be dropped. And this is where there seems to be a um a race of some kind. Yeah. So you'll see the the docs here too. Uh make sure that all the pending operations are applied and waits till all the read handles have departed. Um then it uses drop first to drop one of the copies of the data and then returns the other copy as a taken smart pointer. Ah so this actually I I misremembered this one. It it will actually drop one of the copies and then give you back the other one owned. It will not give you back both copies. Um right. So it keeps track of if you've already taken then it it refuses to give them to you again because if they've been taken you've given out an owned copy. you're not allowed to give out another one even if you technically still have the the raw pointer. Uh first ensure both copies are up to date. Yeah. So this is doing so publish is the thing that makes sure writes are applied to both sides and and swaps the atomic pointers and wait for readers to depart. Um and then we grab the read handle and we set it to null and then we wait for all the readers to depart. Yeah. So this is making it so that any read that happens will observe that the read handle um is now a null pointer rather than a pointer to a real data structure. Uh and then we do this um this wait call and wait is the one that will wait for every reader to have seen the last change to the atomic pointer. Right? Right? So we swap it to null and then we wait until all the readers have now observed a null. Um and then once we've done that now all the readers must have observed a null. Um and therefore uh they all know that the data structure is null and void. And so therefore it should now be fine to drop drop one and give back the other and take ownership of the other. Um, but this is where they claim that there's a bug. I'm sure this one is um last episode with a snapshot taken after the null swap above. Without this refresh, wait would use a snapshot taken at the end of the previous publish and would skip any reader whose epic was even at that point. Ah, okay. So, there's a bit of nuance here. So if we go to publish um publish's job is basically wait for so let me try to explain the algorithm top to bottom and then explain how it's actually implemented. Um so you have two maps left and right. Initially the writer writes to left and the readers read from right. Uh and there's an atomic pointer that points the readers to the right and away from the the one that's being written. The writer then has exclusive access to left. um data structure. So it keeps making modifications to that. Um and then when the writer wants to make a change, it makes a change to that that writable side and then it does an atomic pointer swap. So all all readers in the future will now go to here. Um and then it it um then it waits for so every reader keeps a sort of um atomic epic. It's a it's an atomic counter that's per reader and it waits until it's observed that every reader has ticked its counter at least once which implies that it's read the pointer at least once since the swap has happened. So they must now be reading the the new pointer value rather than the old one which means they can no longer be in the in the old map and this is what the read guard guards against. Um so you you swap it over and then you wait for everyone to move. uh and once everyone has moved now it's safe for you to use the the right hand side for writing because the left hand side is now uh for readers. The way this is actually implemented is slightly different which is that publish starts by waiting and then it does the updates and then the swap. So you'll notice that the wait isn't doesn't happen after the update, it happens before. Um the reason for this is a is a little bit subtle but basically um we do the swap and when we've done the swap we've applied the rights and when we do the swap reads will start immediately seeing the new values that we've written and so you still have this sort of consistency guarantee that when publish returns new readers will see the new value. However, there's no reason to really wait until the all the old readers move over as well before you return from publish. So, you might as well just overlap in time that waiting with the like the code that runs after publish returns. So you do the swap and then you uh read all of their epic counters uh and then you just return from publish and then the next time someone wants to publish which implies they want to apply some rights then you uh only at that point do you go and sort of wait until the readers have moved on because chances are in the intervening time between the last time you returned from publish and the next call to publish some time has passed. So readers have probably moved on. So this wait might actually be really fast. Um but this means that the the last thing that happens when you call publish is a swap. Um we can go look at this um down here. So we do a swap and then we read all the epic counters and then we return which means that after this point uh readers may still be in both maps, right? There's no guarantee that the the readers have have all all moved to one map. Um and so the question now if we go to take inner is so we try to ensure that all the readers have moved over and so yeah so the oplog here are like pending rights that we've not published yet. Um so if there are pending rights then we do a publish. So at at this point uh all rights are applied to at least one map or at least one T. Um uh readers are all in. So, we've waited for them all to move and then we did a publish, which means we did a write uh we did a a pointer swap. Uh and then we did a we did a wait, we did a pointer swap, we did an epic read, which is the readers uh may be in either t and so here after this publish all rights are in uh both t's i.e. there are no pending rights. Um, and so here we have the still have the same property that readers may still be in either t. There's also a world in which the oplog is empty the entire time. So we don't call publish at all like there there have been no changes in which case readers may still be in either t. So uh here readers may be in either T. Um epic is the last scene epic uh is the epic uh epics are synced with the end of the last publish. And then we swap to null and then we lock the epics. I see. So there's a world here where so we need to go look at what actually happens to um uh when new readers are created. So the the structure that keeps track of all these epic counters for the readers is an arc mutex of a slab. The slab is not super important here. Um but it's just an arc mutx. You can think of this sort of as a vector. Um, and inside of that is an arc to an atomic u size. Um, you need the double arcing here because this arc is an arc that's shared between one reader and the writer, right? Because they both need to have access to that reader's count um epic counter. Uh, and we need one of those for every reader. And then we need a mutex around that thing so that we can add more readers over time. That requires inserting into this this effectively this vec. Um and that needs to be shared between the read handle factories and the writer so that the read handle factories can make more readers. They need to have access to this mutx and so that's why the outer arc is here. Um and so if we go look at um factory you will see that uh cloning a factory is easy. You just clone both the arcs. Um the inner here is the atomic pointer to the t. Um this is the atomic pointer that we'll we'll swap. Uh and this is the um the epic counter arc mutx. Now when you create a handle you do this thing new with arc on read handle uh and that will obviously create a new epic counter and then it takes the lock and it inserts the epic counter that we got. And that's all you do. So you just you just add to this list. Um, and if we now go back uh to not here, back to here. The other thing we need to look at is what happens in publish uh in wait um if you have new readers that have come in. So you'll see that what weight will do you see we take the lock outside of weight and then what weight will do is it will um you know it keeps track of what the last epic it saw for every reader is um and it will resize that to um account for the current capacity of epics. So this one is is stored inside of the right handle is not behind a mutx. Uh this one is under the mutex that we're currently holding. Um, and so we extend this by nulls by by zeros to match the the capacity here. And then we loop through uh all of the epics and we check whether they're even. So let's ignore the even rule for now. It's not terribly important, but we basically check whether the epic counter that we have is the same as the one we read last, in which case they've not moved along or if it's different like here, then it must have seen the last swap because they've exited, they've dropped the read guard, which increments the counter. Uh, and so this is the case where you no longer need to check that counter. Um, does this mean that dropping the right handle could take arbitrarily long? Yes, exactly. It will wait. it will have to wait for as long as it takes for all the readers to depart. Um, and so the problem we run into here is if we go back here, uh, or at least what's pointed out in this is that what if a new reader was added, um, after one of these publishes, but before this and it has to be before this right because after this we've taken the lock so that there's nothing uh there's nothing they can enter. So if a new reader is added here, um then that reader will be in either map, which is fine. All readers are in either map. Here we'll swap to null. That reader will not have seen this yet. Then we lock the epic accounting structure and then we wait. Why is this weight insufficient? Uh this weight should wait for all readers including new ones to have seen the null. Why does it not? Oh, I think I know what the bug is. I think the problem here is actually this zero. Uh yeah, I think that's the problem. Let me try to explain why. So the rule is not actually that the readers increment their counter once for every read. they actually increment it twice for every read. They increment it when they um like the moment they read the moment they've read the pointer and they increment it again the moment they drop that guard the read guard. The reason they do it twice is so that if you have an idle reader like one that just sits on a thread that's doing nothing, you don't want that to block right handles because it's not holding a guard. it it hasn't read the pointer. It doesn't you don't need to block on it, but you need to observe the fact that it's idle. And you can do that by observing whether the counter is even. If it's even, that means that that reader is not holding a guard right now. Um, but you also still need to keep track of this like not equal because you could have a reader that's active, but it's active after having read the pointer a bunch of times. So, you still can you it's fine. You've it has moved on to the map you've swapped to. The problem here is if there are new readers added, we add them in with an epicounter of zero, which implies which is an even number, right? Mod 2 is zero. And so therefore, when we go through this loop, new epics uh sorry, new uh new read handles we will not wait for because they will hit this check. And so they will not block our weight returning. And then uh obviously they'll be filled in uh where they'll be filled in in um so in publish there's update and swap. So in update and swap at the end we read in all the epics. So after the the next publish after a uh a new epic counter is added then we add read in their last epic so it's no longer zero but before that so if we if we do a publish and a reader is created after that publish then its last epic will be zero which will be even which will be ignored for weight even though it might already be holding a guard. So I think the actual bug here is that this needs to be a one. This needs to be not even. Um the default right that's what that argument is called. Yeah that the uh defaulted value here must not be odd. Uh and we'll put in a ref to number 144. Uh otherwise new readers since the last um publish won't be waited for. And this this solves the problem because it reads all of the epics and then calls wait. But that shouldn't actually be necessary. Um it's fine for those epics to be stale. Um because actually is it fine for them to be stale? That's another interesting question because imagine that neither publish happens. So this is for sure a bug. But imagine that neither publish here happens. So there's some very old publish that last read the epics. Um and then we swap. Then this weight will wait for them to have observed some pointer swap. they'll they'll have observed at least some pointer swap since the last publish, but that could just be the pointer swap of the publish. It is not guaranteed that they've seen this pointer swap. Um, yeah. So we do actually also need to here read all the epic counters again so that we're when we wait we're we ensure that we actually wait for them to observe this swap not just the swap that happened in here for example. So this is actually correct. Um Interesting. Yeah, this description isn't quite right. I mean, it it gets at the problem, but it's not quite the problem. So, I think I want to write my own version of this one. Um, this is not entirely uncommon with PRs like this where um I could either try to guide them into what I want this to say or I can just write it myself. Um so actually what I will do here is this and then I will do uh minor left right gets stash um and then I will check out PR 144. Uh why? Why? What? Really? Okay. 53 2 one F826. Yes. Yes. Yes. Yes. Yes. That's fine. Excellent. Uh now can I check this out? Excellent. U. So now I'm on their branch. And so now I can go in here and say um refresh last after the null swap above. Um otherwise we would only be waiting for the readers to have observed some swap since the last publish but not necessarily our no swap. in particular. Uh they may simply have observed the swap at the last publish. Um and then this needs to be one again. Uh here actually it can be zero because we immediately overwrite it. Um but we we should keep it one in in um and then now so this is no longer true, right? This is where that comment should be. And this is uh well I guess this can stay here. And then this is uh uh and now block until every reader has moved on. Do the actual blocking loop till every reader has moved on. Yeah, I'm not a huge fan of um of uppercas and comments. I don't I don't know why. I don't know where I got that from. Um it would be better to actually have the test case here. So, I'm going to ask them to um to maybe add that in. Um, and then the other thing I'll do is I'll put in my changes here. And I'll actually leave this as zero and then I'll make the 0ero to one change separately. Um working on publish fix um and then I'll dash pump and then now uh maybe in Okay. And now we'll do this. We'll do a reset uh head to bring back that one little X change I did. Um and then here we'll do this and say uh newly added epics. Uh avoid ignore is skipping newly added epics in weight. Um a zero would be missed by uh would be considered even is is considered even. Uh and so would be skipped in the loop even though that read handle may have done um may have done rights. I may have done reads. There's actually another interesting question here of is one even the correct value because uh this means that if the reader does any value like assuming the new reader was added and then it's done a bunch of reads. Um then maybe it's correct to ignore them because or maybe it doesn't matter because they must have been created after the publish and so then the swap has already happened which means they must have observed the swap. up and so therefore it's fine for us to not wait for them. Yeah. So I think actually this change isn't relevant. I think it's it's okay to ignore them, but the reason is a little subtle. So maybe it's actually fine for it to stay zero as the default value. Okay, I think I think I believe myself again. So that means these can stay zero. Uh although I do want to leave a sort of note to myself here about that fact. So we're going to do check outp. We're going to check out this and then we're going to go here and note that the folder value here is odd. So, new readers since the last publish uh since they were created after the after the swap. Anyway, um however, since they were created after the swap anyway, uh they must have observed that swap and so it's fine not to. And so we don't need to wait for them. So just a clarifying comment. I'll leave these two clarifying comments in there as well. Uh clarify why zero is okay. Um and then I'll push [sighs] uh yes I do want to I don't understand why I'm not allowed to do that. I thought it would sync this to that remote as uh what I've definitely done this before where I could just push to it. Does it say I'm not allowed to push to this because if so that's pretty annoying. Um it also needs to merge with main. So we'll merge that in there. Um but oh did I mean even? Did I misspell? Oh the default value here is even. You are completely right. Oops. Um fine add uh pretty I was pretty sure that GHPR checkout would set up the remote, but I guess not. Okay. So, then we'll have to do this the old way. Uh, which is I need to do uh this is really annoying. Uh, get remote add um this and then get atgithub.com that slashleft right.get Get get fetch uh get to there Freddy. Aha. I think that's why I'm not allowed to push to this PR because I think they've not granted uh the ability to for committers to push to their branches. which I guess they've turned off which is annoying because it means I basically can't take their change. Um well then we will do it uh in our own branch instead. Um so I will then go ahead to main. I'll how do I want to do this actually? I think I want to um candle drop rice and then I'll reset that to this and then I'll commit um race condition in right handle drop um closest 144 four. Um yeah, I guess I'll put in the description here that um uh previously the weight and take inner I guess also affects um right handle take Uh, previously the weight intake inner would um could return even if uh could return if some new uh read handles had only observed the last publish but not the swap to null. uh fix this by uh reading the epics again after the null swap. Um and then way to observe them take over replaces 144 since uh I could not push to their PR branch. want to make some changes to the comments. So, I'll push that up like so. Um, wait. Oh, that's not what that's not what I wanted to do. Uh, that's not at all what I wanted to do. Um, I did a stupid because now I think I just undid all those changes. Yeah. Um, what I actually want to do is reset um I want to reset hard to Freddy and then I want to reset the branch to main. Now I have the right thing and then I want to get commit A like so. So this should now be right. Yes. force release. Okay. Uh and then I will go over here and I'll say sorry for the delay. The middle of a house move. So time has been typed. um wanted to make some smaller modifications to the comments here, but it seems I can't push to your branch. So, I'm landing this in definitely a bug here. Uh, one request though. Could you um open a PR where you add the um the TSAN test case that we catch regressions on this in the future? Closing comment. Now let's see what's this unhappy about. It's unhappy about codev is sometimes just unhappy. So that's fine. Um I can't approve my own PRs, can I? No, I cannot. Okay, that's fine. So this now looks good. This just takes forever, which is fine. So, we are going to It is me who've trained AI to use M dashes is true. I love M dashes. Um I've used them a lot way before AI. Cool. We'll merge this. Um and then I'll keep it I'll keep it open and in tab because we uh if I do another release of left right to to include this change, then um I'll want to post the version number that I release it with in here. Uh but I want to see these other ones first whether that's something we need to change. Uh can we wrap wait and publish under condition for first publish? Uh on first publish call there are no readers on the right handle. So we can just wrap that in not on first. [snorts] I see you don't need to wait because you know that there are no readers in the right handle. So you can update it straight away. I think that's true. First there are no readers can be on the right handles and no need to wait. Yeah, I think that's true. The only thing we need to be careful about is whether there are any side effects to wait. Um, but I don't think there are like it doesn't read the epics or anything. So, uh, would it make sense to drop a co-authored by them on the commit? Probably. I mean, I reference their MR, right, or PR? Um, and so the the the the sort of chain of custody here is clear, but but it would have been good. I agree. Um, I simply forgot that that is a a thing you can do in git. Um, I think this is correct. So, I'm fine taking this one. Um, uh, on first publish, no readers are are in the right handle, so no need to wait. approve. Just want that one change to make the comments more in my style. And then I the update with a rebase. It shouldn't be a problem. I think it's just the CI that's changed. [snorts] And so this is another change so we can get into the same release. Uh so I'll go back to main here. And so once CI is happy there, this is done. Um relax re-entrant null pointer assumption. The re-entrant read handle enter seems to have an unreachable assumption that's a bit too strong. Right handle drop can indeed swap the pointer to null while the outer guard keeps the reader epic active. For instance, right? So let's first look at the read. RS uh so we have an unreachable here. So, self. Uh, oh yeah, this is a this is to check that if you tr try to enter the same read handle multiple times, then the the second one is a no up because you you don't want to count. Let's look here. Um yeah, so uh imagine you like just in one thread you call enter and then you immediately call enter again on the same read handle. The second one should not increment the counter the epic counter a second time because then it would appear to be even. So the second one needs to do nothing. So we have to we have to have this guard this enter guard to make sure that if we are entering twice um then the the second one is sort of a no-op we just give out another guard. Um but the you'll see here the read handle state that goes in the guard uh keeps track of which enter it is so that you know when to actually um escape. So you'll see oops um you'll see it keeps a a reference to the cell of the read handle so that they can keep track of the last one that's dropped and only the last one that dropped sets it back to to um to even. So you'll see this just gives you another read guard. So it it still works. It's just that that one doesn't do anything. But there's a guard here that says if the read handle is null um then we don't give out the read guard. So the question is why is this marked as unreachable? If the pointer is null, no readguard should have been issued. Ah so this is incorrect because and this relates to the thing we just fixed um which is during drop the right handle is going to set the um the the the pointer the atomic pointer to null to indicate you can't access this anymore. Um, so it's not the case that it's just during setup that it's null. It can also be null during tear down. And so then we should indeed uh return none here. Um, and so I'm guessing that's what they're going to show, right? So left, right, new, enter. Uh, in another thread you drop the writer and then you wait until it's actually dropped. And when it is the you should expect enter to return none. Um but instead it it panics. Yeah. And I think that's totally right. It should just return none. Um it would be good to have this as a test though. Um let's see if we go back here. Uh PR check out. Let's see if I can push to this one. Uh yeah. So see I've been inconsistent here about the uh capitalization. Fix that up. Uh the right dropped. Uh right handle may have been dropped. Uh, in fact, must have been dropped, right? I don't know. Must have been dropped. Um, and the epic would be active anyway. So the other readguard that we issued in the first enter could still be some right because it it's still active. It's still sort of holding up the um uh the right handle drop must have been dropped. Um so uh um so even though the first enter may have returned some all we can do now is return since the um point t has vanished. still active somewhere since our first enter must be preventing it from being claimed yet. Um uh slight clarification or More words. Doesn't matter. It's going to be squashed anyway. Can I push to this one? Can I really not head? I don't understand why I can't am I like missing something in my setup of this? Let's see. Get uh get remote add left. Right. Get fetch this one. Get push this. Yeah, but I don't understand what the It's true that the PR creator can decide if maintainers have access, but I feel like normally people check that box or at least I think it's checked by default. This is two PRs in a row where people don't have that. Um uh that's annoying. Um why why are these checks not running? Ah approve workflow to run. Um well I guess we will have to do the same thing here then which is uh check out main and then get pull and then uh make uh re-entrant enter during drop uh and then we will get reset hard for this guy and that we will get reset domain. H right this also needs to merge main. So it gets that change and then we will get reset hard to the fix and then get reset to main. So now we have this uh and now we can actually do um now we can also add this as a test in read. Where's my where are my tests here? Actually we could even do this with Loom. Um, but I don't think we need to. I think this is actually sufficiently deterministic. It is a little bit annoying actually to write tests for this. I don't know that I have been very good about having a easy to extend test suite here. Um, part of what's annoying about it is you would need to have a a data structure that implements the traits required for left, right, which I believe this one does, but it doesn't have that trait. So let's go ahead and simply try this. Um re-entrant enter during drop. So, what's the uh DQ and OP? DQ and OP. And we'll import that. Uh I don't think this needs to happen in its own thread. I think you can just drop it here. Uh and then we can assert this is none. uh value that's fine. This can be derived debug. No. Oh, this is like a test data structure I set up just for this test. And I guess now Oh, and then debug partial eek and eek. And this can also be partially eek. You happy now? Okay. Uh, minor left. I don't know why that was even there because I don't think it needed to be. Uh, read Guard doesn't implement partial leak. There's no real reason why it can't. I think maybe that's a thing we should do at the Whoa. Something isn't happy with me. Uh, why can't readguard implement partial leak? I think it should. Uh, and in fact, I think this is something we should add. Uh, but we should not add that in this PR. So maybe this is another good modification for us to make just not right now. Uh so then we'll just go with the original assert is none. This can go away. Oh, I know why it's happening in a different thread. It's happening in a different thread because this will block forever because this guard exists. That's why uh so this no um so this actually needs to be is none and then this needs to be uh what they have they had like drop writer this which is fine I don't know why they assign it to a I think this can just be that uh and now this is going to complain because I happen to use an arc here. H I'm using an RC so this isn't thread safe. Do I really want to make this whole thing not thread safe? I don't think so. I actually think I want to create a new little test suite for this. Um tests uh simple. [clears throat] Let's call it simple. Um so that means this is going to go away from there. Uh these I'm going to just undo so this stays as simple as it or it stays the way it was. um like so. And then here we're going to make value simply be an i32. It's not going to have this value registry. The value stuff is um a thing I added to be make it easier to write tests to check exactly what operations are applied. But you don't actually need those. Um so we can actually simplify this quite a lot. We just do this. So, uh, don't need the drop. Don't need the value registry. That can all go away. Um, don't need these. In fact, maybe I should just have it be counter. Fact I already have a counter type I think that we can use here. Yeah, we can just take this one. Uh have so instead of all of this stuff, let's just use the one that's already the example for left right. So, uh, we don't need all the docs here. And I think this is also a defaulted value. No, apparently not. Okay. Uh, and now we can write our little tests. Uh, which is a lot of this stuff down here can go away. We don't need all the value stuff anymore and the DQ and everything. Uh 24. Okay. And then we don't need these helpers. And we can just do this. This will be the actual test case. This should now be thread safe and we don't need the handles. [snorts] Uh, great. Yeah, re-entrance during drop. And if I now try to go back and undo this fix uh and here set this to unreachable that failed. Great. Uh and then I think actually let's move test simple to test counterrs. Uh okay some spammer in the chat. Let's get rid of that person. Goodbye. Uh, what do we wanna? Let's just do that. Sure. Report. Goodbye. Hide user. Goodbye. Um, great. So now we have an actual test that catches this. So now we can add test counter. uh and we can check out testdQ so that we don't make any changes to that uh and then here we can now commit this with uh I'll retain this and then I'll also do the um I'll also keep here the patch what have they changed that this used to be a thing you could do. Ah I need to do this I think and then patch because then I can do this uh and then I can write what was it co-authored by um and then I'll say here um it's possible for read handle will enter to observe null even after an initial initial successful enter. Uh specifically when right handle drop is running concurrently. Um so C handle right on drop is running concurrently. Um so the unreachable was a little too strong. Uh and then I will also do actually um replaces number 50 since I wanted to add a test. and modify the comments. But the PR did not allow maintainer pushes push you origin re-entrant. So, uh, I think this Well, that's not happy. Why is that not happy? That made all the tests fail. This is the the one where we skipped the called to wait on first publish. Why? [snorts] Why indeed? Let's go down and look at one of these failures. Whoa. panicked that read 59. What? Why would it fail there? This is what I mean by side effects of uh of weight. So, what else does weight do? Resizes last epics, walks through epics, loads from them, but that's all it does. So why that's fascinating. Wait, it fails earlier, right? 449 length is zero but the index is zero. Oh, I see. um weight is the only thing that resizes last epics. And so now that we no longer call wait in the first case, if you have a test where you only call publish exactly once, what you'll end up doing is you'll never resize last epics, but you will hit this loop which assumes that last epics has every index that's in epics. And so this just needs a resize. This just needs um and when when this panics that causes the other one. So if we now find uh fn weight and we take this from there we do this uh like so [snorts] that needs to be on the other branch though which is uh oh no oh no uh am I going to be allowed to push to this one? What do we think? So if I now do get diff get stash uh PR checkout 149. Okay. Get slash pop uh publish may not uh wait may not have publish. Can I push now? This one I can push too. I don't know, man. Okay. Uh, approve. This one fails just because of coverage. So, that's fine. So, this one we can merge. Confirm and merge. Great. Uh so then I will go here and say in this guy uh with uh 154. So close that one. Excellent. And now this should start to go green. Yep, this is green. Apart from the coverage, we can update this with a rebase to get sure we get the CI for the latest one. Uh, and then we can start to prep a release here. So, this will be 11.8. We didn't make any breaking changes. Uh we'll do a cargo update. Is there anything we're behind on? No, cuz we have basically no dependencies. Although, what is the latest Loom? Is it still 07? Yeah. Okay, great. Then I need to refresh in order to see the approve button, which I don't know why GitHub hasn't fixed that issue yet. This has been a feature for a while. Uh, and then once this merges, we should be good to go. And then we'll open a ah, and then I also need to undo this because otherwise I break the check with the older rust version. And now we twiddle our thumbs until all of those pass, which should be pretty fast. I really should figure out why coverage doesn't work. It might be It might be something stupid. Is there a I wonder what version of GitHub workflows coverage? Oh, why is this on the old style coverage because this has moved to seven now I think. Yeah. So that needs to move to seven. So we can actually fix that at the same time. So uh we'll do here this will move to seven with this commit. All right. Now just coverage which is fine. And so we squash and merge like so. Excellent. And now we pull to get in that change. We cargo update. We undo this. Ah, great. And now we do release 011. What did I bump it to? Did I bump it to seven or eight? Eight. toml. Yes. Great. Um 0.11.8 release 11 8 so that we can check that CI is actually happy. Let us see. and now hopefully coverage should be green too. And then we'll tag and upload. Why not use something like cargo release? H I could the the main reason I there are a couple of reasons I don't I I don't like CI having the power to publish packages. Um even though I know there's been a lot of work on making it secure, I just I don't like it. Um the other is I kind of like the ritual of it of doing a release. Um it lets me I don't know. This is something where I'm like yes, I've I've gone through the steps and agreed this should be a release. Uh cool. So go in and squash that. Boom. And then we'll go back to main get pull. We'll tag uh and then we will cargo publish and get push the tags. [snorts] We'll go back here and we'll go released in 0.11.8. Eight. This one. This one. And this one. Done. That can go away. That can go away. That can go away. And this is done. We can delete the branch. Uh, something I learned recently. You can do git push- origin head to automatically push the remote with the same branch name you're using locally. Yeah, I know. It's neat. Excellent. So, we did left right. Good job, us is only took us an hour and a half. Um, so what do we want to do now? The sponsor of today's video and the first ever sponsor of the channel is Hudson River Trading. HRT is a quantitative trading firm using the latest advancements in machine learning, high performance computing, and systems engineering to make trades in over 200 global markets. They pride themselves in having a deep engineering culture and a relentless focus on performance. HRT is primarily a C++ and Python shop, at least for now. But the kinds of things that draw people to Rust in the first place are exactly what they're facing daily. Delving deep into the lowest levels of their hardware and software stack, including instruction set optimization and compiler tuning, all whilst meeting strict correctness requirements. They're hiring in offices across the globe right now. And if you're interested in joining, check out the link in the video description to learn more. What do we want to do next? Um, let's do HDR histogram because I see there's a bunch of things on there. And then maybe we do inferno after that. So HDR histogram is this one, this one, this one, this one, and this one. And are those the only ones? Five of them. Five. Great. Uh so what is HR histogram? So for those who are not aware, HR histogram is not a data structure that was invented by me. Um it was actually originally written for Java by Jill Teen. uh and then there's been ports to a bunch of different languages and I happen to do the rust port. It is a way to keep a um a histogram. So think like uh you know a bin diagram where the x-axis is some um unit you care about like let's say the number of milliseconds it takes to process a request and the y- axis is the number of times you've observed a given uh amount of time. Right? So you can see things like oh it um yeah 100 requests took more than took 5 milliseconds uh three requests took uh 120 milliseconds and so on and you sort of bin those um and normally with a histogram it's a little annoying because you need to set your lower and upper bound and then you need to have a like you need to fix your bin size. So let's say you wanted to capture everything from requests taking 1 millisecond to a second or let's take the easy case from 1 millisecond to 1,000 milliseconds which I guess is to a second u but 1 millonds to 100,000 milliseconds then you need to choose what resolution you want to have. So um you could say you could put it into five bins. So you would say any request that takes between 0 and 200 milliseconds I'm going to count here. anything between 200 and 400 I would count here and so on. So you would kept five bins. Um and so this way you're only really keeping five integers but it also limits the resolution that you get for the histogram. So you can't tell out of the requests that end up in the 0 to 200 bucket. How many took one millisecond and how many took 200 because they're all grouped in the same way and that's frustrating. So you might want finer grained buckets but the more fine grain they are the more buckets you need. So if you wanted like one if you wanted millisecond precision then you need a thousand buckets. Um if you also wanted to capture like below a microscond maybe more seconds if you had really slow requests then now you might need 10,000 buckets or 100,000 buckets and all of that needs to be stored in memory. So it becomes pretty intensive. Um HDR histogram is a high dynamic range histogram. So instead of the buckets being linear in size like all the buckets are the same size um the buckets are roughly speaking um logarithmically spaced. So if you have um like let's say you wanted to span from you know one microscond to 10 seconds instead of having the what would that be like um 10 million bins um you would instead have one bin that's let's say 1 to 10 microsconds one that's 10 to 100 microsconds one that's 100 to a thousand microsconds one that's 1 millisecond to 10 milliseconds and so on And those buckets are not the same size, right? The one that goes from um 0 to 10 microsconds is only 10 microsconds long. The one from 10 to 100 is 90 microconds long. And so they're not evenly spaced. And the HDR histogram basically has a a an algorithm for determining that logarithmic spacing such that it's still pretty fast to look up which bin a given latency you want to record goes into. And it doesn't just have to be latencies. It can be any anything on the the x-axis. But the intent is that you now get um finer grain granularity across a wider spectrum of data um without having to store something that's linear in the space you're trying to capture in. Um so super neat data structure uh and the Rust version I think is pretty neat. Um and so let's see. So, we have one unound buffer initialization. That should be pretty easy to fix uh from two weeks ago. Uh Marshall Pierce is one of the other maintainers of that crate. Seems Marshall is on top of this one. Has a PR that fixes this. [laughter] The unsafe usage in V2 serializer was only worth maybe a 5% speed up assuming it was correct which it was not. Past me did not worry enough about unsafe. Uh let's go ahead and look and see what that fixes. Resize a lot of dummy bytes. Yeah, because resize um when you when you call resize it resizes a vector. So it allocates more capacity for it and it has to go write zeros or whatever you pass as the the value here into each of those slots and that writing might actually be the majority of the time because the resize is actually pretty fast but it leaves uninitialized memory in that space. Um, so this the unsafe function set length that just lets you set the length of the vector to be equal to something that's greater than its current length but less than or equal to its capacity and just leave the stuff in between uninitialized under the assumption that you will initialize it by writing stuff into it. Um, and so you don't necessarily want the the zeros to be written because you're already going to write some other value. you're not allowed to leave those values just uninitialized because that's undefined behavior in Rust to have values that are readable by safe code that don't have an initialized value. And so it looks like the original audit here was um yeah and code counts only writes up to total length which might be less than the max size here meaning the trailing capacity might be permanently uninitialized. Um and so therefore serializing Instagram triggers this this uh and this was seemingly found by me. Uh cool. So the fix here in this case just be like that optimization is not worth it. Let's just write the zeros. Um although it is interesting because why why are we even extending it to max size? Like why not just extend it to total len instead? And here's another unsafe forget unchecked. This is probably trying to optimize the bounce check, but the bounce check should be gone because of this assertion anyway. So, I don't think that should be a problem. And this is the same. And then forbid unsafe. Let's get rid of the annotations. Um, okay. Okay, I mean this seems fine, but I am confused about why we even do this and why this is in parentheses. Uh it all he he also said that there was a failing coverage rate limitly reached like the last release of HDR histogram was three years ago which is not because it's unmaintained it's because it's just hasn't really needed updates Um, I think I need to update the coverage settings. That's fine. But um, I'm confused why this shouldn't be total shouldn't simply be total len. I want to extend the slice all the way to max size when we're only ever writing len into it. Oh, I think I know why. It's because we don't know the total len until we've done the encoding. And the encoding needs to write into a buffer that it needs a a safe allocation into. That's why so you don't you have no way to know total len here because you compute it based on counclen and counsel len is only computable after you've given it a slice to write into. Um so that's why that's this seems fine to me. Um don't worry too much about the coverage. I think that's a repowide thing I need to fix. Proof. Cool. Squash and merge. Confirm. Okay. So, that fixes this one. So, we'll have to do a release. That's easy enough. This one. um a feature that adds single pass value at percentiles value at quantiles batch API. This is definitely this this looks like something Claude wrote, but I could be wrong. Uh quantiles. Okay. So, um there's a there's an API on on HDR histograms that lets you ask um what is the 95 95th percentile value for example. So the 95th percentile value would be um the value that 95% of your recorded samples are below. Um so for example the median is the 50th percentile. Um so 50% of your values are below the median. That's what median means. But that way you can also compute things like 95th percentile, 99th percentile, 99.9th percentile and so on. And currently the API for this is there's a uh there's a method called value at percentile where you give in a percentile and it gives you back the value there. The way that works is it basically walks all of the bins accumulating the count and then figures out where to stop based on the total count. So think of something like you know the histogram keeps track of I have a thousand values in my map or in my in my structure. Um and so it will start at the lowest value the lowest bin and it will add however many elements are in that bin to a counter. If that counter at the end of the bin is now uh more than or equal to the percentile that was the the if the count that we have so far divided by the total count is more than or equal to the percentile that was requested then you return that the bin the edge value of that bin. Otherwise you go to the next bin you include that in the count and now you check how many percentile you up to now. You keep walking up there until you've reached a point where now you've gone over the percentile they asked for and now you know that's the value at which that percentile was hit. Uh and the observation here is like what if you have multiple percentiles like you want to get the 50th, the 75th, the 90th, the 95th, the 99th each time you're starting the counting from the first bin. So rather than do that, what you have a thing where you can pass in multiple percentiles, get all the percentiles out and you can do it all in a single pass. So it's it's a it's a really good insight. um get the values at several quantiles in a single pass over the histogram. And they presumably also have a percentile. Uh yep. So the difference between a quantile and a percentile is that the um the percentiles are you pass in the value 95 and that implies 95%. Uh the quantile is you passed in um 0.95, right? Because 95 divided by 100. So so values at percentiles is just slightly more easier to understand for humans, but it's it's the same thing. Um it's a little sad that it has to be a vector, though. In fact, I wonder whether the interface to this should be an iterator. Um because currently we need to collect into this vec and in fact any caller that wants to use this data structure would need to have all of the quantiles or percentiles they wanted in something that's contiguous in memory in order to call into this. Um which isn't really necessary, right? The only thing that's necessary is that they are in sorted order. So imagine they have it for example in a B tree set or something then they would need to collect it into a ve and then pass it to this API. If we took an iterator that would be um that would probably be better and then we would just have to enforce that they are in sorted order and that's the part that's maybe annoying. Um let's see. So there's returns of vec. Uh can you return an iterator over the buckets with a cumulative quantile? Um, so in theory we could write this as a generator except that Rust doesn't have generators yet is the problem because then you could for each thing in the iterator you got in you produce an a value in the iterator but then you're basically implementing iterator um over another iterator and we we we could totally have that be the interface but then it wouldn't be a function in this way. uh if it is to be a function, it basically needs to be a generator function, which we don't have in in Rust today. Um but that would be a really neat way to do this. In the absence of that, the return value here will end up being a vector. The question is whether we should require that the input is a vector. Uh but let's first check that it's correct and then debate the interface here. So returns a back 64 with one entry per input quantile in the same order as quantiles. Input order is preserved even if quantiles is unsorted or contains duplicates. Each entries exactly what valued quantile would return for that quantile. But the counts array scanned only once regardless of how many quintiles are requested which is faster than n separate calls for n greater than one. So this is actually something we could pretty easily write a quick check for that these two are equivalent edge behavior matches valued quantile. Yeah. See so it um basically what it does it reads in all the quantiles into a vector and then it sorts that vector down here. In fact, it does this. It has multi it has it creates three different vectors which feels very unnecessary given that this um so HR histogram tries to be pretty fast. Um and doing three allocations here feels like it shouldn't be necessary. Per quantile target cumulative count. Yes. We basically turn the quantiles into counts by multiplying them by the the total count of elements we have. Uh we cap at one. Uh we round up and we ensure we count at least one element. This could just be a clamp. And then we resolve them in ascending order. So one scan is satisfies all of them. I think this these are integers. So this could just be a sort. I don't know why they're using sort by. This reads very weird. Why is this n comparison after the allocation? Oh, because it returns a vector. Okay. And then we hoist the next target into a local so the hot loop stays tight. Per crossing bookkeeping runs only when a threshold is actually reached. total to current index. So this is how many have we counted past and then it keeps track of what the next target count is. So that's the next thing in the order. We loop over the counts. Yeah. So the counts here are the the bins. track the bin. If it's above the next target, then that value is what we should return there. And then we need to write out Yeah, we need to turn that value into the the actually boundaries of that bin and then we move on to the next position in the quantiles array. Uh if we hit all of them we can return if total to less the next target. H this is written very strangely. Um I also I don't think this actually needs to be a separate vector. I think you can just compute the target count directly here when you update it. Like I totally buy that this is useful. I don't buy that this implementation is a good one. Um it's also [sighs] this is one of the things that I think is is difficult with um difficult with LLMs is like this code to me reads like it's LM code. Um and you can see it in like some things as like referring to things as rules. uh or like the m dash in the comment here is like pretty uncommon. Even I don't do that and I love m dashes. Uh and like this kind of articulation. Um, and the reason it matters here is not because I wouldn't accept something coming from an LLM, but it makes it difficult for me to figure out whether I should tell the author here what changes I would like to see or whether I should just do it myself. Um I see. And then they have Do we have um quick check in here? Oh, wait. No, that's the wrong project. Uh, it should be Oh, yeah. We ported. Did we write dare histogram histogram on stream? I guess maybe we did. That's fun. I've I've forgotten all the things we've done on stream, which is kind of funny. Uh that's that's a shame that we don't have prop tests in here because this feels like a pretty good candidate. Um batch and it's actually yeah uh batch value must return exactly what the singular value would have done for every edge unsorted plus duplicate inputs an empty histogram. So we create a histogram. We record a bunch of I guess semi- random values like is this is a the half mercen twister random number generator. Why not just use a random number generator? This is another example of something I'm like who writes this? Like what human writes this? [sighs] Um age value quantiles to the length is the same and check that for each one it's the same as what value of quantile would have given you. Same thing for percentile. Yeah. Okay. Uh test here is fine, I suppose, but I'm tempted to try to write a better version of this. Uh so let's go ahead and do PR checkout. The other thing that's missing here is actually value at quantile. Uh in fact if you are trying to Compute multiple percentiles. Prefer value at percentiles. Multiple There we go. Um, all right. In fact, let's make these nope. value at percentiles value at quantiles I can't spell Um, all right. Let's see what we can do about this. So, I think I want to change the contract here a little. Um, so we could make this return. We could take an iterator and return an iterator and say something like um we stop iterating if no longer in order or if we hit a duplicate. I don't know if that's a nicer interface. I think it is. Uh the other thing I want to check is so rust has a from fn iterator from fn which takes ah an fn mute. I think we might be able to write this as an FN mute [snorts] I think. So let's let's see what happens. So if we make this imple iterator in fact imple into iterator uh item is f64 and then we'll return something that's imple iterator item is U64 and we'll do the same for value at percentiles. And then this can now be percentiles into it map like so. And then here um returns an iterator per input quantile in the same order as quantiles uh but stops early. Um if a quantile is not strictly greater than the previous I uh provide quantiles in ascending order without duplicates. And then we'll put the same down here for uh like so. Okay. And now what we want is we can get rid of this. Uh we can say that target is going to be a closure that moves in total count. Uh you give in a Q and it gives you back Uh Q dot clamp between zero and one. The clamp allows the edge values I think right accuse NF64. Yep. So we clamp it from 0 to one, multiply it by total count, take the ceiling, uh, and then we clamp and we take that whole thing and we do max of that and one. Uh, And so this is normal uh not normalizes but clamps to zero to 1.0 uh minimum count of one. We don't need to do the sorting because we assume that from the input. Yeah, we could do the same as binary search and just say that the values might the returned stuff might just make no sense if it's not in order. That is maybe a better interface. Let's see how it feels when we write it. Um this whole thing we can go away. Uh we do need to keep track of the total to current index. We don't need to keep track of the position. Um, but we do need to keep track of uh so quantiles is going to be quantiles into map target count of Q. It's going to be targets. And then we are going to do um [sighs] mute uh counter. This is where it would be a lot easier if we had um it would be a lot easier if we had a generator functions here because we're basically going to sort of semi-handroll the generator function. But what I want is an iterator over self counts. Uh counts. Uh and I think it needs to be enumerated because I think I need the index as well actually. Why? Why? Oh yeah, because you need it for value four. Great. And then now we do uh stood itter from fn to return an iterator here. Um and it is going to move all these things in there. Um and then it's going to be mutating these and that's sort of going to be the state of the closure. Um it's going to mutate them as each time it runs. uh which is why it's going to be an FN mute, but that's fine because from FN allows it to be an FN mute. Um and so what do we want from this? It needs to return an option, which is fine. It returns none when there's no longer anything to return. Uh so then we say let um next target is target.next with a question mark. So if there's no next target, there's nothing to search for. And so we return um and then we will then loop here. This is going to continue the loop I and uh count is counts. Next. So we will keep taking from this iterate. So we have this iterator. We're trying to iterate over all the bins, right? I'm going to store that iterator here outside the closure so that it gets shared between each time the closure is invoked. And this will just continue that iterator from wherever it left off last. Um as we'll take out the i and the count. Uh this remains right. Uh that is plus equals that count. Um and then this remains. So if that is greater than or equal to the next target then uh of quantiles. Ah, so we actually need to keep um actually we can just do we can do this. So this will actually be quantiles. So next quantile is quantiles.next next. And then we don't even need that to be a function because we can just call it here and say next target is here. Next quantile, right? Um and then now here let's make this a comment. so that we actually get formatting here because this is getting annoying. Uh, and it's going to be upset that I don't return anything. So, let's just say none for now just so the errors go away. Um, why does it ah the total count? So, you see how we get a an error here. It might be hard to read. Let me run it here. uh it says hidden type for imple iterator captures lifetime that does not appear in it its bounds um and in particular I believe the problem here is this closure that we return is not allowed at least here we don't say anything about it borrowing from self but it will borrow from self right because the iterator here is borrowing um from selfc counts here and so we need to make it clear that this is actually borrowing from self um there's a A couple of ways to do that. One of them is, and this is the one that is in the newest Rust version, is to do um use. So we'll do um tick a tick a use tick a and so this is one way to do it. Um and that will be right, but it will only work with versions of Rust that support the use keyword. So, HDR histogram tries to support older versions of Rust, but this will make it no longer do that, which is a little sad. Um, [clears throat] uh, yeah, let's let's see if we want to keep that or not. The other thing is use needs when you use use, you need to explicitly mention all the type parameters that you use, which includes where's my uh, this is this file is way too long. The HDR histogram type is also generic over T. Um, and so we're going to have to say that uh this also uses T. What else is it complaining about? Uh, oh, and I also captured the iterator. Fine. This will be the iterator uh where I is this because I need to say that this also captures I and T and now it's happy. So right. So now we can finally in peace here go if we've passed the target then we can do this bit which is we get the value at that index that we just passed. Why is this a loop? This is a loop because oh this is handle duplicates correctly. So that's why this is a loop because if you have duplicates um this wants you to keep returning the same value. And so I suppose we could do the same thing but I don't I think we just want to ignore duplicates. I think we here want to say yeah. So this is where uh so we would need to keep previous quantile is none and then here we would need to say uh previous quantile is equal to some next quantile. Uh and then here we say if next quantile is sum and the previous quantile uh and the previous quantile is greater than or equal to the next quantile then that means they're not in sorted order or there are duplicates. then return none. Uh we could alternatively just peak the iterator. That is true. So we can make this a peakable iterator and then just look forward. It's effectively the same thing. Um it would mean we don't need to store this, I suppose. So we could say here peakable uh and then say if uh quantiles do peak but it's kind of annoying because you still want to return the value for this quantile. You only really want to error on the next one. Uh and so you would need to peak back which isn't a thing. So I don't think because I don't think we want to return none if the next one is invalid. I I guess we would do it um here right when you're about to return. So if quantiles peak is sum and next next quantile is less than this can just I don't know why these have to be next these can just be called that uh if next is next that is less than or equal to quantile then no it's the same problem right we still want to return sum for the one that was valid so I don't think this works I don't think it can be a peakable I think it actually needs to be this but I do want to make this just read quantile And this uh to read just target um duplicate or not in sorted order. So stop iterating. Um right. So now that we have that we need to do this bit. This is something that comes from um from the the highest value thing we already have. Um so the sorry not the highest value one the value at quantile that we already have is this rule that if the quantile is zero we use the lowest equivalent of the bin. So the low value of the bin um rather than the high value of the bin, right? Because every bin has a left edge and a right edge. Um so we do let result is um if quantile is zero then this um and then we here return some result. Uh, and if we get to the end, if we get to the end and we still have a quantile, then what do we do? I think this one just returns the value. But I'm curious to see what the um what the original one does. So if we go to value at quantile see how similar our code now looks to this one. Um but otherwise it returns zero. Why does it otherwise return zero? I mean I I guess we will mirror that then. So this will be some zero match uh value at quantile. Let's see how that does. Hidden type. Oh, this is for the percentile version. It needs to do this. It needs to look the same. So it needs to have the same here. Percentile percentiles. Oh, percentiles. And now it doesn't need to have a must use anymore. And ah I suppose this is yeah this can be this in byref. So byref is a thing you can do on iterators to say in so so normally when you do for something in and then you give it an iterator the four consumes the iterator whereas this one saying don't consume the iterator instead consume the iterator by reference um which I think is equivalent to us doing this um yeah that way we don't need it to be a while and it is also unhappy about something else. Oh, a lot of ellighted lifetimes. That is true. I should arguably fix that. Um, but these are all in pre-existing code and now it complains because wait, why does it need a compiler? Oh, cargo update. It's probably very old lock file. Let's see this. Uh, I'd write it just with while current total is less than the current total required for this quantile, then advanced quantiles and return after the loop. Duplicates would work automatically. Yeah, that's true. So, we could sort of flip this on its head. So instead of iterating through these, you do the you turn this into a while a while less than and then you only inc you increment this after realizing it's less than. Um, is that nicer? I don't know whether that's nicer. Oh, why is it unhappy? Oh, right. These test data access uh is empty won't work because these are now iterators. So, uh the singular would in input order with the same handling of corner cases. RNG. Don't I already have rand in there? I already have rand in there. So, just rand random mod this plus one, right? Like why why make this? There was even already rand there. Great. Um and now this is no longer going to be out of order. 1.5 minus 0.5 just to see what happens. Um, no, cuz minus 0.5 is going to be clamped to this one. So, that won't work. So, we're going to do this uh error in fact into iter because it's an array. odd copy. Uh and in fact now we can do collect here like so. Uh and then this now can be 0 50 99 99.99 100 and 150 um iteropied. In fact, this also can be iteropied. And then we'll do collect. It's still gonna complain down here. This uh assert this count zero. Great. Um we should just pass in quantiles. Actually, that's what we can do up here. We don't even need this. We can do this. And then we can do the same here. Uh, and this is going to be uh U64. Don't need a V. And this fails down here, too. Uh let's do dot collect there as well. And now these should pass. Let's hope so. Oh, fails because qual one at 579 oo because uh I can imagine it actually failing because the lengths aren't the same. Uh, but that's not what's failing, right? Because 1.5 is going to be turned into 1.0 and that's going to be a duplicate. So, why is it different? That is interesting indeed. Um, although we can, as someone suggested in chat, flip this one around so that it actually handles duplicates correctly. Uh, which we would do by Oh, which we would do by having this say. It would be a little ugly actually because then we would need to reread this inside the loop which makes me a little sad. Oh actually no we would not. We would simply say um I see what we do. We would do this um while we're less than the target then add the bin and do counts. Next uh and if if counts.next Next dot is none. Uh, no, that's not right either. We would say count is counts. Next else uh we would return some zero, right? That's the the thing we've landed on here, which I think is incorrect. Doesn't sound right to me, but okay. Um, then count here now is this. Ah, because we need the eye. Uh, we need to keep track of the I is going to be the annoying part. So, we're going to have to say let mute at count i is zero. Um, at count i is i. And then here we would say, so now we're basically in that if. So then we can do this. Um, and now I think duplicates should work. And now this can also change and say if the previous one is greater then let me stop. I should also only run the one I actually care about running uh which is data access. Data access. There we go. Uh it would be for the case where you want the iterators to be the same length. Give the wrong values for unsorted input. Uh in iterator on arrays is hidden by old rust editions. Yeah, I mean I forget what we've set the rust edition for in HR histogram. It's 2018, so I think this one's okay. Um, great. I think I like that a lot better. I do also want to push a cargo update actually to um to HR histogram, but we should do that separately. Okay. Uh we also need to update the docs here a little bit because it stops early if a quantile is not not greater than or equal to the previous IE provide quantiles ascending order and then we have to do the same here not greater than or equal to um and also our data access test now can have duplicates. So this can be minus 0.5 uh and 1.5 just minus 50 some duplicates. See that that still does the thing. Great. There's a lot of tidying that should happen in this repo. Maybe someone wants to be great and go through and tidy up all the all the lints we now have. Um, so if we look at this, I want to add source lib and test data access. Um, rewrite as iterator without allocations. Uh, you can't ever return the same result, right? What if two quantities should be in the same bucket? Uh I think that's fine, right? Because you would just compute the same result again because if uh if I read the next quantile and it's the same quantile as the one I just computed, then this I won't enter this at all because this was already met for the previous one. uh and the target hasn't changed because the quantile is the same. So I go here at count I has not changed. So this value is the same. So this value is the same. This value sets it to the same value. And so I returned the same result. And so it returns correctly returns the same result if the quantile is the same. So I think this correctly handles duplicates. Um, so let's do this. See if we can push. Excellent. Okay. So, back here. Um, [clears throat] took the liberty to update this directly with a zero allocation version. Um that instead just gives it to the caller to ensure that there that the values are in sorted order and simply truncates the return if they are not. Um the other thing is um HR his history does have a change log and so we need to make sure we actually add to the change log here. Um that return zero should be dead code. This um it corresponds to the case where you iterated all the buckets and haven't found the total you need yet which should be impossible because of the clamp. Yes, that is true. But do I want to make it unreachable? Uh unreachable. Um because target is clamped to total count. So by the time we so last bin must make this false, right? The la when you get to the last bin, the last bin is returned from here. That count must bring total to current index to be equal to total count. And since target is at most total count, the two must now be equal, which means this is not the case, which means you don't go in here, which means you must land here. I buy that. Oh, the compiler does not for an empty histogram. It's not true because there the total count is zero. So you don't even enter this. That's what this is for. So except in the case where the histogram is empty uh return some zero and then we can also do an assert e uh self total count is zero. There we go. Uh great. So we will no add source lib um explain sum zero case. Uh right. And then we also need to change the change log to say uh added uh HDR histogram uh value at quantiles and value at percentiles for faster lookup of multiple percentiles. Um, and this is in PR 138. Uh, and then now this is unhappy because of that. So, we'll do an update with rebase. Uh, this one we've already merged. That should be fine. This one should now run. And then we should be able to merge it. Yeah. Here's another thing that makes me think this is LMS. You wouldn't put 7.2x in here because the x depends on the number of values you look up. So this this is just an arbitrary number. Um, cool. Refresh so that I can hit approve. Um, and then I also want to reset hard. Go back to main get pull. Once we've merged this, I want to do an MR that uh bumps all of the dependencies uh in this project. At least the the cargo lock. We don't actually need to bump the major. So maybe we should do that, too. Um, and then I want to uh add batch Yeah, this stuff is all unnecessary. I can go there. Filipe reddis.com, but also ollie@codeperf.io. I'm confused. whi which which which email do they actually have? Um, also reddis.com is reddis using hr histogram because that's pretty fun if they are. Oh, really? Why does that hide the CI results? Come on. Um, cool. So, that one will leave baking in the background. Let's go look at this thing. Chunked skip scan and value at quantile supersedes 139. What is 139? Ah, this is 139. So, they opened a PR that they then decided to get rid of because the other one is faster. Okay, so we can ignore this one. It's handy one for one fewer PRs. Um, replaces the value at quantile prefix sum scan with a chunked skip scan. The old loop added one count and branched on the running total for every element. Yes. Uh this sums a fixed size chunk at a time reduction with no early exit that auto vectorizes and skips the whole chunk while its subtotal cannot reach a target. Only the single crossing chunk is walked element by element. I see. So instead of counting uh one bin at a time, you count eight bins at a time because that way you can use um vectorzed ops counter non- negative. So chunk subtotal cannot reach the target and it's no crossing element. Uh yep, that's fine. Okay. Yeah, that's that's this is definitely LLM. [laughter] It's like that's not even a question. Um, who calls it a white box par test? Comparing the chunk scan to a linear reference. Okay, that feels like maybe even more tests than we would want here, but let's take a look. Is this done yet? What's taking so long? Oh, yeah. Okay, the Mac OS build. Um, okay. Skip the test for now. I want to see the actual change. Okay. So, instead of walking the bins, we are now walking. I see the Yeah, the finish is just that's the this bit. That's fine. So we walk the chunks, we sum the chunk, and if we've passed the count, then we walk the chunk bits one by one to figure out exactly where we crossed. Otherwise, we just skip this the chunk entirely. And the tail is for if we don't evenly divide eight then we have to walk the remaining counts one by one. Uh yeah and this is unreachable. I agree. >> [laughter] >> The comment says that this point is never reached followed by unreachable with the same argument. Um yeah, I buy that that's faster. Basically you um sim over the counts. But why? This is not a useful test. This is just copy paste of the old code to see that the new code is the same. But we already have tests the test that that that that this function does the right thing. So this test file I just don't want. It's it's not necessary. The existing test already covered that this is correct. So we will kill those. That's easy enough. Is this done yet? 64. I'm basically I'm comfortable merging this. These are these are not going to fail if the others failed. We've made no um we have introduced use actually. So the 164 build might fail. Uh maybe even will fail. I think it will fail because I don't think we had use yet in 164. So then the question becomes, are we willing to bump the minimum supported rust version here? Mhm. There you go. Wait, it also Yeah. Yeah. Okay. Because it fails to even parse the file. [sighs] Okay. Uh cargo MSR. Nope. cargo MSRV find minimum of 168.0 and I want to use check. Just going to do a binary search here to find basically where am I allowed to start using use 182 uh rust 1.82 when did rust 182 came out? It came out in 2024. I am okay with bumping that. Uh workflow checks MSRV1 84. It is like it does make me a little sad because there's no real reason to do this because that there is a workaround we could do here that is technically backwards compatible. At the same time, I do think it's also uh like a two year almost twoyear-old Rust release I'm okay with bumping to if you want to have the latest version of this because I don't want to. This is a case where I would have to make the code a lot um uglier in order to not break old Rust versions. The alternative is there's a um um um there's a crate called Rust version um that lets you have yeah this um where you can say uh conditional compilation as of Rust version something but it adds a build step to your thing which I don't think we currently have uh build yeah we don't currently have a build r uh so this will be use of use. Wait, that's it should be 182. Uh, so someone in chat suggested instead of using use, you can just use plus a. Uh, the problem with plus a is it's wrong. Um, it it does sometimes or usually work, but it is is actually incorrect. Um, especially this has to do with variance. Um you're basically making the tick a here invariant. Um there's a the actual thing is there's a you introduce a trait called captures and you do this. Uh but that's not I don't want to I don't want to do it. This is an example of something where I don't think you should bump the minimum supported R version for arbitrary things. I think you should only bump it if it requires that you hold back the ergonomics of working on the development of the crate significantly. And I think this is an example of that. Um, MSRV for use of use. Uh note that this also bumps MSRD to 1.84 from no. 182 from 1.64. Uh since uh we make use of the use syntax uh which also means I need to change the change log. Um, that 1381.82 This doesn't that doesn't seem right. What what is supposedly the last commit here? 85. Oh no, that is that is right. Okay, get push. Cool. So now that's just going to go run again if we make it to run again. Um, and then we can go check out this one. and then get pull checkout one 40 chunk scan. So here I want to get RM source tests value at quantile scan and sort tests test value at quantile scan. Uh remove unnecessary remove uh I don't even know what to call these tests. uh unwarranted parody tests. And in fact, let's um we already fine. Uh we already have tests that cover value at quantiles quantile and friends. So no need to also check that the implementation equal to a copy pasted version of the old one. New paranoid non-regression tussia uh unsloppy maybe. Um and then the other thing I want to do is uh well obviously I want to change the change log um to say here that um uh HDR histogram value at quantile and value at percentile. Now use uh vectorized operations so that they should be significantly faster on larger histograms. This was number 150. Sorry, number 140. change entry. Uh and then for the actual implementation uh I think I also want at quantile chunks at a time can be done with Cindy. Cindy until we detect a chunk that passes the target in which case we walk. Uh only then do we walk the chunk bin by bin? Why is Clippy unhappy? Consider using as chunks instead. Why? returns arrays instead of slices. Sure as chunks. So chunks is now uh it's a slice of arrays of length eight followed by a slice jar. So this is chunks and tail and then this and this is not going to be remainder anymore. This is going to be tailor. Sure. Um if scan chunk does not perfectly divide uh the number of bins handle the leftovers some bins in chunks. There we go. And then this just to make it more readable. Comment can go away. Great. Simplify [sighs] and Clippy and better comments. Great. Now can I merge this see? I passed. Oh, a junk was stabilized in 188. Yeah, I would have to, wouldn't I? That's okay. Um, squash and merge with this description and this squash merge. This is the other reason actually why I suspect at this point I think I know that it was written by an LLM is that a human would not have caught they would have run clippy and went oh I should use as chunks changed it to ask chunks and then not have noticed that that would break CI um for the rust version in question. Uh so let's do here uh see main get pull this get merge main uh edit the change log because it's going to be um 1.88 S chunks. Uh and at this point I think we should also set uh rest version is 1.88. Yeah, someone said that in in chat too because uh um because at this point we do actually require that version of Rust in order to be compiled. Previously wasn't set because it was the reason we only supported 164 was actually not because of code in this crate but code in dependencies. Uh and so then the rust version uh version um and so if we had set rust version equals 164 we're actually lying because it supported I think down to 157 or something uh but you wouldn't be able to build because all the dependencies required that you upgraded but it wasn't our crate really that was the problem. Now it is and so now I think it is appropriate to set ROS version um uh bump and set MSRV for as chunks. Uh and then I think actually we need to update this uh this release to 1.88 because it uses a few newer Rust features like use and uh chunks. uh iterate on MSRV note and change log push. Okay, so that one's merged. This one's merged. This one is now going to run and then be merged. And then there's no one nothing else for this. Let's check if we have any other poll requests that are relevant. Uh bump clap. Don't think I care about that. Unrecord. No. Uh, bunch of old issues that we're not going to deal with right now. Um, cool. Uh, and so then we'll go to main, we'll pull and we'll do a cargo update. See, this is the other thing is like now the other thing that bothers me about setting rust version in cargo toml is when you run cargo update, uh the default behavior in cargo is to update dependencies only to ones that support 188. But I'm actually willing to have my cargo lock have a newer version of Rust in there, have newer dependencies, and then bump the MSRV in the um in the CI job to be a newer version of Rust for the purposes of CI. Um, but we'll um because I'd rather know that I'm running on an older version of a dependency. Um, but we will find out what that actually leads to. Okay, this uh the 188 passed. So now I'm pretty happy to just merge this one. Um, chunk skip scan invalued quantile goes away rather than the atom branch on every count element empty. I'm going to do that in here so that I get my little nice line wrapping and such uh on every every counts element bin uh sum a fixed size chunk of bins at a time. Um so that the compiler can doization. This is often significantly faster. Uh 150 observes no 140 observes. Um bins one at a time. Uh like so um also bumps MSRV to 1.88 88 in order to use as chunks. Much shorter commit message, but still means the same thing. I'm guessing this is someone watching the stream. Uh, this comment is somewhat inaccurate. We just iterate in chunks and that allows Yeah. Okay. Fine, fine, fine, fine, fine, fine, fine, fine, fine, fine, fine, fine. Uh, fine. Keeping me honest. What Cindy uh V chunks that they can be so the such that the operation can be auto vectorized by the compiler until we detect a tank. There we go. Yeah, chat is uh getting in my PR comments. It's good. It's good. I don't mind it. Uh techn technically correct is the best kind resolve. confirm and merge. Uh great. So now we have this uh we'll go back to main. I'll get pull um bump CI. So this is oh get remote uh add CI. So I have a repository where I keep my CI config. Uh, and so for basically every repo I have, I will merge that CI repo into here so that when I make modifications to my CI config, uh, it brings in like all of the various um, uh, all of the various dependencies, all of the various changes from that those CI changes so that I can do this merge CI. Um, and now I'm bringing in all the latest improvements I've made to my CI setup into this repo. Uh, and I, you know, it's it's git, so I actually get to do a merge. I get merge conflicts and all that good stuff. Um, so I can remove this because it's not used. I can remove that. Uh, let's look at check what has changed here. Um, I do indeed want my version. I want my version. Um sim confused about this part. What did I change here? I do want this instead of that. Ah, hack is moved is why. So there hack was up here. Now Simber is up here. Uh, simmer doesn't need to run on nightly, but it was also running a dock pass with uh config docs RS and that one I already have here. So, these bits can just go away. And then this should indeed change this way. uh and then should change to be this. So we add that uh and then we want to check scheduled what's changed there. Yep, I do indeed want that version. So, I've started um using the actual hashes for all my GitHub actions rather than just the version number because that way I can actually control when they get updated. If you specify something like v6 here, in theory, your CI job can change under you, right? Someone could push to that repo under that branch or that tag. Uh and now your job immediately picks up their change. Whereas, if it's pinned to commit, it will only happen when depends. Um which I'm more comfortable with at least. Uh, so I'm guessing a bunch of them are here. Those conflicts. Easy to fix. Easy to fix. Um, don't need OpenSSL, so that can go away. I don't keep the comment. This stays the same. Uh, why did I remove all features from this? Ah, for bench private. Yes. So indeed I do not want all features. So that can go away but I do want to keep the recorded the rest version. Now the one thing that's important when you do this is that when you merge uh bump CI repository files. Uh when you merge these, you have to not squash. You have to do an actual merge because otherwise you break the git history as the merges won't work in the future. You'll see this brings in a bunch of other contributors that are people who contributed to the CI. Um which like is maybe weird but is what I want to happen. Um so here specifically this needs to be a merge commit. Um but we'll wait for CI to pass there hopefully and then afterwards we'll do one where we um bumble the dependencies and then we'll do the actual release. This is why like open source maintenance takes a while, right? Like this is we're only on repository 2 for the day and we started 3 hours ago. There's like we're averaging about an hour and 45 minutes per repository. Uh, and it's not like there were lots of issues in either of them, right? Chat is the real LLM. It's true. Uh, great. So, we should now hopefully get this in soon. Uh, we'll go back to main and pull. Um, and we'll fork. We don't need to fork off that yet. Um, you also don't technically need to merge it with the merge commit because this there already is a merge commit. Uh, and so you can instead just do get reset- hard uh bump ci. So now you'll see this is at that same merge. Um, and so I can just get push to main, but that's only because I'm the maintainer, right? So instead of merging this way, I can just merge by pushing the merge commit I already have in this MR. Uh, and the only reason I open it as a PR is to actually see CI run and see that it's all green. Um, but I don't actually need to merge it through through this. uh that the other advantage of this is that I can now fork off of this and start working on some other branch and it will actually be forked off of the correct commit that will actually land um bump depths update and actually now we can um uh we could also upgrade the addition actually because we moved to we moved to one 188 and I forget what version that was. Rust 2024 was released as 185. So given we're now already at 188, we can move to Rust um uh Rust 2024 straight away. Um in fact, but I want to bump the dependencies first, I think. So you'll see there'll be some dependencies here where we don't actually get the latest version because we're behind on the majors. Um but luckily I think uh we can fix oops verbose. So serialization here this should be fine to bump the nom bump is probably annoying. Uh and the rand bump I think is probably okay. Rand is only a dev dependency but it will be slight. Actually, these updates are something I found LM really useful for is like I just tell it, hey, update to the latest versions of these, but go change check the change log for what you actually need to change, and they tend to do it really well. This is like a super annoying mechanical job. Uh, where LMS are actually super useful. In fact, maybe let's try that. Uh, it might be a good thing to stick in here. Uh, I do want this to land first, though. Just make sure I haven't broken anything. Uh, I will land the bumping the dependency separately, though. Uh, maybe not. Maybe I will just make that be one PR that bumps the cargo lock and then updates the the dependencies and then um updates the addition all in a row. I think we will do that. We should also fix the um the clippy lints. Why is coverage broken? Ah, coverage is broken because I haven't updated. Let's see conf Okay. So, workflows test coverage. This is still using the wrong version there. So let's um cd domini get pull github work with test coverage. So this no where's my coverage job? This we need to bump to um code cub actions. Surprised actually the dependabot has not picked up on this yet. Um so diff upgrade code cub to 0 0 push that to rest cic and then I will actually um this one's easy to redo back to main actually I'll go back to bump ci uh I will reset that Um, this is where JJ would be really nice because now in order to merge, I want my history to be nice. I don't want a second merge commit because that's kind of stupid. So, I will actually reset hard to origin main. Uh, then I will fetch CI. Then I will merge CI. Then I will do this and this. And then we have to fix up these again. And this one stays. That one goes away. This one stays. Right. What had to change here? Nothing. This can just go away. This does need to be this. And it needs to be quoted, but otherwise that's the correct one. Uh, and what's the last one? Scheduled. That one's easy. commit get push force release. So that should fix coverage. But what happened to the uh what I wanted to see was did this one ultimately pass? It passed everything except coverage. Uh, and I guess minimal versions hasn't finished yet. And minimal versions might not pass. That's fine. Uh, so now I'm okay. Um, I'm okay just uh merging that pumpci. So now this should have that one. Yep. Get push. Okay. So now we can go back to bump depths. Um we can do a cargo update. Uh we can do a get commit. Um update all the things. Uh origin bump deps and then we'll start a new PR. This one's now marked as merge because I pushed the merge commit directly to main. Um, that's just incorrect. Why is that? What's What's happening? Try again. There we go. So, we'll create the pull request so that we can start to get CI running. Um, and then I want uh cargo check. Does cargo check have a fix? I didn't think so. Um, a lot of the stuff is like in things that is a little bit annoying. Um but okay. So now we just need to wait for LSP to be happy here. So this says use max. That's easy. This says use max. Will cargo fix actually do this for me? All right. I think it will. We'll do all of them. Uh yeah. So this masks the lifetime from there. That this is fine. That's fine. That's fine. That's fine. Uh same thing. These are all just mass lifetimes. This changed in like a very long ago rust version. Um what does cargo check say now? What does cargo clippy say now? Uh cargo clippy fix. Fine. Cargo fix. Cargo clippy fix. Nope. Cargo diff. Uh read exact. Yeah, that's fine. That's fine. That's fine. Okay. Preix. Uh, now I want Oh, it's mad at interval log mod because of the indentation. Interval log mod here. I am also mad about that. Thanks, Clippy. That's not pedantic. That's just correct. Uh, great lib. This should be max. This should be max. That feels like something it could have changed for me. Should be max. and d serializer 167 check that all the tests still pass. They should. Um there's something that failed to to give me warnings here in the tests too. Uh so this is actually more more clippy fixes. Uh and then I want cargo fix uh all targets. Why was it not happy about that? Test data access 453. What do you mean? It's never Oh, this this indeed feels like there's a thing missing. That should assert something. I don't know what it should assert, but should assert 4228. Sure. Okay. What else do we have? Uh, example CLI Max. I see these fields are never read because the debug is not considered enough. But the debug might be printed by main. Presumably, yeah, the debug is the only reason we need these fields. So, I can do of unit type. Consider changing the field to be of unit type to suppress this warning while preserving the field numbering or remove the field. No, wait. Unless Yeah, cuz serialize returns one of those that gets bubbled to main which unwraps them. So they they need to exist. Great. Uh, lints in tests. Okay. And then we want cargo d-addition. Why did it only move? Oh, it only moves one edition at a time. That's fine. Okay. I think we need to we need to figure out what to do about these benches 2021 edition. cargo clippy. Uh I like using expect rather than allow for lints. Yeah, that's true. Uh expect uh isn't dead. The reason I thought allow here because is because technically it shouldn't even be surface a dead code because we do actually print out the debug representation here. So in theory Russ could realize that is true but then I guess we should remove the the lint. uh isn't dead, it's printed. Um yeah. Um, and now the question is what to do about all these benchmarks because there's a bunch of benchmarks in in HR histogram that uses um the bench support that existed in older versions of Rust. Uh, this has been like completely this is a nightly only thing that I think they're even planning to remove. Um, [clears throat] I think that's fine. I think we're just going to keep that. Um, so now if we look at cargo update, there's no updates except there are major updates. Um, and so here's what I now want. I now want to bump all of those to the latest version. Uh, the some some of them I think will be pretty easy like B 64. Some of them like the nom upgrade is probably actually going to be pretty annoying. Um, what I'll do is I'll push this first so that that makes it out the way. Why is coverage broken? Uh, oh, I need to add a token to this. That's fine. I'll do that separately. Don't want to do that on stream because I don't want to leak my tokens. Um, so here's what I want to do here. I'm going to go ahead and say, "Hey, Claude, um, bump the major version of each uh, dependency of this rust crate listed in cargo update-verbose. um for each upgrade uh for each dependency. Uh do the upgrade separately separately for each dependency. Um, and make sure you read the change log for the uh create in question to see exactly what changed and what you need to do uh to match the newer major version. Um, make a commit for each upgrade using commit writer before proceeding to the next create to upgrade next dependency and then in theory that can run in the background. Oh, I need to stop uh cargo format all because the fixed things don't know to format. Try again. [sighs] Okay, so that can now be happy run in the background. Um, and then once it's done, we add to the change log and then we can actually issue a new release of HR hisgram. Cool. So now this can just keep ticking on in the background. Uh, yes, that's fine. Uh, and we will in the meantime go to our notifications to see what there is next. Um, so Rust IMAP is a kind of interesting one because uh, Rust IMAP is a crate that I have effectively abandoned. Um, the the reason for this is basically I don't use it myself anymore. I know that it needs some pretty major rearchitecting. Like it's not async at the moment and I think it basically needs to be. Um, and I simply don't have the time to put into it to make it what it needs to be. Uh, and so there's a thread, the maintenance suggestions thread, where it's basically a thread about who can take over ownership for this. I've tried to pass along ownership of it for a while unsuccessfully. Uh, and in the meantime, there's like a bunch of different uh, upgrades and stuff that are just pending. Like someone needs to pick them up. Uh, and arguably I think the way to move forward with the Rust IMPAP is just for someone to write a new implementation and then just sort of import it into the same repo or we rename the repo and then they take over the crate, issue a new ma major um major version that is effectively just a different crate. Um, but actually doing that handover is is non-trivial. So what I'll do is I'll open each of the Rust IMAP things and then many of them we will just close out. Um, and then they'll reply to the to the maintenance thread. So like this one for example, I'm just going to mark as done. Uh, this is command injection. Yeah, I mean there's a there's a bunch of things like this. This is a very very long Okay. >> So, I guess this is Yeah. So, someone has basically posted a uh an example of something that allows you to inject commands into um uh into IMAP operations uh because they're not sufficiently sanitized, which or validated, which I think is true. I mean, rust map was never really written under the assumption that you would have untrusted inputs into the commands, but it is true that if you do, it doesn't really guard you against it very well. Um, there are fixes to this, but again, the the whole point of saying I someone else needs to take over this crate is that I will not be maintaining it. Um, so I'll mark that as done. I'll mark this as done. Uh this is this is IMAP memory exhaustion. Yeah, I mean I don't it should be fixed but I don't care. Um this is the problem of abandoning a crate, right? It's like there are thing there are things there are many other things too where I'm like I know this needs to be fixed but I cannot be the one to fix them. Let's just check if this uh yes fine. Um right. So is actually one of the people who have contributed to this crate in the past um and who's one of the people who who might uh take over maintenance of it. They basically built their own rust crate um that that implements IMAP and does async and they use it in a um in an application and everything. So it's a pretty good candidate. Um, a lot of the discussion that's happened on this issue is basically they've described the way they've implemented that crate. Um, and I think what I wrote somewhere, let me see if I can find um, uh, yeah, for a while the delay was like they weren't ready to open source that crate for whatever reason. And then there's a discussion about like the way it was implemented and everything. And I think what I wrote somewhere was um up here uh >> yeah and what I've written here too right is that so I've already added two people as maintainers of the crate on GitHub um but they don't have publishing privileges yet so instead um instead that's sort of the last thing that I hold back right is like you can't publish under this name until we have decided this the path forward but you can go like now contributing to the GitHub and everything because I am not maintaining it. Um yeah and then part of the debate here is like how should this be implemented? So we had some exchanges over email too where I was basically like I don't think the way they've implemented their replacement crate is the way I would have reimplemented it. But I think I also wrote like I it that should not be the blocker, right? Like think of this as I've looked at the way you want to do it. That's not how I would have done it, but I am also not the maintainer. So I should not be a blocker for if you want to do it this way. Um like I would much rather someone take over the IMAP crate name and bring it back to life than that we now have two crates, one of which is abandoned but has the the the sort of standard name. Um, and so I think that's basically where we've landed here. Let's see. So I finally released IO IMAP. It sits on top of the IMAP types and IMAP codec. Exposes three APIs. IO free co- routines, a stood client, uh, and a full stood client. Um, yeah. So the sonio approach is the one that I'm I'm skeptical of. Um, but they've found it to be good. It's almost in production in a CLI and a TUI a mail synchronizer mailbox watcher. Uh it's more complex and verbose but usually you won't face it because it's hidden behind the client. It is true like the what sense IO does is basically you implement the whole state machine of the protocol and all the parsing and stuff but you don't actually implement any of the IO. So you don't implement interaction with um TCP or with TLS like all of that stuff you leave to the callers and then those are the like basically think of it as implementers of read and write are sort of passed down the whole stack. Good job. You can leave. Sorry, a cat needed to leave. Um, so you pass the the read and write down through the stack and then the the sort of the state machine and the uh the protocol implementations like the parsers and and stuff uh are all like you can think of it as generic over the IO type. Um, and so then you have a client that then sort of instantiates the co- routines or these these parsers and and and um protocol implementations with concrete implementations of IO like like TLS or like TCP. Um, [clears throat] and so yeah, I guess they're basically saying it's sad that it would be split and that was never my intent. My intent was that the if if you have a way you want to maintain it going forward, then please maintain it that way. Like I don't want to be in the way of doing that. It was just giving sort of my um uh my opinions on that design. It was not intended to be blocking. Uh so let's make that clear. Uh, I want to get back to you that the design is working out. uh certainly a good um indicator might be uh a decent path forwards. Um as I wrote above, please don't let my preferences around implementation um in the way in the way of Let me find the crates here. I'm perfectly happy uh for you to basically delete um all the code in this repository and replace with your implementation under the create name of IMAP. Uh, as long as there's a new major version, there's a new major version. Uh, and ideally a mig a migration guide. Um uh I'll also then hand over um privileges on Chris.io for the IMAP name. Uh once you feel like uh you're wait to do that switch over cuz then a bunch of uh current users of IMAP will come knocking. Nope. Uh uh what overhead does that incur if you swap out IO types and APIs with different memory models? It's not with different memory models. it's that you um you basically make the entire implementation generic over the IO or or have like they're co- routines. So basically they they operate over they they're monads if you will over the IO mon they're not quite monads but kind of monads. Um so they um you instantiate them with an IO and then they do their IO through that. Um and so the the overhead is more that you have to construct the state machine entirely without the the IO parts which tends to mean more buffering um more allocations more keeping things temporarily because um if if you have the the IO things like hardcoded in your code paths like you have a TCP thing um when you want to write something out you just write it directly to TCP. uh here you kind of need to store the things that you will write to the IO when you have one. Um so that like when you when the when drive is called on the state machine then you write out the things because only then do you have the IO and so that there is more of that sort of um uh the sort of intermediate steps that you end up having to implement and the API also as a result ends up more complicated and the implementation ends up more complicated. Now, as as um Soywood points out here as well uh correctly, the API for end users doesn't necessarily get more complicated because you can you can implement a a crate that wraps the Sans.io crate um such that it has an an API that just uses TCP and TLS and then under the hood you have to manage all that complexity, but the outer interface can be a relatively standard IMAP interface like a synchronous or asynchronous one. Um, and so it's more that it introduces sort of cognitive and performance overhead. It's not the fact that it's it's a bad design, right? Because it does buy you something which is this ability to be to abstract over the IO mechanism such that it can be not just TCP or TLS, but also things like async or not async. You basically take the whole driving of the IO part away from the implementation of the protocols and the uh and the the data formats. Um, cool. Replied. Nothing more to do on IMAP. Uh, let's now see how Claude is doing. Oh, no. I normally auto mode is better at this. Oh, that's cuz I don't have sandbox on. Sandbox with autoallow. Thank you. And I will keep let that run in the background. So, this is going to keep updating all the things. It's still unhappy about coverage. Um I guess I let me see if I'm able to fix coverage without uh HDR In his Instagram rust uh without leaking the token. That is the question actually. I wonder is that token already there? Uh secrets dependabot. Yeah. So that's missing. So uh [clears throat] and this is still branch is still named master. No branch is named main. main. Yeah. See, I want it to not show that token. Uh, which I can do by doing this so you can't see it. Closing this to new repository secret. Uh, and then also doing this in a different window so you can't see it. Sorry folks. Regenerate this token. So the old one is no longer valid. And then putting that into this uh to here and into where else does it need to go? It needs to go into actions. repository secrets and here. And so now if I go back here, you'll see there is now a code token. And so in theory, currently stealing my credentials. Yeah, I know, right? Uh hopefully I prevented that from being the case. And so now I think if I try to rerun this one. Oh, I'll have to wait for I'll have to wait for this guy to finish. Why is that something makes this one really slow? which makes me think that there's some dependency where there's like a performance problem in one of the dependencies that we happen to hit in this test. And so it's not that we don't work, it's that if you have the minimal versions of things, uh then this particular work um flow is really slow. Uh which I think is okay. But I can't kick off the coverage job again until this job either succeeds or fails. So I guess I will kill it. Cancel. And then once that's killed, I can start the other one. All right. How's this doing? Uh, yeah, that's fine. Cloud reading from NOM. Yeah. So, this is what I was expecting, right? for basic uh yes for B 64 uh it's really just a simple version bump because we're not using very much of the B 64's APIs um the the interface there that we use is is very very limited uh isn't the dependabot secret just from PRs from dependabot not from normal PRs don't you need it as an action secret yes so I added it for both um you're just trying to get me to show you the secrets uh here I'll show you but I will first kick off this one. Rerun failed jobs. Rerun. So, if I go back to here now, uh, and go to secrets, you'll see there's a dependabot secret right there, but there's also an action secret right there. So now maybe coverage will now be happy. We'll find out. uh and then we will see how this is progressing. Rand is the biggest surface. It is used a lot but most of them should be pretty simple I think. Yeah, it's mostly renames I think in Rand I run claw directly on the prod server. [laughter] No. Um I it really depends on the task for this kind of stuff. I've found the the sandbox is okay. Um but you know it the the risk goes up every day, right? Because the chance of poisoning attacks goes up and the sophistication of the poisoning attacks go up. Um but in general the sandboxing u seems to be pretty okay. The problem is the autoallow, right? It's like if you autoallow things to escape the sandbox, well then the sandbox isn't doing that much. Um but at the same time without the autoallow it can be extremely painful um to um to not have to like approve things all the time. Um but for like bigger jobs and stuff I will run it like basically in a VM or in a container. Uh ah I didn't press show secret so that's why the test is slow. I think you're right. I think you're totally right. Uh, were you ever curious about feeding the IMAP crate to Claude? Make this async don't make mistake. Make code reviewers cheer. I have thought about it. The problem is I think it's a I think it's such a substantial rework that you would basically just throw away the existing um you would throw away the entire existing codebase. I don't think you would reuse it. Uh, I don't know if I'm being unnecessarily paranoid, but I don't trust the AI companies at all. I don't trust their built-in security stuff to even be sound. Yes and no. Right. So, the the sandboxing is basically just um uh using something like bubble wrap. Um, so it's basically enforcing what it's creating a little jail for the process effectively for what things it's allowed to read and write and what um what network requests it's allowed to do and that sort of stuff. So the sandboxing is actually pretty decent. Um the the problem is the autoallow, right? So if the process has the ability to say run this command outside of the sandbox without prompting, that kind of ruins things. Um, although interestingly enough, I'm not sure I follow why uh why it still keeps prompting me because this should be caught by the sandbox already. I think it's maybe because I enabled the sandbox after I kicked off the work. Uh, yeah, that's fine. Because in general, I found that the this kind of like approving lots of small like prompts that are just read only and there's no reason to really prompt like this is this is the kind of thing that makes you not pay attention as well to the things that you're approving and therefore makes your security worse. I would much rather it prompt me less often such that I actually look at the things that it prompts me with in more detail. Um, I can press switch tap to switch to auto, but I want it to be in plan mode because I want it to be planning. Um, hence the the problem. What? But what model am I? Oh, I know the problem. Why am I using sonnet 5? That's not because I've also found sonnet to be much worse at issuing commands that aren't silly. Uh, [sighs] I'm going to actually stop this one. I'm going to switch the model to fable. Uh, plan mode sandbox. Yes. And then try that again because the prompting makes it so much slower and Fable is a lot better at planning. I mean, Opus is too. Uh but sonnet is not very good at that. Uh don't ask him for Gary update. Uh don't ask for index crates io. So I found fable to be better at using the tools such that I don't have to hit yes as much for like randomly generated bash scripts. Um I've not tried um Saul. No, I I'm only really using cloud. I'm not um I haven't worked much with uh with codecs for example at all. There we go. See, this is a much more reasonable way to approach it. Rather than try to pull them out of the cargo registry that's locally on my computer, just like fetch them directly from the GitHub page of each repository and then boom. And then you see the auto mode classifier is a lot happier about let letting things just sort of pass through. Um, cool. Now we got code coverage. report is 55 commits behind head on main. That doesn't sound right. Oh, is the CI things. I'm confused about why it claims we're behind. Um, I suppose actually I could merge this and then have the major changes in separate commits, but I don't think I care. I think I want the major ones. Um, but yeah, we'll we'll see. It'll be interesting actually to see how annoying these changes end up being cuz my experience in general has been that um catching up with major bumps and crates like this is a lot of like look up the change log, look up exactly what changed, look at what I have, change the syntax slightly here and there and here and there and here and there to match up and then run cargo check and cargo test and then find whether it works and then do it again and do it again and do it again. and it's like extremely annoying manual labor that can just like can usually just happen in the background while I'm doing the other things. Um, and so this has at least traditionally been things that I basically uh it just saves me time. Um, let's see what we have here. Uh, yep yep those are indeed the updates. So print each dependency separately. It is very important though to review the plans because often they make very little sense. Um, adapting the code to each new major per its change log with one commit per upgrade. All other dependencies are already at the latest majors. Yep. Uh, B 64 should be trivial because yeah, basically nothing. NOM 78 is substantial because oh combinators right forgot about this. So nom which is the parser generator crate moved from a sort of functional syntax where you have these combinators functions and now those have gone away and instead there's a trait called parser so that you move from this this is the old style to this which is the new style. Um and so you basically need to like walk anywhere you use any kind of non- combinator and make this change. Um, and you also the the input mechanisms change and so it's like still a mechanical change but a really annoying one. Um, so yeah, mechanical change is expected. Shouldn't be too bad I think except for the fact that it's mechanical but not trivial with said. Um, and then rand is mostly a bunch of renames. Uh, small RNG goes away. That's fine. Yeah, this seems totally fine. Num, uh, we only really use num in um, serialization. Uh, so that should be fine. Yep. In order increased risk. So, we'll do B 64 then random nom. That seems reasonable. Uh, we'll use cargo ad. Not technically necessary, but it's something I put in my cargo toml because it tends to get confused about which version to put in there because it doesn't always pick the latest one. Um, as a cargo ad just does that more reasonably. Adapt the code per the change log letting the compiler drives the exact renames. Um, verify that everything works. Yes, except I don't want with all features. So, that is something I need to tell it. Um, all features won't work since uh that enables nightly only things. Um, but let's do this and see if that works. Uh you will need to use cargo plus nightly test uh in order for all features to be usable. Yeah, mechanical but not reg friendly is perfect for agents. It's like agent food. Um, seems like a good use of a clawed workflow. Yeah, I think this kind of like upgrade the majors is something that you could pretty easily turn into a um, uh, you could pretty easily turn into a pretty standardized workflow that you just have a well- definfined prompt for. Um, what terminal and desktop environment are you using? This is uh what am I using now? It's a good question. Oh, let me just approve this first. Uh let me check what it made down here. Yes. And use auto mode. Um so right now I'm using Sway um for the actual uh desktop environment and and window management. Um and then I have like DUN for as a launcher. Uh and I'm decently happy with that. Um terminal is elacrity. I'm running T-Max in there with the fish shell. So nothing has changed in my setup there. Um, I'm generally fairly negative on AI, not because the tech is inherently bad, though there is some issues, but the main problem is most people just turn off their brain while they're using it. I think that's very true. At the same time, I also think one of the best uses for like the the agentic coding agents is for work where if you did it manually, you'd kind of turn your brain off anyway, like this kind of um of like major major dependency bumps where if I did this manually, it would be a similar like very mechanical affair where I it would just become sort of roach repetition and I wouldn't really be looking that carefully at every change anyway. So, I'd actually rather have this run in the background, me do something else, like talking to you about other things, um, and then review the result to see that, okay, now I can look at the result and actually focus on them and see whether this looks reasonable. Um, makes me feel better to see John using AI. There's a decent level of guilt behind it for me sometimes. You know, I I I don't feel guilty at all about using AI for for things where I feel like it's an appropriate use, right? Like again this kind of mechanical change for example or there's some amount of like uh you know turning PDFs of specs into initial implementations like there are bunch of those kinds of things where they can be extremely useful. Uh I've also found them very useful for debugging. Um I think they're really good at finding bugs not so good at fixing them but really good at debugging. Um, I also found them decently good at like exploring new code bases, uh, and helping me ask questions about how things fit together. Um, that they can really speed you up there or from gathering like domain context that I don't have. Like if I'm reading a the source code for a crate where, you know, I know Rust pretty well, but I don't necessarily know the domain of the crate very well. And so I can use its general knowledge to like augment my understanding of the Rust code that I'm reading. Um, so there's a bunch of those kinds of things where I think it works really well and I don't think there's any reason really to sort of feel shame about using it in that uh in that way. Um, code review too, it can work pretty well, although it usually is not sufficient, but it is a a useful step. It can also be a really useful first step before you use humans time to review. Like one of the things that that I've seen especially at work actually is it's really useful if people go through the sort of automated code review or AI based code review before I go review it because otherwise I will go through and leave like 50 comments which takes a lot of time. If they instead do like 45 of those the AI can find and point out to them and they can fix them before it reaches sort of my desk. That's way better for me. it's a way better use of my time to read through and not have to comment on all the nits. Um, and so for that kind of thing also, I think it's it's super useful. Um, it's when you start using it for like the sort of turn your brain off for something where your brain really needed to be on um where it gets frustrating or where it becomes sort of loweffort PR contributions like some of the ones we looked at earlier today, right, where um the the observation was correct, right, that this could be made better. Um, the feature request, if you will, was a was a good one. Um, the implementation was not wrong, but it's just not what I would have wanted and not I feel like if you think through it a little bit, that's not really the way to expose it. That's not really the way to implement it. That's certainly not the way to test it for for some of the the like parody tests that was in there. Um, and the the PR description write up was like not super useful, very verbose. And for those kinds of things that that is where I would have some amount of like I guess shame is the right word where I'm like do one step better than that. Like think about if you're the receiving end of this was was this the best you could provide for me to then review? And I think the answer to that is no. Um that doesn't mean that it would be better if you did nothing because I'm very glad they send those contributions and send those um identifications. That's great, right? Like now HR histogram is better as a result, but the way you do it matters, right? It it matters that you um didn't go that extra step. Maybe it would actually been better if you just like filed an issue that said, "Hey, can you make this change?" Uh rather than the autogenerated change that I basically had to rewrite manually anyway. Um, [snorts] but I don't think there should be a sort of unanimous like um AI is bad for everything and and never always something you should be ashamed of. I I I simply don't think it's true. Um, [clears throat] do you think junior devs are cooked because of AI? Like is there hope? I I I don't think they're cooked at all. I think they're, as I mentioned, there's a bunch of things they're good at. There's also a bunch of things they're really bad at. Um, and where I think you need to have like um an understanding of like how these things fit together about what you're trying to accomplish, what the you end user's goal is, what your future goal is for this codebase. Um, I also find that LLMs make a bunch of mistakes where they try to replicate patterns even when the patterns are not necessarily appropriate. Um, there's also a bunch of places where there are no existing patterns to necessarily map well onto your codebase. And so I I just don't think the engineering job is going away. I do think that there are many engineering tasks that would be reduced. I also think there are some engineering tasks that will go away. And so if your job is only those tasks, then yes, it will be a problem. Like you know, if you run, I don't know, a business that um builds custom WordPress templates, I think you're in for a bad time. the company might be in for a good time, but you as an individual developer doing that development, I think AI basically does that job. Um, and so it's not clear that that is a um that that you're in in like a safe work environment. Uh that that your position is safe. But for a lot of the like back-end software engineering like distributed systems, concurrency, um safety critical systems, financial systems, like I I just don't think that it replaces the engineers. Um and then people just accept the result as it's perfect unless it's instantly errors. Uh it's really a problem with computers being math machines that rarely make mistakes to normal people. I do think that people have a sort of instinct for if it's done by a computer, it's probably right. That no longer holds in the face of AI. I think that's less of a problem for engineers. Maybe more so for engineers that are newer to engineering, but certainly for experienced engineers, I don't think they tend to believe what computers say just because they say it because we're so used to computers making mistakes if we're programmers, right? Um, the guilt for me is the ethics. Licensed open source code is basically ignored by all of the AI companies and we all know they're just using scraped open source code. So, so I I'm actually less concerned about this certainly for for for programming because the reason I put my code out in public is not because I want to claim ownership of it. And if other people learn from it, that only makes me happy, right? Uh, and if the LLMs are trained on it, well, okay, that just means that I now get a tool that's better suited for how I think code should be written because my code is represented in what it does. U, so I have much less of a concern there. Um, yeah, they're also really bad at designing abstractions and removing code. It's true. Um, I hate AI discourse. Just the worst opinions flying around. I mean, that is the other problem. I think as a society, I don't think we've figured out how to even talk about AI because there are very strong opinions on both sides of the debate. Uh, and there's not a it doesn't feel like the whole there's a whole lot of room for nuance. Uh, and I I think that's really unfortunate because I think the nuance is where the reality lies. Um, okay. Uh, let's see. So, it's done. All three major bumps are done. Each in its own commit. Uh, cargo update dry run verbose now reports nothing held back. B64 only used the engine API which is unchanged. Rand uh did a bunch of renames. And the now fallible uh now fallible uniform new that we call expect on that's fine because rand is a dev dependency. We only use it in tests anyway. The removed smallg cargo feature was dropped in rand variant. I collapse the nine hand unrolled uniform new entries into stood array from fn rather than repeating expect nine times preserving the historical quirk that the nine byte range excludes U64 max. What we we have to look at that looks weird. Um upgrade to nom 8 pars rate migration turn out smaller than the change like suggest only true combinators need to need the new form leaf one still return callable closures replace the deprecated is digit with asky digit and we got rid of minimal exagert explicit basic for nom depth nom feature entries which are removed to keep the implicit feature consistent Uh yes, I don't think we care about maintaining that but let's go look. So let's first start in cargo toml. So in cargo toml uh I do actually want these to just enable the depths. Uh but I don't think that matters here because we don't it's not the case that we want a feature by the same name. Uh although we can we can opt into that because there's there's no real downside to it. Okay. uh flight nom, but I want nom to be I want these to be fully specified versions. Um because otherwise we might accidentally uh uh update rep4. What? Fine. Lock basis for uh so 02d1 uh num should be 8 0 0. It's just in case we accidentally are now relying on something that came in like 801 nom for example then our minimal versions check would would break. Uh and rand is 0102. should arguably do the same for for the other dependencies too. Um, okay. So add cargo toml uh make minimal versions explicit. Um, right. Let's go ahead and go back and show dashp. No, get log P and see what we got. So, let's go back far enough to that's rand. Where's the first one? Here. B 64. Uh, we only use the engine API which is unchanged. So, we move to 022. That's all. Cool. That's easy. uh for Rand. Uh yeah, small RNG now becomes make RNG. It's fine. Didn't really matter. There was small RNG in the first place anyway. Um, but we need to look at this rand varant stuff. Okay, so this we fixed. Uh, this is changed to make RNG. That's fine. That can stay that way. Make RNG. Make RNG. Make RNG. Make RNG. Uh, uniform new is now fallible. Um, but we just use expect, which is fine because this is in tests anyway. So that all seems like a fairly mechanical change. More make uh gen range to random range that seems fine. Ah, so here we had several hardcoded ranges. that I've now changed into I see bite length is I + 1. Yeah. So that's going to be this guy. So smallest is that one. Largest is that one. Night bite ranges upper bound is UC4 max. So plus one would overflow. Give the historical behavior of excluding U64 itself rather than switching to new inclusive. Ah you see here for nine we don't add plus one. That's fine. But it feels like we should it does feel like um new inclusive is better because new include that that's really what this is trying to do, right? Is uh this is trying to make it inclusive of the largest number. And it just happens that for UC4 max that you don't plus one. So I think we should just move this to um new inclusive and that way this a lot of this comment can go away. Uh range pix uniform from 0 to 9. Okay that seems fine. Okay. Uh, these all seem pretty reasonable. Great. Okay. And then the last change is the nom change. Dependency create dependency tree shrinks. That's nice. We now do the mechanical change to parse. uh and asky is digit. Technically this one can be simplified slightly but that's fine. Um so let's then do uh in a new commit uh move to uniform [clears throat] new inclusive for the nine invite arrays. Um and then I think we'll want cargo format. We'll want a cargo clicky pass. I think some things there didn't look quite right to me. Um, but now we got to just have that happen in the background rather than that would actually have been a pretty annoying thing that like would have taken an hour in the background. Yeah, exactly. So here, um, this this now simplified to just be new inclusive, which is what the old thing was trying to capture. Um chat is going on about AI, but they're going on with each other. Uh okay. So show dashp include U64 max in random number var. Great. Now let's just see that that still passes. It does. Uh let's check that there's no cargo update to apply. There shouldn't be. Uh car we can remove session.mmd. We can cargo format all. Uh we can cargo clippy all targets. It's probably not happy about all targets. Uh can we make it fix those? Let's make it fix those. Why did it not fix the max value things? Like why is max value to max not an autofix sometimes? set is all you need. Uh ah there's another one. I 64 max value and I64 min value. What else do we got? Okay. You I suppose I could have just matched anything here, but uh you min max. I see a U16 up there too. Yeah, of course. Uh, source reser interval log tests line 261. Thank you, Clippy. And then it is unhappy about some other things. Tests testators I agree I don't like that at all. Okay. Get those away from me. Oh, nightly night. I can't spell nightly uh test serialization 58 and interval log 326. Great. format all bunch of clippy things. Okay. Uh let's run the test suite just to make sure nothing silly happened. Excellent. Commit um cargoy all targets get push. Um, this is the other thing I think is important is that when you use AR for some of these things, like leave Claude as a co-author, make it very clear that that part of the change was made by an LLM because that way it's also clear to someone looking through, I did these ones, the AI did these ones. I think it actually matters. Uh, cool. So, once this is done, I think we're in a pretty good place. I think then we can finally do the HR histogram release. Uh, and then I think we'll probably end the stream there. So, we only we only actually got through two crates. Well, three if you count IMAP. I wanted to also get through um inferno, but I don't think there's too much there. Uh, let's let's let's look real quick. Maybe we'll get that out of the way, too. It's only three for Inferno, I think. Why are there a bunch more now for HR histogram? Oh no. Oh no. Right. Uh I forgot I sorted oldest and newest. And so now there's a bunch of newest comments on the ones that we just landed. I guess we should tidy those up at the end too. So uh the release here, that's fine. Uh this is already addressed. Uh ah so now because HDR histogram hadn't had a commit for like three years the pendot wasn't running and so therefore it wasn't proposing updates but now it is proposing updates right so it's trying to bump nom which we've already done it's trying to bump a couple of GitHub actions um which I'm fine with um so we can actually land these that'll be fine Um, the nom one is going to go away though the moment we land this PR. Um, I don't want to land these until that lands because I'd rather them be um rebased on top. Uh, and then we have I'm assuming these are comments from you all. Uh, here the quantile changes but the previous quantile stays the same. Yeah, that's that's a good point. Um, it is true that technically we could start returning sum after returning none. I don't think it matters as as you point out as well because the contract for iterators is once you call none uh there should be no assumption that anything that comes after that is valid. Um, I guess we can um we could fuse the iterator. Alternatively, we could just override um actually it's also kind of reasonable, right, that if you if you have something that's invalid, we return none. If the next thing is then valid again, I'm okay with us continuing to to return the same thing. Uh as you say [clears throat] since the contract for iterator uh beyond on uh that you don't cause UB which we don't in fact arguably it is more useful that the iterator returns to a usable state um uh after it returns none for a an out of order uh quantile This is also interesting because this is the kind of thing that I think an LLM review of this PR might have caught. Uh, okay. So, we can mark that as done. Took me long enough. Uh, more updates coming in. Uh, in 142 + 140 and what were the other ones? 138 and 140. Great. Um, yeah. Then we'll look at Inferno in a second. I don't want to context switch to it quite yet. Ah, so there is one more Inferno one that needs to go over here. Oh, it's unhappy with something. See is unhappy. Oh, it thinks we needed to bump to a new major version. Uh feature has been removed. The feature B 64 nom flate and crossbeam channel. That's fine. Um these these it is true that those are technically gone because so in in older um rust version I can close this. In older Rust versions um when you didn't have this any optional dependency like here flake 2 for example is optional any optional dependency automatically became a feature that consumers could in theory enable. Um it's not just these it's also any optional dependency. Once we made this change then now technically um those features those sort of implied features go away and so someone could in their cargo tumml have a dependency on our crate where they write features equals fleet 2. It is possible in which case we will now be breaking them by issuing this new release. Uh but if they do I don't think this is actually part of the contract this the the sort of implied features here. So I'm okay with this particular breakage. Um, okay. And then patch these shouldn't m minimal versions is probably just going to keep hanging. Yeah, it does make me slightly upset um that minimal versions now just hangs because that's going to be annoying for CI going forward too. uh test sync no drop. So that is using sync histogram which does use crossbeam channel. So I wonder if the way to fix this is actually um Yeah. So, it's probably using a pretty old cross beam channel. So, cross beam channel. So, if I do cargo nightly update minimal versions, then what version of cross beam channel do I get? 050. So interesting. I wonder that supports Russ version 160. So I think we just bump cross beam channel to 0516 and see if that fixes the CI here. Um because it's specifically in the the sync tests which use cross beam channels. So I'm going to try that. I'm going to do cross beam channel 05. We do 15, right? Like uh because 16 was just released. Oh, 14, 13, and 12 were yanked. [snorts] 0 is not yanked, but it kind of feels like it should. It's probably I don't know why 51 would be yanked. Um 515 is over a year ago. So, what changed in 516? I'm inclined to say 515 so that we don't force people to update to the latest because some people like to have an update policy where they only update to things that have were released at least 3 weeks ago or something for supply chain security reasons. Um but let's go ahead and see what in crossbeam channel changed support for rust analyzer autocomp completion and make never constant. So those don't matter to us. So let's go ahead and try 0515 cargo update get diff uh try to fix minimal versions. We'll see if that makes things more happy. chat is having [laughter] an extended discussion of AI and how it relates to Clippy uh LSP improvements inside the macro do nice in general. Uh I was melding at that with Tokyo. Yeah, it's true. LSP improvements for macros are very very useful. is more it doesn't matter to us for this change because first of all we don't even use that macro from the channel stuff in this crate. Uh and so I don't really want to mandate that people upgrade to at least that version of crossbeam channel just because they wanted to update their um HDR histogram. Like technically this could be a lower version. It's just this is just to make u minimal versions happy. So in fact if I pull that up here I'm curious to see whether that uh does the right thing. And then in fact we can actually do the release in this PR2. I think maybe that's what I'll do. So we'll do um we'll go look at cargo toml and we'll say this is going to be 760. Uh so I'm going to issue a a new minor release not a patch because we're bumping the Rust edition, the minimum supported Rust version and the major of a bunch of dependencies. So this feels like it warrants a minor here. Um and actually and we're adding um a new external feature which is the uh value value at quantiles and value at percentiles. Um, so now if I go to the change log, I also need to do this. This is going to be 0. No, this is going to be 7.6.0 released 2026 uh 0718. Nothing was removed. Yeah. So there's it's 3 years since the last release, but the crate is still alive. Actually, 75. So we're actually missing links for 753 and 754. So this should be 3 4 6 0. So this should be 2 to 3. This should be 3 to 4. This should be 4 to 6. And this should be six to head. Uh why not major if the minimum supported Rust version increased? So um I don't consider upgrading bumping the minimum supported Rust version as a major change to a crate. Um, I think that is that puts too much strain on the consumers of the crate because for most consumers there's nothing they need to do. All they have to do is like run cargo update and things will just work. If I make this a major change, it means that no one will get these updates u including like the vectorzation updates that speed up a value at percentile unless they change to 8 point something which is just unnecessary. Um I I I don't generally consider um bumping the MS MSRV to to be be major enough to warrant a breaking change. Um cool. So get commit a release as Z 7.6.0. Minimal versions passed. I was right. It wasn't crosspin channel. Excellent. Um, and then here actually did it say wanted major or did it say it wanted minor? Yeah, major. Okay, that's fine. Um, so now we will do edit um release 7.6.0 with all of the updates. Um, so 6.0 material with the various things already listed in the change log plus uh cargo update. uh 2024 edition uh base 64 nom and rand updated updated to latest major uh rust and clippy lints fixed. Uh what else did we do? and coverage fixed. Technically, that was fixed elsewhere, but it's fine. Uh, if if this fails, I'll be very upset. It really should not. This releases Okay, so now we just did that one and then we can squash this. I don't actually care about maintaining all the individual commits here. I think uh maybe I do because we do have these in here that are kind of handy to keep separate if we ever need to do um regression testing. Uh yeah, I think I'm going to make this a merge commit. [snorts] I like keeping history. Come on, Mac Runner. I believe in you. You can do it. Do you believe in the Mac OS runners? It's been running for six minutes. Come on. Even the Windows runner is finished. Windows runner finished in one minute. Mac OS, what are you doing? What are you doing? You're just compiling the last crate. What does that even mean? There we go. All right. Come on. Come on. Be green. Be green. Be green. Yes. That looks promising. Ha. Okay. Green. It's green except for simmer which is fine. So now we merge this one. No. A commit message is wrong. The commit message is wrong. That's fine. That's just the merge message. It does have the That's fine. It's fine. It's fine. It's fine. But the the OCD in me is is uh very upset. Okay. Uh so then we go back here. We run this script, which I really should open source somewhere. Uh and then we do uh GC main. No. Uh get tag-dv7.6. I lied. I need to first pull on main then tag uh then cargo publish and get push tags. Yes, that's fine. What? I didn't. [sighs] Okay, let's go ahead and uh undo that one. Um, get reset hard to 3733. No. Um, I want to see where we forked off. Yeah, we forked off at Bumpsy. Okay. Reset hard bumpi push. Force release. Undo the merge. Undo the merge. Undo. Okay. Yes. Yes. I know. I have said this is not allowed. And yet here I am being unwilling. Allow force push specifically John because John can be trusted. Use pass key. Push it. What? Yeah. Bypassed. Great. Now we go back here. What do you mean? No one should be allowed to force push. Whoever needed to do that. There we go. It's safe again. I was worried there for a second. Get push. Uh back to my pull request. This one which is now uh going to be uh bump deps this one new pull request. This one is now going to be called this because I don't think there's a way to reopen this one now. So, it's going to be this one. It's going to have this as the description. And then it's going to mulligan on this one. You saw nothing. Uh, and there's no reason to believe this would fail now, right? We can just merge this. We can just merge this. We can just merge this. It's fine. And I will choose not to add a bunch of details to this. No, I'll add it. Fine. [sighs] Fine. Confirm. It'll just be the sever check. Yeah, it's fine. So, as I was saying, now that we've merged this PR with no problems whatsoever, we can go back to main. Uh, we can delete the erroneous tag I made. Um, and now we can we can tag and then we can cargo publish and get push tags. And now let's see how this does. Oh, this is cued because I've probably been running too many CI jobs. But see now dependabot is starting to close its uh close its PRs because we've already done the updates. [laughter] Well, this is the other maintainer of HDR histogram. Uh release of this last commit to this project was 3 years ago. So, not sure I'd say I have plenty of time for it, but luckily it also seems to be in a pretty stable state. Um, I'm regularly going in and gardening it either like collecting um until it makes sense to uh also doing this on stream. So, say hi >> [snorts] >> Great. So, this one's now closed. We can ignore it. Uh, this one's now merged. So, now we can merge these guys. Um, although I am going to tell the pond to rebase all of them. Aha, now they're running. Great. It's okay, John. You can swear we've grown. Uh, yeah. So, Simmer is unhappy, but that's okay. The only reason I haven't hit uh enter on this yet is just I want to I want to see the CIB green on main before I push the tag and actually upload. Um although if if Mac OS is the only one left then I'll I'll leave that to to be so dependent rebasing. This should now all turn green at least in theory. Come on. And then on these we'll also put uh released in 7.6.0. We won't post it yet because technically it's not released yet, is it? But it shall it shall be it shall be. Oh. Um, do you have your own skills in claude? I do have some claude skills. Like I have one for writing commit messages for example that I'm decently happy with. Um, I have one for technical writing. Um, I also have some that I use for uh, so if you look at AVDL for example, which we also built on stream. Um, you'll see that there's workflow prompts. Um, this is some of these are pretty good for change log for writing change logs and stuff. um and also for iterating on this particular codebase in such a way that it keeps getting better. Um so some of this was actually in the the stream we did on AVDL. I talked through how we arrived at this particular uh prompt loop. Um so I have some that are project specific and then some that are are um more broadly used like that are basically in my standard config. If you look at uh configs, don't know how many of these I've actually checked in of the skill. I don't think I've checked in any of the skills yet because I still need to do some tuning of them before they'll be useful to other people than me. But it's uh it's on my list. Um okay. I just want like one of the actual Linux test suites to complete. Don't know. Well, how is minimal versions of Windows finished before Linux has? What is this? I just want just one of you. Come on. I wonder if it's the linker. I wonder if like linking the test suite for this just happens to be really slow. I want to hit the button. Then dependabot is now happily rebasing. Uh the other thing that's nice is dependabot remember how we changed the CI so that it's now tagging commits instead of versions. Dependabot understands that format. So you'll see it it will update the version in the comment of the tag but then also actually update the commit to the thing that points to that tag. Um so that gets preserved even though dependabot updates them. Oh yeah, now we have green. We have green. Okay, now I'm fine hitting enter on this. So now this should upload. Yeah, setting up mold would probably sp speed up um CI quite a bit. It's true. At least for some cr for others it doesn't really matter, but it would also make the CI more complicated. And there are a bunch of projects for where that's not doesn't really matter. Um, okay. So, this comment can be posted. This comment can be posted. This comment can be posted. And these I can close. This I can close. This I can close. Uh, and now I suppose technically it's these where I'd be I'd be shocked if any of these actually failed CI. Um, but we will let them run through. So, we'll put those in the background. Uh, and then I wasn't going to do Inferno, but I guess we'll we'll look at Inferno real quick. Um, so first, so Inferno is the port of um the port of Flame Graph to Rust. It's being used in cargo flame graph for example to render those like flame graph looking SVG files for uh performance information. Um, one of these is adding documentation to the crate. Um the crate is not really built to be super usable for like programmatic access a little bit but it mostly has a lot of command line tools. Um so it has like actual binaries to replace the flame graph pearl binaries but it does have a slight library interface and this is trying to add um like module level documentation to the the the Rust API part of the crate. Uh, this one I've actually already replied to, I saw, which is, um, it is good to add documentation to this, but I actually want to do a pretty substantial refactor of the API to this crate. Um, so this one is an MR that I I posted back in March. That's I finally had a chance to like sit down and go over this crate and make its API what I think it should be. Um, and there's been an bunch of open issues for like, oh, can you do something like this? And finally, this is now made. Uh, and I think this will actually become the 1.0 release of Inferno, but I'm basically waiting from feedback from the people who said they wanted these features on, hey, is this what you were after? And I haven't heard back from any of them yet. Um, but you know, when that eventually comes in, then this is the way to go. Uh, and this one does uh it doesn't add too much in terms of documentation, although it does add some. Um, and so my hope is once this lands, then it'll be more useful to add actual documentation on top of um that new API rather than try to document the stuff that's already there. Uh, so this one is just a there's no no action here. So I'll mark that as done. Uh, this is uh I'm profiling programs on unfortunately Windows and using Vtune. It seems ample XCL is being deprecated. Now instructor to use vtune for the CLI. Ah [sighs] so uh let me dig this up. So this is why did I do that? Uh the stream cd in minor inferno. I really need to tidy up my uh dev directories a little. Minor inferno. Minor inferno. um get poll. So I have the latest. Yeah. So I don't think there's actually anything in here apart from documentation. Ah our support for Vune. So um this tool takes the output from various different debugging tools like Perf uh and then turns them into that flame graph visualization but you basically need to implement a a parser for each output format. So Perf produces things in one format. Vtune produces things in another format. Um Drace from Mac OS produces things in the third format. Java produces things in the fourth format. And so there's a bunch of different format supports. And so the format support for Vtune has been the format of this uh Ample XCL tool that comes with Vune. But apparently that's no longer the tool to use. And so if it depends on whether this is just Vune now ships Vune is like the perfect equivalent on Windows I guess kind of um if V2 now ships with the CLI. The question becomes whether that CLI produces call graphs in the same format as the old tool and just use this CLI instead or whether it also comes with its own format because that would be a little bit painful. Um uh yeah so let me go. Thanks for the report. Do you know or can you check um uh whether the whether Vune produces the same output format as uh what's it called? Ample amp ample XCL. I'm sure there's a good reason for that name. um or whether it also has its new output format it uses. If it's the former then yeah we should just update all the docs but if it's the latter also need to implement a whole new format given that I don't have YouTube myself. uh would probably be a lot quicker for you to just do a simple local check. Um this is also something you end up doing a lot in open source maintenance is like relying on other people because like I don't have a Windows setup that I could easily use for this kind of development task. Uh I have one for gaming but not one for this kind of stuff. And so rather than me trying to like spin up a Windows VM and then try to install Vune and then get the old tool installed and the new tools installed and then compare them, it's a lot easier to just be like reliant on the community that needs that particular support. Um, and sometimes that will also be the reason why something stagnates out like an issue or PR is like I don't I'm not in a position where I can test this or document this. Well, I need input from the people who supplied it or the people who will use it, but until I get that, I'm sort of stuck. Uh, and then you you end up people asking like, has this made any progress? And the answer is no, because the people who need it haven't told me the things I need. Uh, and so it does really become this sort of community of maintenance uh more so than like one person being the one who is sort of the the sole proprietor of the um uh of the crate. Um so hopefully this is the simple change but I guess we will find out. Um this has been closed. Ah yes. So so someone made this huge PR back in February where they made lots and lots and lots of changes to um to Inferno to try to make some documentation changes, some feature adding and some performance improvements. Uh and I basically told them I looked through some of the uh the changes and um I basically told them look you have to split this into multiple PRs. I cannot review this on its own because it's like if you look at it you know it uh you can't tell cuz it's behind my picture but it adds 314 lines removes 264 but the changes are like all over the place. Some of them are reasonable some of them are a little weird. Some of them completely change the the structure of some of the formats. Um, and so I'm like the many of these may be good ideas. I can't review them in this form. So could you please split them into multiple PRs? This is also not uncommon in open source maintenance that you basically push back not necessarily on the change but the mechanism whereby the change was delivered right so it might be it wasn't documented or it wasn't tested or can you change the code around so that the diff is minimal or in this case I can't review it in this form. I need to review it in more contained changes where I can you maybe say no to some parts of the change but yes to others. Um and so this is the kind of stuff where I basically told them I'm happy to look at these but you need to split them up. And then they split them into this. This is sort of the first change uh where I left a bunch of comments and then it seems like 5 days ago they closed the PR. I'm not quite sure why because I think this was a somewhat reasonable change. Um, in fact, most of the most of the bits in here that I comment on were more like things to tidy up, not really a push back against the change as a whole. Um, and I don't know why they closed it without any uh context. Uh, just noticed you closed this any particular reason. seemed like it was decently promising and we'll see sometimes people just vanish uh or they get tired of open source or they did it as part of work and their work situation has changed. All sorts of reasons why these things happen. Um sadly not not uncommon. Um, right. Some of these now can go away. So, for example, this one I can mark as done. This one I can mark as done. And then these we're still working on. And this can go away. And this can go away. Uh, and then we have the last inferno change will probably be the last change of the day. Uh, avoid integer overflow when computing percentage for large sample counts. Ah, I think I remember this bug report. So way back when Yeah, I think this was like someone ran a recording for a very long time, so they ended up with huge sample counts. Something that seemed entirely implausible. Uh, but then they ended up because we do some like multiplications to to scale things up to make the SVG look right. And the multiplications basically made them overflow um when turned into into integers because we do the sample count times 100. And if that overflows a U64, you get an overflow, you lose all the precision or or it crashes. Um, and so I basically told them, look, something looks suspicious, but also we could try to tweak this computation so this problem wouldn't occur. uh like for example turning into an F64 first before multiplying by 100. Um but this was back in 2020 and I guess here now someone is fixing it. Um when a stack has a very large sample count 100 times samples overflows uh performing a flo floating point over uh avoids the overflow it's unchanged from normal magnitude inputs. Uh yeah and I think I actually I already reviewed this change because it it's very straightforward and I just asked you add it to the change log. Um, and they did that last week and then three hours ago they pinged me. Uh, okay. Let's uh let's go in and take a look. Yeah. So, it just moves that to be an F64 multiplication. Same for that. And then adds a regression test and adds a fix to the change log. So, this looks great. So let's get that in uh approve workflow to run. Then do we need the individual commits here? No, I don't think so. Uh and so what are we at here? If we look at the log, ah we're just one version like the last release was not that long ago. So the the only thing in the change log is going to be this fix which seems fine, easy enough for us to land. Uh once CI is happy. Um so we will get that in there. This will just become uh 128. I I'll wait with that actually until this guy comes through. Oh, okay. is we got we got we did get four projects I guess in the end right we got um left right HDR histogram IMAP if you count it and then inferno which was not very much and this is also the nature of these things like sometimes there'll be a bunch of issues open but they're all actually pretty quick to reply to sometimes there'll be like two things but they take an afternoon to do anything about like left right because it's there's so much stuff in the details right Um for large sample counts that can go away. It just merges them anyway. Confirm squash and merge. Excellent. Get pull uh uh head cargo. So I want release of 127. Sorry, I want release a 128. Uh 128 cargo update, cargo lock. That's great. Cool. Uh and change log needs to change because this oops what did I do here? Uh so this becomes 7 to 8. This becomes eight and forward. Uh and This goes here. 128 2026 07 18 0 128. Actually, wait. There we go. That's what I meant to do. And also, there's an extra space, right? No, there's not. I lied. Uh, get commit and cargo update. Are we behind on anything? We are not. Okay, great. Uh so we'll do get push u origin head over here release 128 just to see that CI is still happy. How are these guys doing? They're still running. That's still running. That's still running. Let's go ahead and squash and merge that. And I'm also happy to squash and merge this one. Can this still beh update? Very very frustrating. So now they both have to be rebased again. So dependabot has to go do that. So we can wait on this guy. Um and this too we can uh do this hard to release 0128. Um and then we will do cargo publish and get push tags and get push. Uh once I do this so many versions. So once this is done, then we're all good and then it shall be the end. Oh, we're so close. Yeah, Windows latest is going to be fine, I think. Don't think there's really any Windows specific code in this crate at all. I don't think so. I mean, there's there's like implementation of Windows specific parsers, but they're compiled on every platform. Do I need to tell the penobot to No, it's already rebasing. Great. It's already rebasing here. Come on. Come on. And here we don't need to hit the merge button because we're going to just push the commit directly from uh from main. Excellent. And then we can go here and say released in 0128. Hooray. Done. Excellent. Delete branch. How's this now doing? Uh we are pretty close now. That can be marked as done. And now it's just these two dependabot things which these I can handle on my own. You don't need to watch me wait for CI for dependabot bumps. Uh any last things on either of these? Let's see. So, if I swap this back to newest to oldest, just those two. The rest is just random other stuff. Look at us go team. We did it. Now I got [snorts] to go actually have dinner and water. But we what we did it for about 5 hours and we got through four projects, maybe three, maybe two depending on how you count. And this is the world of open source maintenance and why sometimes it takes a while before uh people actually comment on um on the thing that you uh the the thing that you left a PR for or commented on 3 weeks ago and you still haven't gotten a reply. If you have enough projects to maintain, 3 weeks could easily go by before you have a chance to look at it again. It's not because anyone hates you. Um with that I will sign off for today. Thank you for watching and uh I'll see you next time.