Minokawa Community Call - 2026/08/26

Hey, Kent. How's it going? Okay. So, >> I don't know if anyone else is coming. We shall see. It's one minute and >> we have someone. >> Hey, Yuri. How's it going? >> Hi guys. Thanks. I'm fine. How about you? >> Not bad. We're doing a hackathon. So, we're we're actually becoming users of our own system. I know it's >> quite an experience. >> Yeah, >> it's really exciting responsibility to host such kind of meetings. We also host uh committee meeting for hyperledger every Tuesday. >> All right, let me share my screen. In discord, I asked if it's possible to allocate some time uh to discuss uh compact improvement proposal that we submitted and I can share a couple of details. I prepared a tiny deck with explanation. >> Give me one second. Desktop 2. That is a desktop I want to share and I want to see this. Okay. Uh you said you want to discuss maps. Um all right. Um I have from our conversations the interaction or relationship between private estate and private events. Um, we might want to have Enigo in here, but I know that he's not going to come today. Um, so I can postpone this to the next week. >> Or did you want to discuss it, Kent? >> No, not without other people present. And um, also I don't know that there's any I'm thinking Kevin Right now I don't know that there's any point in discussing Ari's proposal without Kevin also but um the meeting is recorded and uh you can share this meeting together with no. >> Yeah, if it's just a presentation that's fine. I just don't think we're going to make you know we're not going to get all the questions asked that we might need to get asked. Um, but you know, Jonathan and Bryson, I you know, others on the call can can ask questions. I'm just hoping that we can get Kevin in here, too. >> Yep. >> I think he's got um more of an opinion on this. Um >> All right. Um what myth was that you wanted to discuss? Oh, can I share my desktop? >> Sure. >> Okay. >> I'll do this. You should see it right now. >> Yep. >> Oh, that's what I was talking about. Maybe you see it. >> This is the uh proposal 730 about adding Rust code generation to the compact compiler. We already did this prototype and we created a couple of uh tiny projects based on this prototype but uh problem is slightly bigger than uh it should be because there are around uh 55,000 of lines of code and I fully understand how it's tricky to figure out how to apply such a big uh contribution. Uh the first of all uh we are following the contribution guideline and uh uh we submitted this proposal where we described the purpose the way how the uh rust uh codebase is going to be generated out of the smart contract. a thin layer of the midnight uh compact runtime that should be used it as well to uh wire up the generated codebase with um midnight runtime and uh other aspects. I don't want to go through all these comments and descriptions but uh I believe that it's a first step in the long process. No rush with all this stuff because we have a deck that explains uh why it might be useful not only for uh our projects that we started to do but also for a midnight uh community and all midnight ecosystem. So first of all it's absolutely clear that as a midnight core is already in rust. Um you know that midnight CK repository that is public midnight uh ledger and uh majority of back end services are all implemented in Rust and the main uh ledger stizationization primitives also Rust codebase and the uh web assemblies and layer for TypeScript that compact generates uh also partially derived from this uh Rust codebase. And uh when we played with uh identity layer, we found interesting problem. Uh we have uh we use Rust application for prototyping and we generated the uh code out of the smart contract. Then uh there are packages from midnight to create with web assemblies to run this codebase and uh this code return it back to our Rust application uh prototype. And in order to run all this stuff uh uh we use it uh web view internal with uh web assemblies embedded into this and with a bridge uh just to uh call the smart contract generated uh as a typescript application. At the same time it was really annoying that uh we know the code base of midnight is in rust. The prover is in rust and majority of backend services in rust and all this layer is just additional um interops serialization um part to debug stuff and uh we just thought why can we just uh generate code in rust and use it directly with uh all midnight libraries because majority of types are already in midnight ledger and midnight scale. Um it's a a new time right now. A year ago I was not able to use AI agents with strict specifications requirement quality gates gates to pick up these tasks. But in a couple of iteration uh we also explored that there are other guys who are interested in different back ends not only TypeScript. So this is a quote from uh previous meips that people tried to submit in order to support our languages. And at the end we came to the um compact compiler that can use uh minus minus target rust or minus minus target type script in order to emit different uh types of codebase based on the smart contract. either TypeScript or Rust. And the way how it goes is um yeah there is a TypeScript intermediate uh representation that we have in compact language. There is a default branch like print types language and we just added print rust branch that emits the same uh rust code but specific to the rust type safety and uh other rust uh language aspects and uh after this compilation we iterated tons of times in order to reach the identical contract state byes in circuit witnesses and uh um other serialized uh values. Um it's just example of generated code in uh JavaScript and in Rust for uh example counter. We picked up all smart contracts and generated TypeScript and Rust codebase and tested it uh with simulator and with standalone environment in order to ensure that it works as uh expected. I will share this deck later with you. Just share with me uh your emails. uh and the biggest uh um problem is there are 55,000 of uh code uh new code in the compiler and I believe that it's a headache for any mainten maintener to figure out how to do how to deal with this codebase but the good thing is that around uh 12,000 of code is test for provide parity verifications between different TypeScript and the Rust sweep contracts. And uh usually you need to uh carefully review around eight around one key of lines for compiler that emits the Rust code. And uh uh we you need to review the uh runtime that created to wire up the code base generated with a ledger. And what >> that breakdown of code that you had back there, the part that was um >> other passes, the the the part of the um code that was the bottom one, the non-rust passes, those 862 lines, is that effectively fixes for improvements to other parts of the compiler? the on the way the uh we found a couple of issues that we fix it and improve it but uh uh usually this flag is um can be treated as experimental. So the compiler works with typescript in the same way and it's possible to rebase it on um any revision. We have ledger 8 and ledger 9 uh compatibility suit. Ledger 8 is slightly bigger because of recent changes. Uh but uh I will demonstrate to you Ledger 8. >> No, I was just cur that the non- Rust passes. I was wondering if those are sort of bug fixes and improvements that will be helpful to the compiler no matter what. >> Uh yep. In Rust, if something is wrong or not supported properly, uh Rust code just don't compile. In Typescript you might face some runtime issues but in this case it just don't compile and it's uh side uh effect benefit using Rust. Uh we already have a couple of projects where we use uh this uh uh generated Rust code and uh they works fine. We prepare the risks and concerns that you can ask about who is going to maintain this uh codebase uh how to properly merge it what is acceptance criteria. These questions are open and uh I don't want to go through these questions right now just to save some time. uh I will share it with u you guys uh or whoever is going to help us if you're interested in this initiative and uh we are looking for particular decision to uh make it with you and there is a plan to split all these 55k lines into six or seven manageable PRs after you figure out which ledger you're going to support ledger 8 or ledger 9 and uh can uh just two more minutes I just wanted to show you visual studio with uh example that it really works if you look at uh yep there is a smart contract uh that we used for midnight with a ledger witnesses types and circuits and this is a project that generated uh is able to generate a rust codebase. This is analog of uh JavaScript emission of code and all this code that is generated out of the smart contract is compilable and we have end to end test that have a parity between Typescript and Rust implementation. Uh I can share all the links with you but now it's time for questions. Um do you have a sense of what makes the Well, first let me there's two questions I have. One is um what are the feature what are the features that are are missing uh language features that missing? Um, I saw that note as we were scrolling past. And then the other is do you have a sense of why the rest back end is potentially larger or more difficult to add um uh at this point or is this something is this possibly AI bloat or yes it's wipe coded under our supervision plus prototype it and we reiterated a couple of times when we uh found a couple of issues and we have this stats with codabase. Uh so 39% is generated output between rust and uh typescript and test to compare the by par and that >> I was looking specifically at the size of the rust the the print rust passes I think it says something like 12,000 lines. Do do you mean uh generated code for the same smart? >> Okay, sorry. Never mind. No, I'm looking at the wrong one. Um, so the compiler rest passes. Okay, it's 10,000 lines. I mean, the the TypeScript one is like 4,000 lines. Do you have a sense of why why the Rust one is much larger than the TypeScript one? Uh I don't know what is the current size of TypeScript but >> it's like it's around 4,000. It's under 4,000. >> I uh tried to emit the co on the way uh I found a couple of problems and one of the problems was uh conflict in the data types. If you have a same uh declaration in a smart contract and you emit uh the same type twice and uh in Typescript you have a uh type full and full uh with index two or something like that. I improved this optimization and um the compiler creates a table of the types and if there is a same name and same arity and same types in the uh body of the strct the emitter for us uh creates only single type for both uh declarations in the typescript sorry in the compact uh smart contract. I can find this PR and uh as the latest improvements. >> Okay. Um still it's quite quite a bit larger. I'm wondering if um if that's >> if that's inherent or if >> I will I can feed all those questions to AI and we will find answers because yeah the column base is pretty solid. Um I um so there is something as you were scrolling past I'm sorry if I missed it in the earlier discussion but what are the what are the fe are there actual features that are not implemented yet for the rest back end there was something I I saw as you were flipping down further in your slides >> it's um do you know in um default by default Midnight uh and shielded company ships uh contract runtime package. It's a web assembly to wire the imports and types from generated code to the API and uh all other uh libraries of midnight wallet and other primitives. So in order to wire the Rust generated code into the midnight runtime, we also need to have this uh layer in Rust. And this layer should be called like midnight contract runtime, but it's a Rust crate >> and that is missing right now. You're saying you're saying that that that layer does not exist yet or does exist? >> We created this package. Uh we created this. >> So So the question is what are are there language features that are not implemented? >> Other language features uh I guess everything is implemented. So we have full uh support for us. uh there is still risk that we don't have parity with ledger 9 changes because I saw that they constantly adding but uh it works for all smart contracts that we have currently as examples and for our smart contracts and for other dups. So we tested the parity for leure 8 but as far as I know you're not going to support it anymore and it's frozen. uh so I expect that we will make a decision to if you're going to proceed with this to have some uh meaningful PRs into the ledger 9 branch. Yeah, I think the the real answer there is we will continue to support ledger for a while because at least until mainnet forks uh until we get the hard fork and mainet um but you're right, we're not we're not trying to add a lot of new features to it. Um going forward, you know, we're always going to have to support multiple ledgers. There's already a ledger 10 being created. So, um, yeah, it's it's it's never it's going to be a never- ending thing that we're going to have to support multiple ledgers. Um, no, I I like the idea of having a Rust back end. Um, and by the way, it's it's taking us from uh at least two present backends to to present back end. So, it's not doubling the number of backends because we still have the ZK back end. And the Zcara back end is is actually two backends right now because there's a ZKR v2 and a ZKRV3 backend. So this we're always going to be in this situation. And most real compilers have multiple backends. The trick to to managing it is to put as little in the back end as you possibly can. Have as much of it done declaratively as possible. And and I think that's where um often times the AI generated stuff falls falls short. Um and in my experience, they're not it's not very good at at it's it's very good at writing working code. Um it's not very good at writing concise working code that is uh you know where there's very little code duplication. Um, I think that'll improve over time, but but but right now, so I think that would be our our biggest concern would be that there would be a lot of stuff that could be commonized that isn't commonized. Um, and that's something we we we would want to look at because that's where that's where the overhead of of ongoing maintenance occurs is having to to maintain uh, you know, things that ought to be the same, ought to be shared that aren't being shared. But yeah, I definitely like the idea of having a Rust back end. I think it also gives us a a path forward towards having a WM back end. And um yeah, so it's it's definitely interesting. I just um I'm not I'm a little shocked at the at the sides of the back end right now. And I'm thinking this is probably because there's some um you know not as concise a code as as you could have. Um I've been thinking about this as well and uh I have two ideas how we can uh manage this. First of all, this feature is um fully compatible and additive to the existing compact flags and compact compiler. And if we can say that this feature is experimenting and uh you still we are still stabilizing it. uh we can start putting some codabase into the next uh release branch without breaking the main line and putting release at risk. And uh I'm willing to split the contribution of this uh 55k lines of code into a couple of phases that we can uh manage together and uh properly review ensure that there is no duplications and all quality aspects are met including the contribution guideline and u any compatibility stuff. >> Okay. I'm actually so I'm actually I'm actually looking at this is exactly the thing that I was asking about earlier. The language sub subset is incomplete. Um is that that's you're you're saying that that is actually not true anymore. Um I cannot test this. I don't know. Uh I tested plenty of variations of the I tested all types. as arguments, as witnesses, as uh disclosed, not disclosed. So there are plenty of combinations uh of all types and records and other stuff that uh I was able to imagine and I also multiplied it to make combinations in order to build different types in Rust and prove that it works in both Typescript and Rust. I would say that uh it worth to run um a couple of to burn a couple of millions of tokens and check uh what gaps is missed in a rust emission according to the specification of the compact uh language. But all smart contracts that uh I was able to find and test, I tested it for uh by parity between Typescript and Rust on Ledger 8. >> Yeah. K. This is a list of risks that's here. >> Yeah. >> So it's what if it's incomplete? What if it diverges? Um not necessarily a list of truths. >> Yeah, I get that. Um I was just curious if there were any known uh there's so so there's none that are known. Um you know the construction of the compiler and the use of the the I don't know if you're I don't know if it's using nanopass infrastructure I assume it must be at some level because that's the input to it but to some extent that can help prevent it from having these problems. also a if the tests or um if the tests for the Rust back end are adapted from the existing tests for the TypeScript back end which I assume is the case um then we can probably be as certain of completeness in the Rust back end as we can be of the completeness in the TypeScript back end right >> no the sitation is even better uh because uh there is no new rust uh back end. I fully rely on midnight ledger types because it's already in rust. So there is just a scene layer that is called uh midnight uh contract uh runtime compact runtime where we map uh the types of the uh smart contract ledger to the uh types in midnight ledger. It's mostly serialization the serialization and other stuff uh that related to midnight ledger. I would say that if we compare this solution to TypeScript in Typescript uh you have a couple of web assemblies and uh Typescript uh wrapper to allow developers some the friendly APIs and other aspects. In Rust implementation, we use uh midnight ledger types and uh packages directly without bridges and uh without um yep some other layers. It's like generated uh rust codebase run type and everything else is uh midnight ledger. >> Um can I ask a couple of questions about how um did you use the same agent to uh to write uh the compiler passes and then test it? Um as I demonstrated we have a project midnight dip it's one of pillars of cell identity layers uh that we uh building for uh midnight ecosystem and we have a smart contract in this uh uh project. So uh it's uh fully TypeScript. There are four packages and uh the smart contract code itself. Uh it's pretty big with job signatures and uh with other records. So this contract is around one uh up to 1k of uh lines. The biggest complexity is key um 16 as far as I know. And uh we have a package in Typescript that use full native midnight libraries. Uh if we look at the other packages like uh domain API API is a project with full standalone support and uh midnight node indexer and uh proof server and we just use it uh the same gener generated Z keys because um you know that it's tricky to distribute the approver and verifier keys together with zk gears keys. So we have um zip uh package with keys and having compact with rust code emission this archives with keys we have full compatibility between typescript and trust implementation. So >> yeah, my question is my question is did you use the same agent for writing the test uh for the back end for the rust back end that you added or did you use a different agent? >> In my experience when using the same agent for writing test that's that's very similar to a developer writing their own test because it's already biased. >> So that's my question. >> I've got it. There were around minimum three different agents. So I use colleagues include with cross review uh of the contribution and another colleague uh part is here but thank you for joining but also uh made integration of this codebase and he uses his agents as well. So we have plenty of PRs and a couple of iterations to make it work and you also can uh fit uh this codebase to your agents to find any gaps, problems or improvements. We happy to fix this as well. It's constant improvement process. >> Um does anyone have any more questions? let us uh we have a couple of open cops that we have to go back to and follow up with people on it. Um and this week the team was busy. So I'll make sure that next week we get some time to uh gather our thoughts and concerns on on these scopes. Uh any questions while we have Yuri here? If you have access to this uh clothe artifact, give me your email and uh because probably I cannot just copy and paste it. >> Oh, sure. Um I don't know where you can find our emails. They want to just share them here. Uh I'll send you an email actually. are thinking about it. Oh, if you take that, I'll send you an email with with our email addresses. Okay. Uh, thank you so much for for presenting, Yuri. I appreciate it. Um, anything else people wanted to discuss today? Going once, going twice. All right. Uh, thank you everyone for attending and uh, until next time. >> Yep. Thank you guys. >> Thank you. Thank you or somewhere else. Byebye. See you.