Making Generative UI work in production

Um, my name is Zahid Khan. I work at a company called Thesis. We help companies build um, UI for their AI agents, right? So, I'm pretty sure all of you are aware of generative AI. How many folks are have heard about generative UI? Maybe show of hands. All right, cool. So, So, I just want to give like a 2-minute intro to generative UI first. So, this is lightning talk, so I'll hope to keep it quick. So, um, if you have used any of these modern, um, AI coding agents like Claude, uh, you got Codex, right? So, when you ask something, it might Is the mic closer? So, if you ask something to the agent, it might ask back a question in in a form, right? Or if you are, you know, visualizing if you're, you know, going through some, uh, you know, uh, project management or something, they might generate charts because as a human it is more easy for me to read and visualize these data, right? So, uh, that is what, uh, OpenUI or, um, Thesis does. We help you build similar kind of, uh, UI for the AI agents, right? So, generative UI is defined as, you know, having contextual and personalized, uh, UI for the human based on the interactions that have been gone so far. So, a quick example would be let's say your end user is, you know, summarizing a sales report, right? It should show them a chart like a quarterly sales report, right? So, instead of, you know, having like a text summary that Q1, Q2, Q3, it's more visually easy for them to go through the charts, right? So, if you're building like a support agent, right? So, instead of asking the uh, user to fill five answers for five questions, maybe generate a form asking like what date was the date of purchase, you know, what category. So, they can just go through the options and select quickly. So, we have gone through like multiple iterations over 30 40 years of UI and for AI agents we are still back to text, right? So, we are trying to change that. So, that is a quick intro for generative UI. Right. So, uh So, we started in 2024. This was like when I think Claude Sonnet 3.5 had come in. Right. So, we were very sure that we can't just make the agent generate HTML and JavaScript every time because that is very slow, right? We were looking at real-time UI generations, right? So, if you look at HTML generation at that time it would take like 1 minute 2 minutes to just generate one page, right? So, we were targeting around 500 ms 500 to 1,000 ms, right? So, the architecture we came up was similar to server-driven UI. So, there's a front end, there the user asks a prompt, right? And on the back end what happens is it generates a JSON schema of which all components and how to compose it. And it sends it back to the back end. And on the front end there is an SDK that is able to convert that schema into like a React UI uh components, right? So, the benefit of this architecture was one, it is streamable like when you see text coming in and like in ChatGPT, you could have the similar experience with UI like you can see the cards coming up as as it when the LLM is generating it. And it is like 10x faster because we are generating fewer tokens. So, let's say you're trying to build a page, it would take around 4,000 output tokens, right? But since with the schema you're able to cut down it to 400 tokens roughly, right? So, this was the initial architecture. And once we took it to production, like we had like really good response from our you know customers. We had like 10,000 developers try it out try integrated it. So, we came started running into few issues, right? So, you know, we were betting on LLMs to be faster, right? But what we have seen is that the intelligence is growing, but more or less the TPS, the tokens per second are more or less similar. So, it is roughly between 40 to 60 tokens per second for state-of-the-art models like you know Claude, Sonnet, or let's say GPT, right? So, so now the users were complaining because they have like very orchestrated large workflows where there's reasoning, there are like multiple tool calls. So, now the generations have become slow again. So, we were targeting 500 like 500 milliseconds for the first UI generation like the the time to first token. That has slowed down to let's say two to three seconds, right? So, um like because the models have are doing a lot of work, the generations became slow back again. Uh the other thing we saw was the JSON generations were breaking because again because uh there were like a lot of complicated workflows, the JSON became like very nested. And uh the model kept messing up in between like they miss out uh maybe like a curly brace or maybe a quotes. And that used to break the um generation. So, we had to like regenerate it like we had to do JSON fixing, which used to slow the TTFT again. So, that was another issue. And the third one we found with enterprise customers where they want to stick with their existing design systems, right? So, they have their existing uh you know SAS application, and they want their AI agent to look very similar to that. And uh they were not they wanted full control over the component libraries that were using. And uh the fourth one was like custom front-end framework. Again, the same issue. We started off with React, but we had customers who wanted support like uh you know, native support on other front-end frameworks like View and Angular. So, um we were experimenting um I think uh in the first quarter, sorry, in January uh 2026. So, we came up with two major changes. We decided to ditch JSON, and we had come up with a simplified language uh called OpenUI Lang. And the on the other side, we had decided to split the OpenUI Lang to uh you know, we removed the you know, the UI renderer, which is part of the SDK, and the design library, which contains all of the components. So, I'll just get back to uh these two changes one by one. So, what is OpenUI Lang? Okay. Yeah, so it is a line-oriented uh language. So, the insight we had was two things. One is that uh you know, LLMs are better at writing code compared to JSON. Right. And the second part was uh like the mental model that we have is that the model has to keep track of all of the curly braces that they had generated earlier. And that was what what was causing issues with breaking braces, breaking codes. Right. So, we made a simplified uh line-oriented language where uh the arguments are you know, a positional based. And so, what is happening it helped with streaming. And uh again, since there were lesser tokens used, we saw that uh you know, JSON compared to JSON, we had like what, 67% uh efficiency. And again, uh due to these architectural changes, it became more robust. We had lesser uh you know, uh schema errors that were coming up in production. So, you can see a quick like we are generating the same number of components and you can see there's like 300 tokens on this side and there are around 850 tokens, right? So, this is what changed, you know, when we switched from JSON to OpenUI Lang. And yeah, so regarding the decoupling that we had done is that we made like a design library outside the system. So, you could have like Shade CN you could have, you know, any other you can have Material UI, right? You can just make a schema out of it and then send it to the back end and the LLM now acts as a harness like the schema sorry, OpenUI acts as a harness and it is able to convert you know, the schema into OpenUI Lang uh uh response and then back on the front end like the front end has context on the front end already has imported the design library and on the front end you have the React render any React renderer, for example, React and it is able to convert that OpenUI language into a uh UI or you know, deterministically. So, um so, you can, you know, swap this for like say Vue or say Angular, right? And the design library also like companies like if you bigger companies like say Airbnb, Razorpay all have their own design system and they can just import that and start, you know, having the same, you know, UI um feel and look as their existing applications, right? So, that is uh uh what we've been working on. And this is also show like a quick demo of the application. So, So, here you can see it is streaming the um response as, you know, um as with the LLM's response, you can see the UI is also streaming accordingly, right? So, my moving So, on top of this, you know, we've built similar experiences that you would see with um ChatGPT. The network is a bit slow here. Um I think I'll just switch. Yeah. So, I mean, that is the end of the talk. Um if you have any questions, you know, Yeah. >> Um >> I don't think we need to pass the mic. >> All right. >> Yeah. >> One of the main reasons I think the JSON is output or any other >> Mhm. >> one output was used so that you don't have to tell the model like uh it already knows how to >> Exactly. >> That's the main thing that we discussed. >> So, what are the challenges that you faced in uh asking these frontier models to follow your new kind of a online languages? What is the overhead and in terms of >> Mhm. >> like every time you have to spend some context to get it followed, right? So, >> Exactly. So, this was the same reason that we were very hesitant to move out of JSON. Like, we had this thought since the end of the year last year. So, we were then we experimented for 3 months before pushing it to production. So, on uh So, you know, what we've seen is the input tokens are very cheap now. Plus, if you add in caching as well, so it does not hit you that hard and uh so right now uh if you look at the language spec, I think it is around 1,000 to 2,000 tokens. Right, there are like I I can agree that um there are some uh you know, challenges still. For example, you know, once people saw Open UI Lang, they wanted to generalize it. They wanted as a full replacement to JSON. Right, so we tried it out, but JSON was working well for across the models. What we've seen is this works really well for state-of-the-art models and few models like uh we like say Qwen and Gemini 3.5 was working really well, but if you go beyond like let's say 3 billion models uh 3 billion parameter models, we're seeing like poor uh performance. But, for anything above 30 billion, we are seeing like really uh good adherence. Uh I think a lot of people we've seen other companies also experimenting with similar approaches. Even Vercel has Zero Lang. They launched 3 months ago, which has which just has the same ideas that we have. And there's also another company called Aero JS that does more or less the same thing. All right. So, uh we're still working on it. Like like I said, we consume around 1,000 to 2,000 tokens uh on the input side, but yeah, we see overall there's a speed in improvement. Yeah. Sorry, improvement in speed. >> So, two other quick questions if you don't [clears throat] mind. Uh so, one is about uh yeah, the Gemini models are like they have like much better uh throughput in terms of tokens per second. >> Right. >> So, uh yeah, so this flash kind of models, do you uh the the does this work with that? And second question is probably around the coverage. So, we saw some few things that could be covered just by single lines, right? So, the coverage in terms of other things that you want to generate in UI, are you able to convert what would be the JSON equivalent of that into your language? >> Yeah, yeah. So, from a JSON perspective, it is like a full language. So, you are able to convert everything. That is not an issue. Like I said, the newer like what we are seeing is like the smaller model like Gemini and all are having some issues. Like if you go like I said, if you go for 4B or 2B, then we start seeing issues. But, you know, for other models, we are seeing better performance. Right? So, we also expect we are yet to experiment with diffusion models, right? So, we we actually had issues with JSON rendering as well. With diffusion models, but we are still working on it. >> Thank you. >> Yeah. >> Yeah, so I'm new to this space. I just wanted to since the renderer is decoupled from the generation of the components, like is it would it be possible to render my own components in Plotly stock through an MCP for example? >> Yeah, that's a good question. We are still figuring out. So, we had built something with MCP apps earlier, right? So, the thing we are still figuring it out is basically we can generate so, with the new versions that we are working on, we want to build dashboards. So, instead of connecting it to a data data set points, we are connecting it to the data sources directly. Right? So, our idea with Plotly is that you can have like multi MCP apps. Right. So, let's say you're a CEO, you want to connect to your calendar, you want to connect to your Stripe, you want to connect to your Google Analytics. So, we want all of that to get connected. So, we are working, you know, working to figure out how to make that work. But with single MCPs it's very easy. Like we had already built one version. Right. So, the idea we are working on next is on agentic interfaces. So, think of it this way, like our thesis is agents need to move beyond chat. Like we want every application to be generative. So, the user does not really know that it's a generative AI application, it's just an app for them. Right. So, we build these mini apps, so which are very custom for that user. Yeah, right. So, that is what we're working on. So, on we actually built this for Open Law as well. Right. So, you can have like So, think of it this way, you just you have to keep prompting every day for the same things. Right. So, you can have like a quick dashboard and you can keep checking it out. >> Thank you. Great domain in particular. >> Right. >> Uh hi Charlie. >> Hi. >> Um I think I'm going to take like a step back from Open UI. My question is more just it's a new frontier, right? Like you're thinking about uh creating applications differently. Users are driving their own interface in some sense. >> Yes. >> Uh so, I'm excited to see that. I definitely think we need to move past chat as well. But in terms of what you've seen so far, what would you say is like most out there in terms of this is like an exciting new interface that wasn't possible before. Um there are there any examples that you have that are just you >> Okay. So, we've been uh it's like personally I've been tracking few YC companies that are working on this. So, what this is what we call um SAS 2.0 where every company tries to become their own lovable. Like people are able to build like companies are helping users to build their own applications within their environment. So, let's say like Salesforce, right? They have all of the integrations. Then the user is able to build their own applications, right? Within Salesforce. Um, let's say all of them have their own context within and they understand the domain better. And you know, so that is what we're working uh I mean we have not seen like people reach that stage yet. So, that is what you you what we're working on. So, the other side uh this is for the companies. So, we are thinking of in two domains. One is for business software and also personal software. So, for personal software that is the idea behind building for Open Law. People are building their own app sections with you know, they're trying to connect all of their own tools, right? So, that is where we see like that is where we want to push more people, right? So, right now people have been using Open Law on Telegram, right? So, now I mean not now, but yeah. Uh so, with Telegram, you know, everything is in one chat window, right? So, now people are moving into their own dashboards. Like even Hermes had to build their own Kanban board kind of thing. They had their own desktop. Because once you have like a system of agents, then it becomes hard to track the agent and their work as well, right? So, we are also experimenting here. It is like like like you said, it's a new frontier, but yeah. >> Thanks. >> Anybody else? So, you typically use JSON structure to because there is a logical listing >> Right. Right. >> How did you break that nesting? Seems like your language I I didn't go through the actual >> So, um it is like So, you you remove the nesting, right? You can just tell key a.b, you can just say a.b again, right? So, you can still do that. >> Got it. >> So, like I said, the thing even like we were very hesitant to move out of JSON as well. Because like he said that it is already trained on JSON. But like the intuitive understanding that we have is, you know, models have issues with all of the curly braces, all of the quotes. So, we have This is one way to like remove those, right? And the positional arguments help it out. >> I understand. Now, now in your example, you mentioned a.b. >> Mhm. >> So, instead of tracking braces, it's now tracking >> a. >> That is more easier for like we are still we are not done like we are trying to build a >> You have done your research. So, I have no idea how >> Exactly. We We are trying to make publish this. So, we actually had a conversation with Alan from Google. So, Google has their own framework called A2UI. So, they had similar questions around why you are using Open UI Lang versus JSON because they found it more easier. So, we are working together to build like a public benchmark for, you know, UI as well. So, that is work in progress right now. >> Nice idea. >> Thanks. >> Yeah, a couple of questions. Do you mean what sort of control do I need to plug my SDK meaning that I have to render I have to have my SDK embedded onto the front end whichever is handling it. >> Mhm. >> So, for example, like Telegram and all, will that be possible? How do I embed my HTML for this? >> Yeah, so right now the focus is on like web and mobile, but you know, Telegram has now rich web apps as well. So, you can embed you know, HTML into that, right? So, we are actually thinking on those spaces as well. So, what we've seen in Microsoft Teams also, you can have like HTML canvases as well. So, there you can have like, you know, a dedicated UX experience. But for like, let's say Slack, they don't have that support. Like they have like a block it what they they call block it. So, there is like inherent restrictions on what you can do for security because of security considerations. So, yeah, so right now like the focus is on you know, web and you know, mobile. Yeah. >> Now that was for open UI, like did you explore some other thing like Dune or something before you sort of decided on building an app? >> Yeah, so with Dune what we've seen was when there's like strong nesting, like the token efficiency does not work out really well. Plus with tool calls and all it started breaking. So, that is one reason we explored and had to ditch it. Right. All right, thank you, folks.