Sanjay Gupta, Velaura | theCUBE + NYSE Wired: AI Factories

Palo Alto Studio Connection Silicon Valley and Wall Street. I'm John Fost here with Dave Volante, my co-host. Welcome back to the Cube studio here at the New York Stock Exchange. I'm Jim Allen, co-host of NYC Wired. And today we are talking all things AI factories where the conversation around AI has largely been about models, GPUs, and the extraordinary amount of compute we are going to need. But there is a more fundamental constraint emerging, and that is power. Joining me now to unpack that is Sanjay Gupta, co-founder and president of Valora. Sanjay, great to have you. >> Thank you, Jam. It's a pleasure being here. >> We connect Silicon Valley towards you on the show. That is our tagline. You're in from the west coast. You guys are heavy in the infrastructure space. Raised a lot of money this year. Big valuation. Valori, I valued over a billion dollars. Break it down for me. Give me the 101 on this company and the thesis. >> Sure. So again, we're very excited to be in this particular area. Our focus is purely on AI compute, but we approach AI compute from a very different angle. We actually approach AI comput from the area of power and really low power >> because the general thesis is that there will be a lot of compute for training and inference but we'll be running out of power very soon. The amount of power being consumed by AI data centers is rapidly growing up. So we've got unique technology which reduces the power at the same performance by an order of magnitude. >> So our customers are big hyperscaler customers and they are delighted with that technology. And then we have a second area which is a very exciting area and that's in the world of physical AI. I think robots, drones and the like. So >> we talk a lot about that in this show. So we're definitely going to get into that. But first of all tell me how how do you have a more efficient power methodology on the chip? Is this what what level and design does this happen at? Like give me the technical spec here. >> Sure. So I'll give you a little bit of the background. So we had a company prior to this where we had special computational technology for blockchain and more specifically Bitcoin mining technology and interestingly in the world of Bitcoin mining it may not sound that high-tech but it truly is at the cutting edge of power efficiency. >> It is the most important metric is reducing the amount of power what they called jewels per output or jewles per terahash or flops. And in that we perfected technology which is at the cutting edge of silicon. It's really at the leading edge of that and a variety of proprietary technology we developed to reduce the amount of power. And what we found is that you can apply that similar technology to the world of AI compute because it's mathematical calculations in the blockchain world >> and it's a different type of mathematical calculations in the world of AI as well. >> Funny story. I thought about renting out my apartment during the World Cup in New York so I could go home to Ireland for a few weeks. And I saw in all these Reddit threads, be so careful that you clarify these aren't Bitcoin miners because your electricity bill is going to be absolutely astronomical. So, and I thought that was so funny, right, that people have actually had that happen. So, but one thing we know is that, you know, it's a powerheavy business. Talk about identifying a unique TAM for velour AI though, right? like deciding okay there is like a a mathematical rhythm to this that we can actually use in a transferable fashion and break into the semiccon space because it's a pretty noisy and it feels like saturated space yet there seems to be not enough supply to meet demand talk me through your kind of the business mind behind it >> so so I'll give you an illustration so if you take a typical GPU uh or any type of XPU for instance you know order of magnitude they consume around a,000 W and of that around 40 to 70% of that power is consumed in these arithmetic calculations. >> Our technology brings down the power consumed in that by a factor of 2 to 4x. So we actually operate uh the particular compute at what is called ultra low voltage or low voltage and we have got a number of unique proprietary techniques and methodologies where we get back the performance when you actually drop the voltage and using that we can improve the power efficiency like that order of magnitude 2 to 4x applied at the chip level that reduces the power by somewhere to 200 to 400 watts on a 1000 watt GPU. M >> now if you take that's the 20 to 40%. If you look at the amount of power that is expected in data centers in the next year in just in the US there are different estimates but people say it's around maybe 25 GW of new power reducing that by 40% would be 10 gawatt of power that's an extraordinary amount of power and value. So we're talking about a TAM which runs in the billions of dollars just for the ultra low power compute area. Let's get into the chips for a second from the perspective of product and design fit. Right? We hear GPUs, LPUs, CPUs, TPUs. We hear, you know, there's a plethora of chips emerging. You know, we talk about them on this show every day. Different purpose, different missions in some respects, but at the top, they all kind of converge, right? >> How do you think about it from the perspective of product fit for this technology? Like where is it really having immediate realized value? So uh a lot of these calculation like you mentioned whether it's a GPU a TPU some other type of NPU today all different kinds of like processing units which is what PU stands for >> at the core a lot of them do matrix multiplication >> okay >> and then they do associated mathematical functions around that matrix multiplication so our technology applies to that >> where we can improve the power on that at the same performance in the same area by an order of magnitude and these have got different applications ations in the world of training. They have got applications in the world of inference. Inference is taking off in a very big time way and it's expected to continue to build in a goal. So reducing that power for those data center chips whether it's training, whether it's inference, whether it's at the edge is is where we're focused on. So even today we are working with some of the top hyperscalers in the in the world on this technology. We've done some very successful Prusso concepts with them and we're they're including our technology or planning to into their future roadmap in the coming you know months and years. >> Can you talk to any specific example any kind of customer use case any one hyperscaler? >> So I can talk in the abstract because of confidentiality reasons of course I cannot name them unless they want to be named specifically. >> So there's only so many number of hyperscalers and these are all big complex companies. So as you can imagine you when we first met them they said this is not possible. Reducing the power at the same performance by a factor of 2 to 4x is is just not thinkable. So we then had multiple technical discussions with them even to get to the point where they were comfortable in sharing their underlying code their secret sauce with us to work on that we can reduce the power on. So just getting to that point was was quite a journey. >> Yeah. Once we got to the point then they say okay let's actually do a proof of concept and we've been doing those proofs of concept they have actually been blown away with the proof of concept and now they're in active discussions around including it into their real road map in the chips that'll be coming out in the months and years uh so that they get a competitive edge in the marketplace and anyway addressing the power issue is going to be fact a lot of people say it's probably going to be the most important area because you'll have compute you will have more and more compute in the in the market but you will not have enough power. >> Wow. >> And that is what we're trying to address at least on this side of the market through this solution we call it's called Titan core. Uh and that's this ultra low power technology that we have. >> I mean we hear a lot about grids being maximized right all over the world over too. Not just here in the US but in Europe we hear a lot about this. So electricity and power is certainly a global problem I would say like many others. Talk a little bit about the customer dynamics that you're seeing unfolding. You mentioned inference, right? Nvidia have earnings tomorrow. Inference has been the word of the year so far. We're in the inference era. Long right training models. We also you mentioned in your opening there around sovereign, AI in the edge, physical AI, robotics, >> right? >> Where is the where are you seeing new TAM like talk about what you're actually seeing unfolding and also who's truly dominating in the space do you think? >> So it's interesting two different sides of the marketplace. So one is you call the data center uh compute market. So that's where you have for training you need some very heavyduty uh compute that is taking place you need a lot of memory and then you know you have ongoing calculations that are done. So that's one side of training when you're training larger and larger models for AI. The area that's expected to explode it's already is is inference because then you want to make some something out of it and that is exploding in a variety of ways. So just more recently I'm sure you know a lot of you and your your listeners have learned about AI agents which are now just getting started and that is exploding the amount of compute that will be required in the coming in the coming years. So within again training you know you have different elements then you have inference and then you have AI agents coming through which is going to expand the market substantially. Different part of the market is physical AI. So physical AI is like I mentioned robots, drones of different type large market today. They are poised for phenomenal growth. Power is a big element out there. >> A lot of other things are critical for that market. So we're also focused on that market as well. >> And a lot I mean in that market right if we really break it down if we think about it from the perspective of deterministic a lot is happening on the device right. So when you think about lower power, lower compute on the device, yet it still needs a level of, you know, high efficiency, there's no real room for error in the situations that you mentioned there, like robotics, drones, you know, we we talk a lot about defense technology on the show. You know, talk a little bit about the competitive messaging around that, especially as you're trying to sell into this space, right? Company, you know, I'm sure it's a busy and noisy space, a lot of legacy business. Let's talk about how you position that from a GTM perspective. >> Surely so. So on the physical AI side, it's actually a very interesting market, right? So even today, you find that you have computation chips, but they lack in a few different areas and we've talked to a lot of now customers in the marketplace. A lot of robotic companies, a lot of drone companies in different parts of the market. We at this show called Automate, which is one of the largest shows in physical AI. So three or four. One is that the chips consume too much power and as you mentioned Gemma they are on the device. So one is the power consumed especially if they're not plugged into the wall >> then battery life is a big issue >> for sure. >> The second thing linked to that is thermal >> because if they consume more power they also generate more heat and that creates a problem when you put them on the actual device or robot as well. So that's number two. The third part is that a lot of the robots today you find are very fixed. So there's some industrial application be a fixed arm pick up pick up or they'll be in a cage. The breakthrough that's expected to happen is when it comes to autonomy where you have the robot coming out of the cage interacting with humans like you and me >> and that's where safety becomes critical. >> Yeah. >> And safety you want to treat very carefully but at the same time you don't want to have false positives. So you don't want to have the reverse case where a cat runs across the factory floor and all the robots freeze. You want to have the right amount of safety which is critical and a lot of the solutions today do not have that. And then you mentioned about deterministic a lot of the solutions today they are more probabistic in nature. >> Whereas if you're down to make a decision like a human that you walk into a room and you need to react you need to react in that time frame and that needs to be deterministic. So deterministic and low latency with a high bandwidth and throughput are critical. So that's where the physical AI special purpose-built silicon is critical and that is what we're actually focused on. >> I want to ask you about the source of the power and your predictions for this market 101 15 years from now. Right? We talk a lot of electricity. Obviously it's the OG. It's the it's you know it is it is what it is. >> Sure. But we've also had fleeting conversations around nuclear, some level of progress with solar. People kind of debate that, right? At the end of the day, it's it's, you know, up for debate, I think. What are your thoughts, though, in terms of where this market is headed and where do you think we really might see some new adoptations of a power source in the next 10 to 15 years? Do you think it's likely? >> Sure. So, so again I can talk about it more indirectly because those are people that we know in the in the industry that are running large scale data centers at different types. So we hear actually you know through them. >> So clearly alternative sources is going to be critical whether it's different forms of renewable energy uh nuclear energy solar energy. Today the costs are prohibitive but you hear again about you know the whole area around SpaceX and trying to take data centers into space to get solar energy. Today the capital costs are quite extreme but in the next you know 3 years 5 years as the capital costs come down and the operating costs are much lower then that could be an alternative approach as well. So I think all of the above are going to be needed because to sustain the type of growth that we're talking about there's going to be a lot of power and then there's also going to be a need to reduce the power consumed at the same compute which is what we're focused on. So for instance, if you can double the compute within the same electricity footprint, that is a game changer. >> Sure. >> And that's the type of game changer that as Valora we are trying to focus on. >> So I want to talk a little bit about your career and your current position at Valora, right? You guys went to market in January. You've already raised 110 million million over billion dollar valuation. Like a lot's happened in a fast space of time for you. When I think about that, I imagine, you know, you and your co-founder waking up and Danny thinking, "We're going to go hell for leather on this." It's not easy to get time with Nvidia or AMD or whoever, you know, I won't guess, but any of those hyperscalers. Everyone's trying to fill into them. You've obviously done something right. You're a seasoned exec in the industry. >> Bring that to life, but I'd love to understand like, you know, give me some thoughts on how you think GTM is working well for folks and how you approach it. Sunday. >> Sure. So personally I've been in the industry for over 35 years. I did my undergrad in electrical and electronics engineering >> from one of the institutes of technology in India and then since then I also worked across multiple phases of industry cycles. I was in management consulting for a long time. >> Uh do you have the war wounds to >> so I've been around you know a lot of wars around that all around the world you know there I was at Mckenzie as as an associate partner in Mckenzie. Then I worked in a lot of Fortune 100 and Fortune 500 companies leading large multi-billion dollar P&Ls. I've seen the transformation technology from the internet days uh of the late '9s early 2000s to the boom of the mobile phones to the boom of the data centers and now to this wave of AI. So it's been other co-founders have a very strong pedigree in technology and in Silicon Valley in the Bay Area. One of our co-founders and CEO he's been a veteran. He's had three very successful technology companies at each of these major changes of the internet, mobile and and cloud. His name is Rajiv Kimmani and he's very well known and he's got a tremendous background. Another of my co-founders, his name is Manum Guli and he's had he was one of the lead was the lead architect for a number of generations of Apple's iPhone and iPad chips. >> Wow. >> And then he had his own company called Nuvia which he sold to Colcom. And my third co-founder, he actually was the uh CEO of a listed company, in fact, listed on NASDAQ for over 10 years uh and has been in the industry. So all of us, we actually got a very strong background in the industry and that itself brings credibility to the market and also brought credibility to our investors as well. So a lot of our investors uh we've worked with over many years and there's a level of trust and that they know that this is a team that will make things happen if they say it'll make it happen. Now, well, you've got serious chops between you there in terms of everything you outlined. So, still year one of this company, exciting mission, lot of money, got some runway now. What's ahead? Are we going to what does the next four to five years look like? Are we going to see you ring the bell, not a NASDAQ, here at the New York Stock Exchange Sunday? >> So, first of all, we're just we're excited about where we are. We're just getting started. This is just the start. So, first of all, it's a validation in the team that we've got this investment. It's a validation in the idea, validation to a certain extent in the TAM and a validation in the belief that we will execute. So we will be executing against it. We'll be looking for tremendous and exciting growth in the area of AI compute both for ultra low power and physical AI. We would love we've had a great partnership with the NYC. It's been wonderful to actually have connect with you and Brian and the rest of the team at the NYC wired. So that's been you know fantastic and we look forward as said you know the future what it will behold including you know what what can happen on the NYC you know down the line. >> Well we certainly look forward to watching and cheering on your journey. Sanjay thank you so much for joining us in NYC Wired. >> Thanks a lot. Thanks very much for having us. >> I'm Gemma Allen here at the Cube Studio at the New York Stock Exchange. This is NYC Wired AI factories. Thanks for watching.