Faraj Aalaei, Cognichip | theCUBE + NYSE Wired: AI Factories

Palo Alto studio connection Silicon Valley and Wall Street. So I'm John F co here with Dave Volante my co-host. Hello I'm John Fer host of the cube here at the cubes New York Stock Exchange studio. Of course we are Peloto studio connecting Silicon Valley to Wall Street. This is our AI factory series. We talked to the leaders who are making it happen who are helping expand the AI era the AI infrastructure and all that is enabled and accelerate on top of it agents physical AI and you know the need for semiconductors the need for systems is in high demand and this next segment will address that cognition CEO Farage Ali welcome to the cube thanks for coming on appreciate it >> thanks for having me John >> you know cognition you guys are now announcing your product it's been stealth for a while you might have solved the biggest problem that everyone wants small, faster, cheaper kind of in the Intel Moore's law way is what people want. They want smaller form factor for AI. You guys have a chip. Talk about what you guys are doing. What is the product? What is the vision? >> Well, uh so what we're building is really the industry's first in four different areas. John uh first of all we are training our own models which are physics informed meaning that they understand um chip design transistor design and how it all comes together to make a product and that's a fundamental um difference than uh essentially general purpose LLMs that get bolted on into some kind of a chip flow. This is built from bottoms up for the chip industry. The second part of this is is that we uh put all of this in an end-to-end system in end-to-end infrastructure that allows a designer to get in and start from idea generation all the way up to a complete chip design in one system and that helps you build uh products in a much shorter time. Right? We think that we can for a given task we can probably break it down by a factor of 10 in terms of time collapse and then of course the cost of developing that product get significantly reduce time to market improves and and all of that. The third uh part of this whole thing is is the compute infrastructure is how do you take a technology for the last 30 or 40 years has been put together with serial format essentially in silos and now collapse it into what we refer to as a spherical approach where you're optimizing around all the elements of a chip design into one system uh you know at uh at at a reduced time. And then last but not least, it's a system. It's an infrastruct design infrastructure system that understands workflows, chip workflows, the intuition that's built by an engineer that has worked for 30 years into this system. So not only it can get you there faster and can get you there cheaper, [snorts] it can get you there with confidence that you're designing the right kind of product and there's no hallucination in this kind of a kind of >> So you got AI everywhere on the chip side to help you guys do that faster, but also it's a full stack intelligent chip. Now I want to ask one question if you don't mind because I hear this all the time certainly here in New York. Everyone talks about oh make a tape out in Silicon Valley and on Silicon Angle.com my my site uh is we cover the valley. We've been seeing the chips evolution. It's a long process. Tape out TSMC. I hear people, oh, we got to get the more chips and it's literally years. Is the innovation the speed of chip building? What specifically is it that you guys have? Is it the speed to get a chip? So, if I'm building something, a robot, I just get my own chip. Is that kind of what it is? So what we're building is actually a software uh a software product that helps designer get the chip done with a lot less resources in a shorter period of time. And yeah, you can actually build any chip you want. Our customers are semiconductor companies or frankly anybody that builds chips these days which includes all the system guys, the cloud guys and and so on. And so with this software you can you can actually just uh interface with it like you do with any kind of model um today. Uh it's it's really a uh a foundation model uh for chip design. Uh that's the way you should think about it. And um yeah and and as such you can you don't have to be a chip expert anymore >> to design chips. Yeah, >> you can have an idea and a concept about how to approach a market and what a market needs whether it be in the physical uh AI or whether it be in in some kind of enterprise infrastructure you can use this system to build your chips >> project you know it's interesting because if you're an engineer and you're in the se you know how hard it is to build a chip >> it takes years before you even see a chip then you got to put in trials take it to customers and then you start making money in volume um we're in an highly accelerated market right now even things like quantum computing which was a fantasy just a few years ago. It's highly accelerating fast. AI has come on the scene. You're seeing the models come in fast. What about the share your thoughts on the speed because you know if you asked someone 5 years ago I can build the chip in a month like what are you crazy? >> Yeah. >> I mean this is talk about the velocity of the innovation cycle now and why is it possible to do this now? >> Well, let me start John by saying that you know and I've built many chips. when you start to design a chip, you know, this chip isn't going to be in production for another 5 years because it takes you a couple of years to build it. That's been the story so far. And through it, the semiconductor industry has done actually quite well. But in order to keep this growth and given how fast software actually uh steps up, hardware needs to catch up. We can no longer go on and taking two three years to design chips. It needs to be shrunk. And this this system allows you to do all of that and create bespoke devices. Now, bespoke devices have other capabilities, meaning that they're lower power, >> they're better performing devices and lower cost through the marketplace. So that kind of keeps that whole cycle going of innovation on the chip side feeding innovation on the software and you have a virtual cycle that that's happening with >> you know we we talk a lot on the cube and on silicon angle too we write about it a lot which is you know when you see these big u data centers the GPUs there's a lot of horsepower involved a lot of density pick your approach pods density when you look at the innovation um around the bottlenecks because okay there's a lot of scarcity of resource chips supplies and constrained >> engineers will will go to the constraints and they'll engineer around it and one of them is how do I get the same performance at with a different set of [snorts] hardware and software and I think you're starting to see that you guys are doing that you're essentially coming in and filling in a a demand cycle that's saying hey if you want performance and by the way it's distributed computing so you can actually have it all connected >> talk about this dynamic of you don't have to have the big iron the clusters to do some of the things that are required in AI. >> Yeah. With with ACI Enterprise, you can go two ways. You can build a bespoke device that gets better performance and lower power in a much shorter period of time. Or in case of physical AI, you can actually take a model weight and pull it into hardware within a matter of days and not even weeks or months. And that solves a huge problem on the edge because on the edge uh if the information keeps changing you can't keep just yanking the hardware out and replace it with the different hardware. So you need some amount of programmability in there but that programmability has to be enabled in a way that it connects the model weights to >> to circuits running in hardware with low latency and high performance and and that's the magic what this technology provides. >> Talk about the open weights. There's been a lot of discussion on open weights. Obviously open source has been a big driver of the innovation. What's the importance for open weights in this paradigm? Because you now have open source free now you have solutions. You mentioned some of those weights. Do open weights play into that and what's the impact of that? >> I I think open weights are crucially important to the industry as a whole. Um you know in a lot of in a lot of cases um a lot of manufacturers want to be in control of that uh that piece of software or the weights of the model and open weights allow them to stay independent and move at their own pace for their product u evolution. Of course you know closed source models also provide certain capabilities and every industry is different right and so we should have a mix of both and >> which is a good thing. >> Yeah it's a good thing it's it's fantastic. it it creates applications a lot faster and and makes it easier to build valuable products out in the marketplace that uh that serve a purpose. >> You know, one of the things we cover a lot is the enterprise and obviously edge is exploding now. We're a lot more this year. Um you know, the classic enterprise doesn't have billions of dollars to spend on capex. >> So they're going to look for open wage. They're going to look for the kind of solutions that would fit into a classic data center. They they'll probably connect to a hyperscaler. They'll probably connect to a Neocloud for service, but that's not the endgame. There's gonna be a lot more action going on on the premises. >> Absolutely. >> Hospital. Share your thoughts on how you see that developing and how you guys fit into that. >> Well, look, um several industries, defense industry as an example, uh semiconductor industry where where where we we play a big role there. um they um have very high expectation for information security and for being able to essentially have air gap systems, right? And so open rate solutions uh provide you the ability to take those and make them your own build your system architecture around them and keep them proprietary the the total solution proprietary and and certainly you know we're enable of that uh enabler of that and and even in our own case for example we are developing our own physics informed models those models are really important to our customers in the sense that they know they can use them and they know all of their information stays private and where it is in their environment and that's critical point for semiconduct industry because in semiconduct industry your IP is everything you have right and you do not want to be sending that here and there and not knowing where it goes right so we give these customers the comfort that they can get maximum intelligence and chip design while protecting their IP at all times >> talk about what it means to be physics informed because I think this is a unique feature also workload aware you guys have that it's kind of an intelligent chip what does that mean physics informed it just knows everything about physics or specifically about the materials. What is the what is the what does that mean? >> Yeah, it's a very good question. So, you know, general purpose LLMs are built essentially to answer any question you have, right? When you come into industries like and and the way they have that knowledge and intelligence is because they've trained on gobs and gobs of data that has been collected everywhere. Well, there isn't gobs and gobs of data for semiconductor industry. So a model, a general purpose-built model does not know how to deal with the physical reality of a transistor, right? So what we do is the the way we train our models is make sure that the models that we're building understand transistors, understand it has a physical formation, it has power um capabilities, it has frequency capabilities. And so in order to then be able to start from architecture level and end up at a chip, if you're coding for a chip, you need to know what this thing is going to look like once it became synthesized into transistors. If you don't have that, you can't build models that are good in semis. So what we've done is we've invested since day one in building the largest data mo in the semiconductor industry and use those data mo those data stacks that we've created in training bespoke models for chip design. So it is physics informed because the moment you start to say I want to do X Y and Z it understands how that's going to translate into transistors on a chip and that is the magic. >> It's like having a team of engineers >> on on staff. Absolutely. That's what basically it turns into. >> Team of engineers who are expert across the semiconductor field across all various applications at your fingertips. So it's really a digital co-designer. >> All right, let's get into the news. You guys are launching the artificial chip intelligence ACI enterprise world's first full stack intelligence infrastructure for chip design. That's the that's the tagline. That's the that's the sentence. Take us through the product, what your plans are, target customer, is it a designer? I mean, we're hearing OpenAI is designing their own chip. Anthrop is building their own chip. Everyone wants to build their own chips. >> I'm like, well, good luck with that. But now, >> and we're here to serve all of them. >> Take us through the value proposition, the product opportunity for you guys. >> Sure. So, artificial chip intelligence, ACI, um, is really, if you think about AGI, we are we are ACI, which means we're focused on the chip and chip intelligence. And the way we bring that to bear is that we combine that with very uh detailed workflows on top of these models and a customer can start by saying I have this idea or I have this specification and I want to build a chip and we take him through all the steps that the industry has been going for 30 years but we do it at blazing speed. We do it at 10x the speed at minimum. And so what does it do? It allows you to code in chip language. >> It allows it to synthesize this, simulate it, synthesize it, and create something that meets your requirements for power, performance, and area. >> And that's never been done before. And you can do this at at software speed really. >> Yeah, there's an old expression in the industry uh we've said it many times in the cube. It's quoted most by Andy Jasse who's the CEO of Amazon. It says he says, "There's no compression algorithm for experience." Actually there is you guys are doing it. You guys have this special specialism >> with the semis. Specialized intelligence is the hottest area around general intelligence. Great. I want to know what's going on in the world. The the the frontier miles crawl the internet. They know all the general answers. So AGI whatever. >> But now specialized intelligence. Talk about the impact because you guys are essentially doing that. You have specialized intelligence on the chip chip design >> all the process management process experience. You guys have compressed that. >> Yeah, we have compressed that. So, think about it this way, right? Today, if you want to design a chip, it's going to take you hundreds, if not thousands of engineers, depending on how how complicated the chip is, and it's going to cost you several hundred millions of dollars to build a chip. What we're set out to do is to change that entire game, change the industry in that way in that you don't need thousands of engineers to design a chip. You need a very few, but you need an intelligent ACI enterprise system and you'll arrive at the same chip, perhaps more bespoke and perhaps even better performance uh at a fraction of time. This changes our industry because now you can go from idea to shipping product at close to the speed as people develop software and take it to production, right? And and and close that gap. And the more the more you close that gap, the more profitable semiconductor companies are going to be. >> Talk about uh your background. You it's not your first rodeo um in this area. Um and bespoke was always kind of a I won't say bad word, but people would roll their eyes. I'm a building a bespoke system. Whoa. No, we want to have a broad market opportunity. This is a unique paradigm shift. Yeah. >> What's it like? I mean, compare it to what you've done in the past. You have multiple chip companies. >> Yeah. So, I've built and taken two semiconductor companies public. And what I tell you is because of the long time that it took to design chips, we pack a lot of stuff in these chips. We make them as general purpose as we can. That comes with cost. That comes with the cost in terms of time to market. It comes in terms of actually the cost of the device. And so in the world today where we have different models and different workloads, actually building being able to build a general purpose or build a bespoke chip. But if you want to build a bespoke chip, you're going to have advantages in the marketplace because it is going to operate at at at performance you want and doing the kinds of things you want and not much more than that because now if the world changes, you can react to it very quickly. It's not like the old times where I have to wait those 5 years to get a chip into production. >> Yeah. And the programmability you mentioned earlier, talk about that aspect of that requirement in today's world. >> Yeah. So um you know there is you you can build uh programmable devices obviously um today FPGAAS I think are going to be well position by the way in this marketplace for stuff that happens at the edge in the physical AI uh field um and being able for us to take from idea to an FPGA program that runs hardware at speed it can happen in matter of hours to days and that closes the loop in and really being able to bring these products to marketplace and changing their behavior on the fly almost. >> Got it. Um, final question for you. Now that you guys are launching, launched, what's the focus? What's your plan? >> So, our plan is to expand our market reach with our customers. We're engaged with 40 type customers today. Uh, and this is just the beginning of uh of our product introduction. Um, our plan is to continue to make our model more and more intelligent. uh continue to add features and capabilities and really make this system easy to use where you don't need to be an electrical engineer to use this system to arrive at the solution for chip design. >> Awesome. Congratulations and looking forward to catching up after the launch. >> Thank you very much. I'm John Furrier with the cube AI factories again the innovation the engineers will go to the constraints and one of them is small form factors high performance edge robotics starting to see a lot more activity more power is punched into the system when you have the smaller faster programmable devices again another example of the innovation here in the cube thanks for watching