Navigating the AI Frontier: From Harvard Prof to Palantir's Head of AI Systems — Dr. Matt Welsh '92

Today we have the honor of speaking with Matt Welsh, an AI visionary whose extensive career spans leading roles in both industry and academia. Currently the head of AI systems at Palantir Technologies, Matt also dedicates time as a volunteer high school teacher at Trillium Academy in Seattle, empowering twice exceptional students with generative AI skills. Throughout his career, Matt has been at the forefront of ground-breaking AI initiatives, founding ziggy labs.ai, serving as the chief architect at several startups, and growing engineering teams at companies like OctoML, Apple, xnor.ai, and Google. He's also spent several years in academia as professor of computer science at Harvard University. Today, we'll explore Matt's journey, discuss his current and past roles, and learn about his insights into the ever-evolving field of artificial intelligence. Dr. Welsh, welcome. >> Thanks for having me. >> Of course. Just to start off, um, Alisa, would you like to take it from here? >> That sounds great. Thank you. And thank you again for being with us today. Um, could you share some of your formative experiences or mentors that sparked your interest in computer science and AI? >> Well, yeah. I mean, it all started really at Duke University back at something called the TIP program, which I'm not sure if they still have this, but it was during high school. I was able to go and um, live on campus at Duke and spend a few weeks basically being almost like a college student and just writing code all day. And being able to go to Science and Math after that was also like tremendously formative for me. And at Science and Math we had [snorts] these big computers. I'm sure they're no longer around. The main computer on campus was a single VAX system that everybody on campus used. So, it was basically one computer for all of the campus. This is around 1991-92. And um I had an opportunity to help run that computer for uh the student group. And just the opportunity to get exposed to really what at the time was cutting-edge technology and being able to get involved in a community of people that cared so much about what is possible with computing uh at science and math and just being around other students that were, you know, inspired by the same things. I think that was that was tremendous. That that really launched my entire career, really. >> That sounds amazing. I think you answered our next question, too. >> I understand you spent years in academia, including a professorship at Harvard. How did you navigate the move from academia to working at major tech companies and startups? >> Yeah, well, it was tough. Going going into academia um after grad school, I went to grad school at Berkeley and um immediately went in and became a professor. And um that was a common career path. A lot of people would go straight from grad school to being faculty. And um you know, I never really seriously considered doing anything else. It was It was sort of the safe thing to do. I'd been in academia my whole career up until that point. And so, staying in academia was a very comfortable uh transition to make. I spent about 8 years at Harvard, really enjoyed my time there. Had a fantastic uh set of students and research projects. Um and uh it one when you get tenure at most universities, you go off and you spend uh a year doing a sabbatical somewhere else. And you have a choice of whether to do a sabbatical and where you might do it. I chose to to join Google. Um and my intention was just to stay there for a year and come back and continue teaching and doing research at Harvard. But as soon as I got into industry, I kind of recognized that there's something in me that I was missing. That was the opportunity to build things that lots of other people were using. And as a professor, you don't really get that chance very often. Some people do, but most don't. Um so, being able to be really hands-on in building products and systems that were used by really billions of people uh at a place like Google was just so satisfying for me. That was I found myself much more drawn towards that practical aspect of building things that people use versus the more abstract and maybe theoretical aspect of, you know, thinking deep thoughts and and and coming up with new concepts. I'm much more hands-on and applied uh than I am uh someone who's a who's a great researcher. So, I think it just felt more like a hand-in-glove kind of situation where um I just belonged in that environment and I really enjoyed it. But, it was a big transition. I mean, academia is a place where, you know, you are rewarded for being the smartest person in the room. Academia normalizes people challenging one another and um sometimes getting into like really intense arguments and fights uh over your ideas. And um uh in academia, it's all about you your personality as an individual and how you're rising above others um to be the the the the the focus of attention. Whereas in a industrial setting, uh more often than not, it's not about those things. It's about collaboration. It's about working well with other people. It's about um finding middle ground. It's it it's not about dominating others. And so, making that transition was challenging for me because I'd been trained in the in the area of, you know, how how do you stand up and and dominate other people in the academic sphere moving into a more collaborative setting, you know, that was a bit of a transition for me. I think I figured that out by now. But it's not always easy. >> You've discussed the differences in the spaces of academia and industry, but what elements of academic research did you find most useful when tackling real-world industry challenges during that transition? >> Yeah, I mean I I I do think um academic research and and you know, doing a PhD for example can be tremendously valuable. The way I kind of think about it is being an academic and doing a PhD, it teaches you how to think and it teaches you how to tackle really hard un uncertain problems. So the difference between, you know, someone who has, you know, graduated from college and gone straight into industry versus somebody who went on to grad school and spent time in academia, very often I observe that it's the people that have spent some time in academia that are more comfortable with ambiguity and with problems that are um not always amenable to just, you know, hacking away at them until you come up with a solution. Like you have to stop and think and consider. You have to um run experiments. You have to test your ideas. You have to collect data. Uh you have to analyze that data. And so I you know, beyond the more specific research kind of topics and things that we cover in a university setting, I think the the training you get in terms of how to solve hard problems, that's the critical thing. That's the really important thing to take away. And you don't have to be a professor for eight years to get that kind of training. You don't even really have to do a PhD necessarily, but I think it's good to be exposed to that. >> That's really great. Thank you. So from co-founding Fixie AI to leading teams at Nori AI and Ziggy Labs AI, what inspired you to take on these leadership roles and how have they shaped your view of AI's potential? >> Yeah, so I started working in AI before it was cool. >> [laughter] >> Um and I founded Fixie as one of the very first companies that was trying to do things with large language models. Um this was before ChatGPT came out. And we were building a company to explore what could language models do that other kinds of software could not. So, for example, if you wanted to automate a process, have um uh an AI call out to a website or perform some action on your behalf um as an agent, this is a new idea that was not really possible until large language models came along. And interestingly interestingly, when I started the company uh I I had a very hard time convincing venture capitalists to give us any money. They hadn't heard of language models, they hadn't used them, no one had tried ChatGPT. And then after ChatGPT came out, literally literally, people were pounding down my door trying to give me money for this company. So, it was such a vast change um when ChatGPT came out and popularized this idea. The reason that I wanted to do this in a startup setting was mainly because it wasn't clear that you could easily approach these problems, these kind of cutting-edge technology things in a more conventional setting like, you know, big tech like Google or something like that. Um and the reason really is that um you know, larger companies, they they they move slowly, they have uh inertia, uh and they have their own kind of predispositions in terms of how they build things and how they think about the world and how they think about technology. So, if you really want to branch out you need to do so in an environment that's going to facilitate that. And startups are a great way of doing that. It's not the only way, but it's a great way of doing that. Let's say the other side was, you know, starting a company is just a very challenging endeavor and you know, Atlas, I think you've got some experience with this, but being able to think about what is the company going to be and how are you going to make money and how are you going to hire people and how are you going to, you know, present a vision to the world that other people find compelling. There's a lot of skills that you have to have and there's no way to learn this stuff other than to just try it. And I clearly was not prepared for this when I started started that company. It was something that I'd never done before and it was like drinking from a fire hose for the first, you know, year or more because I just was learning so much every day. But you can't you can't read books. You can't read blogs and and get that kind of experience. You have to you have to do it. So for me personally, it's been about just challenging myself, trying to do something that I didn't know how to do and seeing if I could do it. And I don't think I was that great at it. You know, there's lots of times I've tried new things in my career and found out that it wasn't wasn't the thing I was going to be good at, but you know, it was fun to try. >> Can you describe one of those specific challenges that you faced in that startup environment and how you overcame it? >> Yeah, I mean, you know, I think I think one of the biggest challenges was just figuring out what we were going to do as a company, you know, like when language models first came out, there wasn't a market for them and it wasn't at all clear where they were going to fit into the world of technology. How were people going to pay for them? What were they going to pay for? What was the product? What was the killer app, so to speak? Um and we had many, many, many hypotheses. And one of the biggest challenges that we had as a early stage startup in this space was we had raised enough money that would allow us to go and try too many things at once. So, we ended up raising $17 million, which isn't a huge amount, but it's more than most early stage companies would need to get off the ground. And in retrospect, raising that much money ended up being an a liability because we had so much cash that we felt like, "Well, we can try this, we can try that, we can do this, we can do that." And it was like being a kid in the candy store, right? It's like everything was available. Like we could do almost anything we wanted. And we had all the time in the world because we had all the money in the world that we needed, right? That turned out not to be a great thing. If we had only raised, say, a million dollars or two million dollars or something, that would have focused our attention on one thing at a time because we wouldn't have had the luxury of dabbling with all these different ideas. We would have had to commit and really execute in a short period of time in order to make sure that that money was not going to run out. So, I think the way that evolved over time and what we learned from that experience was, you know, raising too much capital it sounds good on paper, it's not always great in practice. And, you know, trying to hone in on a focus that you can really say, "This is the one thing I'm going to do right now." And the whole team has to be aligned behind that. That is a huge learning from my time working there. >> That's really great. That So many great lessons to learn from that, Dr. Wilks. In addition to your experiences in entrepreneurship, academia, and industry. I also understand that you volunteer at Trillium Academy, where you teach generative AI to twice-exceptional high school. So, what motivated you to start this initiative, and what excites you most about working with young learners? >> So many years ago, we had a group of students that really wanted to learn how to program in C, one of the one of the most popular programming languages at the time, and we didn't have a teacher on campus who could do it, and and fortunately somehow someone found a person at I think NC State who volunteered to come in and and teach a group of us C programming on, you know, in the evenings or in the afternoons, like kind of as a volunteer thing. That was such an amazing experience for me as a student, and my son goes to a school here in Seattle called Trillium Academy. Now, this is a school for what we call twice-exceptional learners, which are people that have both learning differences, you know, often autism, ADHD, dyslexia, things of that nature, and are also incredibly academically gifted. And so, and those two often very very often go together, right? And so, Trillium is kind of a really unique environment because the students there are so incredibly intelligent, but they also don't learn in the kind of more conventional ways. And I wanted to see if I could try to give back, you know, give something back from my experience as a high school student getting that opportunity to have kind of an expert come from outside and teach at the school. And since I'd done a fair bit of teaching at Harvard and elsewhere, I realized, you know, hey, I could I could probably teach a course here that might be of interest to some of these students. And at the time, there was no one there who was teaching any computer science. So, I just volunteered. I said, uh you know, "Hey, how can I help? Uh would it be of interest? Would you all want someone to teach some computer science?" There's a lot of interest. They wanted something. Um and I decided not to teach a conventional programming course, even though that probably would have been easier to do. I said, "Well, look, you know, the world is completely changing with all of these generative AI systems. What would teaching computer science from a gen AI perspective look like? You know, if you were to imagine a brand new way of teaching computing that was focused on gen AI as the foundation of everything, what would that look like?" So, for example, you know, instead of sitting down and writing a bunch of computer code to do something, could we teach the language model how to solve a problem for us, right? So, one of the students, for example, he taught the language model how to play the adventure game Zork, which is one of these classic text-based adventure games. Uh another student worked on how to teach the language model how to play chess. Another student was working on teaching the language model how to generate uh Wikipedia articles, this kind of thing. And it was a really interesting um experience, I I think for me and for the students, because um instead of spending all of our time learning the more kind of old-school ways of writing computer programs, we were really partnering with a language model and having the language model do a lot of that work for us. And that was really fascinating experience, and it's something I hope to repeat in the future. >> You mentioned the ways in which you help the students learn and understand AI, but have the students' fresh perspectives or questions influenced the way that you think about AI development or its broader impact as well? >> Yeah, absolutely. I mean, this is a generation of of uh students that are uh growing up in the era of AI where AI is just expected to be there, you know, and it's something that they leverage all the time. Um not not just in the kind of like computing context. I mean, they're using it to assist with coding and um you know, interacting with ChatGPT all the time to ask questions and get clarifications and coaching on things, but also just broadly like it's just something that exists for them in a way that um it absolutely did not exist for me when I was learning how to how to how to be a computer scientist. Um so, I think though that that perspective of, you know, AI as as like a a deep and extremely powerful resource that is available to all of us, um I think is radically changing the way that we go about things. I mean, I've seen that same transformation happen with when the web and search engines first came along. Like, if you can possibly imagine a world before Google, right? Where if you wanted to go learn about something, you had to go to the library and find a book and open up a book. You know, you couldn't just type in, you know, "Hey, how do you do this thing?" and you have, you know, an article sitting in front of you in in in, you know, 100 milliseconds. We're seeing that same kind of shift happen with AI, and it's going to be interesting to see how the world continues to evolve with this. I know a lot of schools and teachers and students are concerned about, you know, what does it mean when you've got this AI assistant always kind of in your pocket, so to speak, and um you know, how do you know that people are putting in original work, and how do you know that people are doing deep thinking when when in fact AI is such a huge accelerant to that? And I don't know where that's going to net out in the end. I think it's a very hard problem to solve, but um I want to at least see what happens when we embrace that, and and really see what what kind of ways that leveraging AI can maximize human potential. Um and I know that's controversial and we're going to you know, we're going to learn a lot over the next you know, 5 to 10 years about that. >> With such extensive experience spanning from academia, startups, and large tech, what key advice would you give to students and newcomers who want to excel in AI? >> Well, it's hard to give really concrete advice here because the world is going to be so vastly different even in a year or 2 years especially in the AI field itself. Um you know, a lot of people are concerned about well, does AI mean that our jobs are going to go away? Does it mean that majoring in CS doesn't make sense anymore uh because there will be no more programmers left, you know, so understanding how the career trajectories of people are going to unfold uh in the future, that's certainly hard to predict what that's going to look like. Generally speaking, I would say uh it's going to be um really valuable for students to um be aware of what's happening in and to be nimble and prepared, right? Um I think the Boy Scout motto is, you know, be prepared and I think that's what I would recommend here meaning um learn as much as you can and have broad-based skills, right? Um that over-specializing in something really niche um when that niche thing might turn out not to be relevant at all in a few years is is not advised. I think especially when you're young and you're thinking about going into college and then beyond college, what are you going to do? I do think the broad-based background uh is going to be incredibly useful uh for people going forward because it'll be you'll be more prepared for changes that are happening in the world, you know? Um in in my career, I you know, originally I called myself a systems person or a networking person and then I started calling myself a embedded systems person. And so, you know, there's this sort of tendency to want to put labels on yourself and to say I'm an expert in X or Y, and this is what I'm good at. Um I think um it's been really beneficial to be um very broad-based and to try not to have too much of a pigeonholing of what you're able to do, so that as new technologies come in and you're able to find ways of going and learning about those and and leveraging them. Um I think that's been uh something that I found really beneficial for myself, and so I think you know, my key advice would just be, you know, be broad. Be as broad as you can, because I think that's going to benefit everyone going forward. >> That's great, and I think you answered our last question as well. So, thank you so much for meeting with us today. >> Yes, thank you for >> That was a real That was a real pleasure. Thanks for having me. >> [music]