2025 Philip Gamble Memorial Lecture: David Card

Uh Well, thank you very much, Aaron, and um everyone for coming um on a Well, at least it's not raining day. Um great to be here. Uh this is my first time in Amherst and it's a I've had a great visit so far. Got to meet a bunch of the graduate students in the in the department. Um So, today I'm going to give a um a lecture on um cause and effect and evidence-based policies. Uh I have to say um the last few weeks I've started to wonder about the idea of how important evidence-based policies are, but um we'll talk about that at the end of the lecture. Um In the field of economics, um really from the beginning of the field, uh economists have been active in giving policy advice. Um most people who've who've heard the name Adam Smith might not know that the whole argument of The Wealth of Nations was in opposition to the idea of mercantilism, which is the idea that uh trade deficits should be zero. Um the famous economist David Ricardo uh developed the argument that we still use today when we talk about uh free trade that um different countries have different relative advantage in different things. Um of course, not all economists made arguments that we would be quite as happy with today. Uh the founder of the American Economic Association, Francis Walker, um wrote a very uh uh you not really a correct article in the Atlantic Monthly magazine in early 1900s arguing very strongly for restricting immigration. Um Now, interestingly, if you go back and look at what these economists how they were making their arguments, you would notice right away that they were largely based on uh essentially uh theoretical arguments. Uh sets of assumptions. And it's clear that Smith in particular was very aware of all kinds of empirical evidence, but he didn't have the tools to collect data or analyze it or anything like that. So, he was basically trying to propose a set of theories that would be in support of his arguments. In the last 50 years, however, there's been a gradual shift towards understanding the importance of empirical evidence. And many uh policy questions revolve in part, not entirely, but they revolve in part on essentially causal questions. So, a causal question like how does the arrival of a migrant affect the labor market opportunities of natives? And knowing that effect, you could potentially be better situated to give some kind of advice about a policy concerning immigration. So, what I want to do today is I want to start by talking a little bit about the whole idea of scientific evidence. Then I'm going to talk about kind of the gold standard in scientific evidence, which is a randomized trial or experiment. And I'm also going to talk about something that's pretty important to understand the whole idea of causality, which is the idea of counterfactuals. And I hope to leave you with a lasting way to think about that that's issue. Then I'll talk about social experiments. So, in the 1960s, economists started to be involved in experiments conducted on social questions like welfare reform and so on. And then I'll talk a lot about the idea of natural experiments or non-experimental designs. And this is um what is often used by economists to try and answer causal questions when you can't do an experiment. And then I'll talk about applications to two of the topics which Aaron uh, mentioned, um, minimum wages and immigration. And then finally, I'll uh, I'll talk a little bit, particularly in the context of the issue of immigration, about the question, um, so what what the the evidence says one thing that economists have brought, but that doesn't seem to be that important for policy. What's really going on? Uh, and I think I actually we have some understanding of that which every day, um, seems to become more and more relevant. So, let's start with scientific evidence. Uh, so in high school, uh, at least when I was in high school in Canada in the in 1970s, we were taught about the scientific method, the idea that you propose an hypothesis and test it. Um, and the classic example of that, actually I can kind of vaguely remember, this is supposed to be a picture. This is a a well-known um, picture of Boyle, the guy who wrote Boyle's law for gas. And Mr. Boyle had a side gig on um, showing how important gas was or air was to life cuz it wasn't fully understood. And the way he did it was he invented the vacuum pump. And he had a glass jar and he had a dove in the glass jar and in front of large audience all across Europe, he would suck the air out of the uh, uh, jar in the bell and the dove would die. And that pretty much proved to the audience that air was crucial to life. It wasn't really something that was well understood. And I I bring this up as an example of something because the way that uh, science science is often taught to students, there's sort of this idea that if you have an hypothesis, there's going to be a single decisive measurement which can prove right away whether it's right or wrong. And there's a really famous example of that which was extremely important uh, in the 20th century, and that was just after Einstein's relativity theory was published, a bunch of scientists realized that Einstein had proposed himself that one way to test that theory was to look at the bending of light when Mercury flew in front of the sun. And it turned out when they set up the equipment to do that measurement just a few years after Einstein's work was first published, it turned out that basically his theory predicted the amount of bending that they were able to detect fairly precisely. And that single decisive measurement actually led to his instant fame. There were headlines all across the world different magazine newspapers and so on. Now, the problem in a lot of areas and one that probably you've thought about a little bit would be health. But this is also true in economics is that um it's almost impossible to devise a single decisive test for most of the hypotheses because there's so much variation in what's going on. Okay? It's certainly the case that, you know, if your head's chopped off, you die. But it's less clear if you get a COVID vaccination, are you prevented or not prevented from getting COVID or are you prevented or not prevent prevented from getting sick? And that fundamental confusion arising from the absence of of a single decisive measurement is extremely important in understanding how the rhetoric around these arguments can get kind of muddy. Now, if outcomes vary a lot for other reasons, for instance, health is very different for different people, one way to think about this and it was understood by the middle of the 18th century would be to think of treating a large number of people. And a classic example of this, uh who figured out uh about the smallpox vaccination, what they did was they took a bunch of They realized by observation, it was probably known from Arab times, that um the women who milk cows never got smallpox. And somebody figured out that there would be a way to transfer from the um the pustules on there on the infected person with the cowpox, transfer that to someone else and then see if those people didn't get smallpox. And this picture here, what's going on is that this uh dude with the hair is supposedly trying to uh look at her pustules and extract some pus from it, which he's going to use to inoculate someone else. Uh And now, even with that idea, treating a large number of people, there was still a lot of uncertainty. And um uh it was really until the beginning of the 20th century that this was kind of resolved, how to deal with that. Uh so, for instance, in the 1870s, they established a big agricultural research farm in um England, and this agricultural farm had been planting different varieties of barley and uh wheat and corn uh or not corn, um oats, excuse me. They didn't have corn. Um all around this field the fields of this plant. And uh they they had this sort of uncanny situation where one year it would look like this variety of barley did well, another year it would look like another one did well. And so, it after uh something like 35 or 40 years of running these so-called experiments, they really hadn't gotten anywhere. And in the early '20s, they brought in an Australian named Fisher, Ronald Fisher, and he straightened it out. And what he did was he figured out that if you want to have an experiment with large variability in outcomes, you need to have an organized control group. And so, what he proposed was to divide a field into groups, and this is a picture of the experimental station, and you can see it's a 5 by 5 design, and different varieties of the same crop were planted in different slots in that 5 by 5 grid. And within a few years, this seems like an incredibly straightforward idea, but it was revolutionary at the time. Within a few years, people started to realize if we wanted to do an experiment with these really variable outcomes, we needed to have not just a sort of a a simple comparison group, but we need to have a a an organized pattern of comparisons between the different treated groups. And so, Fisher really uh straightened things out um pretty substantially. Now, to understand um the things that emerged out of Fisher's work, it's helpful to go back to the very simplest kind of experiment where there's only a single treatment and a a set of patients or set of fields that are going to be what we call the control group. And uh the idea of a a modern experiment, which was basically invented by Fisher, was to divide who was in what group randomly. Okay? So, you take a population of patients that all have a certain disease, you manage to convince them all to be part of the study. You interview them, and then you toss a dice, and the people with the even numbers of spots are in the treatment group, and the odd numbers are in the control group. And ideally, in these kind of experiments, the subjects don't know whether they're treated or not, because it's possible if you think you're being treated, you can get better. That's the idea of placebos, and so they invented this idea of placebo pills. And also, it's helpful if the people running the experiment don't know who's treated or not, because sometimes they have incentives to say that the treatment works. And so, you don't want them fiddling around uh with who gets what. Uh and so, after that's done, you then let the experiment run for a while, and all you do is you compare the average outcomes for the treatment group and the control group. And I think most people uh are aware of this. I I don't know whether they actually teach it in high school, but um you would hope that most people are at least informally aware of that's how it works. Now, why does that work? The reason why it works is because under random assignment, there's nothing different between the average people in the control group and the average people in the treatment group. And so, if after treatment, they have different outcomes, you would say it's likely that the reason why the treatment group is healthier, for instance, is because of the treatment. Otherwise, these two groups would have been the same. And uh the way that that's spoken of these days in economics and other fields is that the control group provides a counterfactual. Now, uh if you're trying to explain this to people of my generation, the way you would say it uh I'm not sure that you know, many of the younger people here might not have seen every single episode of Star Trek, the first version, but um many many episodes of Star Trek revolve around the idea of the um mirror universe, where there's like a good Spock and the bad Spock and the good Kirk and the bad Kirk. And those the obviously the writers of um Star Trek were fascinated with the idea of counterfactuals, because we you know, in the in the bad world, like somebody made a decision a thousand years ago, and they've evolved into Nazis. And in the good world, we've have into the great people of uh you know, the Star Trek universe. So, a lot of policy debates arise because we don't have the counterfactual. And being a microeconomist, I love to make fun of macroeconomists and you could say that really the fundamental problem in macroeconomics is you never see the counterfactual. You can't create this mirror universe to see what would have happened in the absence of treatment. And so, people can debate oftentimes endlessly about questions because there isn't really a clear-cut situation where you can say, well, if we hadn't done this to the monetary policy, we would or we wouldn't have had a recession or inflation. So, these ideas of experimentation were used in agriculture. They got Then people started to realize that you could use them in health. And probably one of the most important examples of that was in the early 1950s when the March of Dimes charity basically put the money together to try out Salk's new polio vaccine. And in 1954, they did an experiment with 1.8 million children. And they basically would set up This is a line in front of a school. The children are going to be vaccinated. And I was born just a little bit after this, but my parents remembered This This part of the experiment was done in Canada where I grew up. They remembered this exact scene. And the way it worked was the nurse had rows of vials and every second vial had the Salk vaccine and the other one had just saline. And people The nurse didn't know who she was inoculating. The people didn't know. They got a number that they were attaching to themselves and ultimately were assigned to treatment or control status. They needed this really large sample for two reasons. One is because the incidence of polio, even though it's a terrible disease, um is very low. It's only 50 out of 100,000. So, if you unless you have a an experiment with many hundreds of thousands of cases, you're not expecting to find very many polio cases. So, your treatment group could be totally healthy, and so could the control group, unless the control group is so large that there's a very good chance that they're going to have at least some cases of polio. And the other reason why they did it this way is because polio is one of these weird diseases that shows up in pockets. And you have to have It'll be very intense in one part of the country and not very intense in others. Right now, we've got this measles epidemic going on in Texas, which is very much like that. So, you're going to have like big outbreak of measles, and you're going to see over the next year that's going to spread. There's going to be places where there's suddenly a measles outbreak. Uh unfortunately, because of the doesn't look like there's much support for vaccinating anybody for measles. Um okay. So, now onto the subject of social experiments. So, in the mid-1960s, when um the economics department at UMass got going, uh there was much enthusiasm for all kinds of experimentation on the social dimension, uh uh kind of contributing to this overall um belief that we could make country better place. Kind of hard to imagine now, but that's the way people thought. And uh in 1966, a young female PhD student at MIT named Heather Ross, said, well, there was a debate about how we should organize uh welfare system. Uh and actually, there was some of agreement between uh both conservatives represented by Milton Friedman and liberals represented by James Tobin at Yale that one way that they could kind of agree would be have something that they called a negative income tax, which was the idea that if you had no income, you got a certain amount from the government. The more you got yourself, the less the grant you got from the government became and eventually you moved out of any grant at all into being self-sufficient. And Heather Ross proposed to HHS, she literally wrote a letter to the secretary of HHS and said, "Why don't we do an experiment?" That's kind of hard to imagine if you're a grad student today. I don't think this is going to happen. But um they did it. Uh and eventually four of these experiments were uh conducted around the United States. The first one was in New Jersey and um the thesis advisor of my thesis advisor was the guy who organized it. And um my thesis advisor was kind of the research assistant on the project. So, this is sort of a topic that's near and dear to my heart because it's sort of a generation away from my work. Um a parallel experiment was conducted in Manitoba uh among the Canadians. So, this was an another example of how these ideas were spreading around the world. Um now, by the '70s um enthusiasm for all this Great Society thing had kind of faded out. This was the disco era. Uh this is the era of polyester suits and all that stuff. And grand visionary social vision had really faded into oblivion. Um people realized also at that time that there were some problems with the original negative income tax experiments. They were In a way, they were very beautifully constructed that they're overly complicated and people who did them didn't anticipate some of the problems that we now, when we run experiments, understand. So, what happened then in the 1980s is by the time I got to graduate school, there were some a new generation of social experiments, but they were very, very very They had just a single treatment group, a single control group. And they were focused on a very narrow kind of question, like what would happen if we took people who had uh drug addiction problems and gave them some kind of training. And those those kind of experiments um set the stage for the widespread use of RCTs in economics more generally. Uh and this is a picture of Esther Duflo and Abhijit Banerjee, uh two development economists who won the Nobel Prize a couple years ago for their uh very important work in spreading the use of RCTs and in evaluating anti-poverty programs around the world. And they created uh a huge infrastructure for doing that, called J-PAL. So, this idea of doing RCTs now is in fact very central to a good part of economics and labor economics as well as uh development economics and other fields. Helping to provide scientific evidence. Now, uh RCTs uh can estimate the counterfactual because with the control group, you can see what the treatment group would have done if you hadn't given them the treatment. But there are some problems. One problem that arises in a lot of settings is you can't really be sure when you first run the experiment that the result that you get in the experiment can be scaled up. Uh and this happens in health quite a bit because when they do a health experiment, often times the treatment group is part of a survey, a closely surveilled group of of patients who are receiving regular care, who are every time they visit the doctor uh over the sequence of the experiment, maybe for a year, are receiving other treatments and sort of watched and carefully monitored. And then what happens if you ultimately try and give the same drug that worked really well in the experiment to patients in the field, a good chunk of patients will take the pills for a few weeks and then kind of forget. Right? A huge fraction of medicines are sitting in the cabinets at home. People didn't finish their their courses of antibiotics or they started taking high blood pressure meds and then didn't like the side effects. And so a huge chunk of this efficacy of these drugs is actually mitigated by bad behavior of the people who are being treated. Uh and then similarly or probably more importantly, there's many kinds of things that can't be randomized very easily. So it's pretty hard to imagine randomizing um something like a minimum wage or randomizing something like testing for the effects of immigration. Uh and so um in the 1980s, people started to think, well, how could we do something closer to an experiment where we get closer to having a counterfactual even if we can't do an experiment. And people that were involved with this realized that there and this was an idea that had been recognized actually through the 20th century. So there were some early studies all through the 20th century of things like minimum wages. And it had actually been recognized um in the uh 19th century by a doctor who was interested in the causes of cholera and noticed people may be aware of this noticed that there was a source of water a situation where there were two sources of water in one block of London. One of the sources was upstream further and one of the sources was downstream further and this this guy eventually noticed that if you got the water from the downstream source, you were much more likely to get cholera and and eventually used that evidence to write a paper saying that cholera was probably caused by organisms in the water. At this time, of course, this is before the germ theory of disease, so people didn't fully understand how something would be transmitted through the water, but he his results basically suggested that. And if you're in London, you can go to the place where there's the water spigot is coming out of the pavement and there's a statue of him. So, if you want to see the very earliest kind of natural experiment you would know. But, there's other examples that and I'm going to talk about one of them right in a few minutes, which is about for instance if a if a single state, for instance the state of New Jersey was to raise its minimum wage and nearby states didn't, you might be able to use nearby states as the counterfactual. Uh another example I'll talk about briefly um in the 1980s there was a violent separatist movement in Spain in the Basque region and it was so uh disruptive that people were convinced that it probably was causing um economic uh disturbance. People were unwilling to invest in the Basque country. Um big companies were moving away. Uh it was kind of messing up the entire economy. And so, you might be able to say, "Well, what if we could compare the Basque region to some other parts of Spain?" So, these ideas uh have gradually become kind of regularized in economics um and are sometimes referred to as natural experiments. So, that's a situation where you have something like an RCT in that there's both a treatment group group who is exposed to some new policy or some kind of treatment and a control group who are not. For example, we want to know the effect of a jobs program for long-term unemployed, maybe they introduce this program in one city and not in another city. We might be able to use the other city as a control group. The problem with this is that you're never 100% sure whether the proposed group that's the comparison group actually is providing a valid counterfactual. Okay, so you would never really know 100% for sure that what happens in the comparison group is what would have happened in the treatment group if they hadn't been treated. And so that is kind of a fundamental problem with non-experimental inference. In a randomized assignment setting the fact that the treatment group and the control group are randomly selected means that they should have been the same if it wasn't for treatment. So you have automatically the validity of the counterfactual. But in a lot of non-experimental settings uh it's more of a question of debate or uh trying to find evidence. Now a lot of times treatment and control groups differ in fundamental ways. For example, in the case of this New Jersey and uh nearby states comparison I'm going to show you some evidence in a minute. Stores in uh fast food restaurants in New Jersey were a little bit smaller on average, a fewer employees, than the ones in nearby states. So you might want to take that into account and say, "Well, even with the absence of treatment, things were different." And the way that people realize this could be done would be to have data from before and after the treatment. So what you do is you say, "Okay before this treatment the treatment group and the control group have some differences and I'm now going to assume that those differences would persist at exactly the same rate in the absence of the intervention. Then after the intervention occurs we re-measure the differences and see is it bigger or smaller. And if the difference between the treatment group and the control group has gotten bigger, then we would say that the treatment raised the difference. Okay? Or if it got smaller, we'd say it closed it. And what you're doing there is you're taking the difference between the treatment group and the control group in one period and subtracting it from the other one. So, it's a difference of differences. And I had the great uh uh opportunity or luck or fortune uh to be working on a paper in 1982 or '83 uh I guess it was probably January or February '83 with my colleague Orley Ashenfelter and we were working on a trying to evaluate training programs and it was the middle of the night. The paper was due the next morning and I still remember Orley was writing the paper and he said, "I got it. We're going to call this thing a difference of differences." And so, I was present the day that the economist first started using the term difference of differences. Uh I don't know if I've looked before it's did anybody think of that it's kind of an obvious thing, but we're going to claim credit for it for that for now. Okay. So, how does this work? Well, I'm going to give as a first example the effect of the Mariel boatlift. Um and I uh have a poster for the movie Scarface par- partially because it's kind of an entertaining movie, a little bloodthirsty, but entertaining. But, uh more importantly because the first few minutes of Scarface, if you ever watch the movie again, have actual video of the Mariel boatlift from Miami side. Uh so, what happened in March 1980, there was a demonstration in Havana and Fidel Castro was kind of unsure what to do. It was a bunch of Cubans had gone to the um Peruvian embassy and uh they wouldn't leave the grounds of the embassy. So, he eventually ha- figured out an exit strategy. And what he said was, "Okay, an- if anybody wants to leave my uh Havana or Cuba in general, that's fine. All they have to do is go to the port of Mariel." Now, he wasn't really thinking exactly how this was going to play out, but there were many many many Cubans in Miami, which is only uh a few hours boat trip. Well, about an 8-hour boat trip from Mariel. And uh so over the course of a couple of months, 125,000 people left Cuba and went to uh Miami. And about something about close to 3/4 of them ended up staying in Miami. So over the course of a couple of months, you had something like a 7% increase in the labor force of the um Miami uh urban market. And um I was uh teaching at Princeton and one of my undergraduate students, uh a man named Constantine Alexandrakis, said would came into my office at that time Princeton students, they still do, had to write a senior thesis and he said, "I want to write about the Mariel boatlift." I didn't know anything about it. Uh he introduced it to me. And he did a a kind of a not particularly successful paper because he couldn't pull the data together in time, but it made me think that I could try and do something with a comparison group. Ultimately, what I did was I said, "I'm going to play around with combinations of other cities which on average, when I used these four, ended up kind of looking like Miami." And so I used a combination of Tampa, Atlanta, Houston, and Los Angeles. And uh just to show how this works, here's some data. Uh this is the um since I'm an economist, people who are taking economics will know about this. Um I have to use a logarithm. So this is a logarithm of the weekly wage, okay? Uh and it's the real weekly wage, and I'm using data from 1973 to 1991. Okay? And each point is the real wage in a different year. And this is the year when the boatlift occurs, right here. And you can see if you looked at that figure and you didn't know anything else, you might say, "Well, actually what happened was wages were going down." This is for um low-skilled workers, people with uh high school degree or less in Miami. And it looks like they were going down kind of all through this period, but especially after the boat lift, they maybe took a bigger dive. Okay? And it's a little bit hard to know exactly how much of that is the boat lift and how much it is isn't. And here is what happens when you compare it with the counterfactual, which these days everyone calls the synthetic control group because it's a combination of other cities. Uh and you can see how helpful this is. Because now we can sort of say, "Well, actually there was a kind of a downward trend everywhere." And even when we look a few years later, it doesn't look like 1983 or '85 that there were so big of differences, but there is some variability. So, there's room for uh you know, argument and debate. But on average, I would say if you thought of the difference of differences between 1983 and 1977 for instance, you'd say not much happened. Okay? And what this illustrates is the really fundamental role of this comparison group. There's so many situations where something happens and somebody looks at on the ground what happened and they'd say, "Oh, it must be this." But they're not thinking what was the counterfactual. Okay? And it's really important that counterfactual because there's so many situations where there's something underlying that's going to get confounded, uh mixed up with the true treatment. Um now in this particular case, this was a very small paper. It was published in a very um obscure labor economics journal. Um and it I have to say at the time I wrote the paper, I thought, "Well, this is kind of interesting, but I didn't think much of it." Um but it was followed up by a bunch of other studies over the over the next 20 or 25 years. And strangely enough, they all kind of reached the same conclusion, which is when you have a mass migration, so there was a one a repatriation of French people from Algeria back to France, a similar repatriation of Portuguese nationals from Angola back to Portugal. Uh there's a massive migration of Russian Jews to Israel, a huge one one of the largest uh movements of population uh for uh you know, until very recent times. And a similar migration when the at the end of the Soviet Union when ethnic Germans were allowed to move back to Germany. And all of those studies showed very small effects. Sometimes a negative effect, but most of the time not much of an effect. So, that's led people to think that even large changes in the population don't necessarily have huge effects on native outcomes. Now, just to show that the way this can be used in a slightly different setting, this is a study by these two guys now um Abadie I know how to say his name. The other guy, they're both Basque. But if you know anything about Basques, you know, you don't want to try and pronounce their names. So, they did this a study. They're comparing the Basque region to an average of Catalonia and Madrid. Okay? Catalonia is where uh Barcelona is and uh the Madrid region. And they they averaged them. And what they do, they've got the the the the troubles in the Basque region start in '76 or '77. And you can see this is GNP. You can see that there's basically a fairly consistent evidence that relative to the counterfactual of Catalonia and Madrid, there was about a 10% loss in real GDP per capita, which is a fairly significant cost. I mean, uh uh of this uh disruption. Now, there's something interesting about this uh that's very important, reason why I bring it up. You might ask yourself, well, how did they get 85 and 15? Where did that come from? Well, this was Abadie's idea. This is a brilliant idea. I have to I always give him credit because it's such a great idea. What he did was they played around with the 85/15 ratio until in the previous period from 55 to 75, the two lines were closest together. Okay? And this is a technique that economists use all the time. I do some consulting at Amazon, um, which I have to say is not everyone's favorite company, including mine. But, uh, this technique is used all the time to evaluate, um, different proposals for how to deliver things at Amazon or whatever. So, this idea of he it's now called a synthetic control group because it isn't really an average of any particular thing. It's a particular weighted average that gives the best evidence in the past of mimicking, uh, the treatment group. And so, the fact that these track each other so well here gives you a little bit more confidence that if there hadn't been this terrorism event, they would have continued to track each other. Now, of course, you don't know that for sure, but that's that's the argument. Okay, a second example we mentioned a a couple of times is New Jersey minimum wage. Early 1992, I was teaching at Princeton. Um, my colleague Alan Krueger was in the office next door. Uh, and we learned about the plan in the New New Jersey legislature to raise the minimum wage in the state from 4.25 to 5.05, um, which sounds like a small amount of money. Um, but it, you know, if you look at it, that's like 20 some percent. So, it's a fairly large increase. Um, and what we tried to decided to do was we were going to construct a prospective difference in difference analysis. So, we were going to interview the stores before the minimum wage went up in these two states. And then we were going to go back and interview them after. Uh and we were very uh influenced by the idea of an RCT, so we wanted to find do everything as closely as we could like you would do if it was a real RCT. So, we managed to get um uh very, very similar uh surveys conducted at all the state all the uh restaurants and so on. Uh now, there was a problem with this design. We only had one observation prior to the rise in the minimum wage. Okay? So, we didn't have the ability as Abowd had in this graph to say, "Okay, in the years before, New Jersey and Pennsylvania have been moving together." So, we weren't as confident that our comparison group was going to be necessarily as good. But, we had an idea and it came about because we were living in Princeton, and Princeton is kind of a high-wage part of New Jersey, and actually in New Jersey, everybody was already paying $5 an hour. All the fast-food restaurants. So, we kind of knew that there were at least some restaurants in New Jersey that weren't going to be affected. So, then we realized when we started this design that we might have two comparison groups. We might have Pennsylvania and we might have high-wage stores in New Jersey. And so, that gave us a chance to have a little bit more confidence in what was going on. Uh and so, this is what we found. So, this is the number of uh full-time workers in each of the fast-food restaurants um before and after Pennsylvania's decline. Now, before is um February, and after is October and November. And Pennsylvania is declining and New Jersey is flat. Now, this is the exact opposite of what you were expecting. You were expecting New Jersey to fall if you were, you know, took economics 101. You would expect New Jersey to fall and Pennsylvania to be flat. And so, that's a bit disconcerting, potentially. Uh but, what about these high-wage stores? Well, they only had to raise wages by a couple of cents to meet the new law. And this is what we see when we look at them. Now, we've got Pennsylvania, which is going down. We've got the high-wage stores in New Jersey, which start a little bit smaller than Pennsylvania, but they move in parallel. Okay? And the only one that's moving in the opposite direction is the low-wage stores, which had to raise their wage. So, we concluded from that in our paper that it didn't seem very likely that the minimum wage caused employment to fall. It might be that the minimum wage caused employment to go up. If anything, the data pointed in that direction. Uh which was pretty controversial. Um as Aaron sort of briefly mentioned, without mentioning some of the swear words. Um okay. Uh now, again, um this was a a very low-key uh paper experiment. It only had like 500 observations. It was done kind of quickly. There were some flaws in our design. And I think if that was the only study that had been done, that would be kind of the end of it. The way economics works these days is somebody does a study, it looks interesting, somebody else will try it. And so, there's been many, many uh very good papers that have followed up on our paper. Often times, combining information for dozens and dozens of state-specific minimum wage increases with multiple compar- comparison groups. And And so, these studies are much, much better than ours. But luckily for us, uh they basically find uh on average very, very small effects of minimum wages on employment. Big effect on wages, small effect on employment. So, earnings of the low-wage workers have definitely gone up. The low-wage working population is arguably much better off as a result of this, without seeming to bankrupt the business. There are a few people um whose names we won't mention who sometimes manage to find negative effects, but uh you know, usually that seems to involve a a lot of data fishing. So, now I want to now let's this is the last thing I want to talk about which is Okay, we got these evidence things like think about the case of immigration. I said to you that all of the studies that have been done almost always find that when you have a big inflow of of workers, you don't see wages of migrants or excuse me, wages of native workers being destroyed. You don't see big rises in unemployment. You don't see the kind of scare stories being verified that people often raise. Nevertheless, in the last 20 years, it's really hard to point to any country that has done anything in terms of liberalizing immigration. In fact, over the last 5 to 10 years, almost all countries in the world, including many of the more high immigrant countries like Australia and Canada and New Zealand, have cut back on immigration. And the US, which historically accounted for the most largest number of immigrants coming in, as you know, everybody kind of knows, it's gone completely in the opposite direction. So, what's going on? Does evidence not matter? Well, some years ago, um this was even before President Trump's first term, um some years ago, um Ian Preston and Christian Dustmann and I got the idea of trying to find um or evaluate what it was that people were thinking about when you ask them about immigration policy. And so, what we did was we proposed a set of questions to the European Social Survey. This is a survey that's done in 21 or 22 European countries. And we were thinking of the following idea. The idea was, what if people they know about the economic effects of immigration, they might even think those are positive, but that's not what they're concerned about. What they're concerned about is whether their daughter is going to have a date with one of those foreigners. That's way my father would have said it. Um, so what we called it in the paper because we couldn't think of a euphemism, we called it compositional concerns. So, I'm concerned about the race, ethnicity, religion, language of the immigrant group. And if those characteristics are different than me, I'm going to be threatened by the immigrants. This is an idea that's very central to sociologists and political scientists perspective on immigration. That that threat of immigrants who are of different language or religion or or ethnicity is really important. Now, this is a this slide is a uh I actually prepared this lecture 3-4 months ago. Um, at that time there was a poll, I think it was done in November, 29% of Americans say that immigrants are invading our country and replacing our cultural and ethnic background. Now, uh today, you know, you don't even have to do a poll. Everybody kind of knows a lot of people think that. Including many people who are in power. So, it's very clear that that that these compositional concerns matter. So, what we did was we interviewed people and we asked them uh sets of questions. The first set of questions were about economic things, like do you think immigrants lower wages? Do you think they fill jobs that natives won't fill? Do you think they help or hurt public finances? And then we asked them the other questions um about do you think it's better for your country if everybody shares the same religion? How do you feel about having people of different cultures, of different languages, um or different ethnicities? And we put those two sets of questions together. We averaged the ones on immigrant economic concerns, we averaged the one on the kind of compositional concerns. And then we related those two components to how people said they thought should there be more or less immigrants allowed into our country. And what we found was the people who are opposed to more immigrants, 80% of the source of their concern is the compositional issues. Economics is there, but it's not very important. Most of it is this kind of um old-fashioned essentially uh you know um tribalistic. I would call it tribalistic view about how a country should be run. We want the We want the country to retain be remain in our tribe. Our tribe defined however you want to think about it. So, uh we concluded that probably the economic concerns about immigration, which have been dominating the economics literature for decades, were probably like second order or maybe even third order. So, minimum wages, you could make an argument that there's a bit and bit more uh impact of the evidence. Um not so much in the United States, but uh in 1998, the UK introduced its first minimum wage. Uh Germany introduced its first minimum wage in 2015. That was interesting because at that time the head of the German Economic Council was one of my former PhD students who argued strongly against it. Uh and uh last year the EU introduced a target for member countries of a of a minimum wage of at least 60%. Many EU countries don't have a minimum wage like um Italy, for instance, doesn't have a minimum wage. Um and in the case of the UK and and Germany, very careful evaluations done by, you know, with a massive amounts of data, really great uh analyses basically showed exactly what we found in New Jersey the first time, which is surprisingly there wasn't such a big effect. In fact, if anything, wages could go up, people would move to better jobs, and it looked like the low-wage labor force was made uh better off. So, I'm not saying that all of these changes were driven by evidence, but I think the evidence might have helped a little bit. Not to my former friend uh former uh grad student, but to other people. So, to summarize, this is my last slide. Um we can use experiments and actual experiments to answer difficult causal questions, and it's all about the counterfactual. So, remember Bad Spock, okay? From now on, when you think about RCTs, you say, "What we're trying to do is we're trying to figure out what would Bad Spock do in this situation?" Uh methods like difference-in-differences are widely used by academic researchers and policy analysts. They're also widely used in business. So, a lot of economists these days work in businesses, and um a lot of times they'll use RCTs if they can, but if they can't, they'll rely on synthetic control or these other kind of methods. And I think it's really important for economists to realize basically people university of any uh persuasion to realize evidence is something, but evidence is not everything. Uh and depending on the regime, evidence can be almost nothing. So, I think that many governments do not care about scientific evidence and seem not to care very much about science. Uh hopefully, we'll be back on track in the future, and uh you know, we can get to a more rational world where some of these uh sources of evidence actually do matter. But, I'm afraid for now, it's uh somewhat dark times. But, I I think that we have the tools, and we know what we're doing. We just need the chance to influence things a bit better. So, thank you very much. And I I guess now I take questions. Is that what happens? Oh, are you going to moderate it, Patrick? Yeah, okay. Okay, so people that are here some of my class in the common room will be in the back and I'll be handing out the assignments. Okay? And um I wanted to mention something about this assignment. Uh this assignment my understanding is it accounts for 2% of your grade. Uh and I I I think you should mention to the professor the famous line from Woody Allen that 90% of life IS SHOWING UP. OKAY, WE HAVE MICROPHONES IN THE FRONT, so we ask that those of you have a questions please line up at the microphone. Okay, here we go. All right, David. Thank you so much for this very interesting talk. Um I wanted to ask you something uh from uh the last slide before this where you uh said that uh many European countries are latecomers to the minimum wage Yes. movement and some countries like Italy never instituted the minimum wage. Right. On the other hand uh the minimum wage movement the minimum wage was introduced uh many decades ago in the United States. So, what do you think explains the difference in approach towards the minimum wage in Europe versus the US? Do you think it has something to do with the um social safety net the pres a stronger social safety net in Europe versus the US or some other reason? Um it's possible that's the most important factor might be um uh high presence of uh union uh collective bargaining agreements. So, Italy um for instance, virtually everybody's covered by collective bargaining agreement. And the the the sectoral agreements that they have, they're not like the agreements in the United States. What they do is they set a minimum wage for occupational groups if you're in a certain agreement. They don't set the maximum wage. So, all they do is set the minimum. So, basically in Italy, this is also true in Portugal, this is true in Austria, um and Spain, almost everybody is covered by some kind of a minimum. In France, almost everybody's covered by some kind of minimum from these collective bargains. Now, the collective bargains in some of these countries were not, you know, Portugal and Spain were run by fascists in the '40s. Well, quite a bit longer than that. Uh but, they still had some collective bargaining system. And so, it seems like I think with that system in place, and the Swedes, for instance, they also talk like this. They believe that the collective bargaining agreements play the role, and it's a little more flexible than a national minimum. So, that's possible. Okay. Next. Okay. Yeah. Thanks for the talk. So, uh I have a somewhat like I I guess some of your talk was about almost of your talk was about the evolution of economics as a discipline. So, I have a question about that. Um the turn towards sort of scientific evidence and causal methods using natural experiments, I think prima facie one of the promises it has is to reduce the role of ideological bias in how you build your theories and all because, you know, a case could be made that this is just the data and this is just what the data says. Uh but, two things make me wonder about uh whether we should sort of attenuate that view. One is that the empirical methods we use have become like there are so many different methods to choose from, and they might be getting more and more complex. So, even though you might feel that a certain empirical exercise is just pointing you towards where the data is, it just becomes harder to find the ideology behind it. And one thing I particularly think about and I'm not saying ideology in a pejorative sense that someone else has. I'm sure I have it and a lot of people in this room have it. But, one thing I've often wondered is probably where ideology might manifest in empirical work is where you decide your stopping point is. So, for example, if I'm inclined to believe that immigration does not affect native wages. And if I see a paper that passes the minimum threshold of being a decent paper and finds that result, I'll be like, "Yeah, you know, the evidence actually shows that immigration does not affect native wages." But, if I find a similar paper that does not show that, I might then try to find like, "Oh, you know, I could actually be clustering differently or I could be finding fixed effects or I could be using this new method that I saw on Twitter somewhere." So, what do you think about like, you know, do you Yeah, what's your thought on like, you know, that? my first thought is stay away from Twitter. Okay. Um but um I think it is it is interesting. The in principle, this is what the entire field of meta-analysis is meant to do. And um I think done correctly, uh a meta-analysis can be somewhat informative about that because when you're doing a meta meta-analysis is a summary of the studies in a subject. Uh and you're trying to put together sometimes 50, 100 studies. And um in principle, what you're looking for is not just the the published studies, but also the unpublished studies, the ones that people tried and couldn't get published because often those are the most disruptive. Uh and they might be the ones that would be quite important. And one of the things that you do when you do a meta-analysis is you look at um is this a result that um you know, is very far out there, but coming from a very small sample, cuz that can happen. And if all the far out results come from small samples and all the kind of zero results come from large samples, then you would say, probably this is a real zero, right? Uh so I I think the meta-analysis puts a lot of discipline on it. I mean, the other thing that goes on in economics, which is a little different than some other areas, we don't make any money from our studies. Right? It's not like drugs. And it economics the way it works in economics, if you had um tomorrow a really good study that showed that the minimum wage killed jobs, there's a lot of journals that want to publish that, because we love to trash the past giants, right? That's what That's our That's a good thing about our field. So, if you could show a a really good design that was quite dis- We like disruption. And and I think probably we, you know, we want to make sure that our younger people keep that spirit of saying, okay, these guys that thought they had it all figured out, but they were all screwed up. It's completely wrong. Because I think that keeps everything is constantly being retested and relitigated. So, a combination of no real strong incentive one way or the other, it's and then people can get uh have a career by finding a result that's somewhat unusual, um then I think I think that's helpful. Yeah. Hi. Uh thanks for a very nice uh talk. Um so, my my question is about the thing that you talked about at the very end about evidence-based uh policy. And my question is really whether, you know, um the idea of evidence-based policy isn't somewhat misconceived. And and what I mean by by that is that, um you You people, you know, whenever you're you're thinking about um a policy question, there's really two things. There's how does the world work, right? And then what the effects of a policy would be. And then there's sort of what do people care about and what are their goals, and uh so on. So, it might be that people are misinformed about policy and they think that about evidence and they think the world works in a different way than what you're saying and in that regard they don't respect evidence and they don't respect science. But another thing is that they just might care about something altogether different. And in your example, actually, you know, it seemed that the way you characterized it at least is not necessarily that people were misinformed, but that they cared about something different when it came to immigration. So, I wouldn't think about that as a If that's really what's at heart of the disagreement, I wouldn't think of that as a situation where people don't care about evidence or necessarily that people are being irrational, but rather that they just simply have different goals. And I would think that more generally you would really want to think about evidence-informed policy because the evidence of how things work is only an input and then you always have to look to people's values or what they care about. Right. No, actually, I I totally agree with that. That's That was what I was hoping to convey. I apparently I failed. That was what I was hoping to convey with that section was probably like fixation on these employment costs of of immigration or measuring like how much does it hurt or help the federal deficit is not the first-order thing. Mhm. And and you know, we we could do more in economics to study these other effects or we could um collaborate with our sociologists and anthropology friends who know more about some of those areas. Um and and it's totally true that I think, you know, you you if you asked uh you know somebody running the current immigration policies right now they they're probably some of them at least are probably aware that there's going to be significant economic costs of closing down immigration because there's going to be no workers for lots of industries. But they're going to say well I don't care about that because I'm willing to give that up in return for um getting you know a bit more tribal control. Right. Yeah, I definitely think that. How's it going? Um so you spoke earlier to Closer to the mic. Okay. Um you spoke earlier to some of the naysayers that you experienced um when talking about the minimum wage and I'm sure some of them probably uh claimed that uh raising the minimum wage had would have an impact on prices and that would then impact especially like low-income families. Um and that was kind of their rationalization for not implementing something like that. What would be your counter argument to that? Uh well I think the evidence is pretty good on that now that the minimum wage is probably do cause some rise in prices. The The rise is relatively small because like if you take hamburgers okay? Um the amount of labor in a hamburger is relatively small. So the the the portion of hamburger costs that you can allocate to labor might be 25 or 30%. So even if you raise wages 20% it's only going to raise the cost of hamburger 6%. And that's a that's a kind of an extreme rise in the minimum wage by our standards. So first of all you know people eat lots of other do other things besides buy hamburgers. So if even if they put all of their effort all of their money into things what were provided by like hamburgers they still they would the the low-wage worker they would be worse off and other workers other people are worse off and in a sense they have to pay for it. But of course when you raise the minimum wage probably somebody has to pay for it. Profits have to go fall in a pure monopoly model or prices have to go up in other kinds of models cuz the the money has to come from somewhere. You know, like there's no way any it can't come from somewhere. So, it's very important I think for economists to help people understand that, right? That it's not like we can mythically create higher wages without somebody paying something. Yeah. Um, if I may just a follow-up question. Um, how how do you think that could affect, you know, so-called like mom-and-pop shops where profit margins could be pretty small and so, um, you know, maybe they have a pretty loyal client base and they don't want to raise their prices. Prices can be sticky there. Yeah. Um, how how do you think that could affect them? Um, presumably the effect is bigger on them. Uh, but in a lot of situations they have an exemption. So, for instance, in California there's a higher minimum wage for fast food restaurants, but it doesn't apply to franchises that have less than a certain number of stores with that explicit idea to protect uh, small mom-and-pop type of industries. And, um, in the case I mentioned the the German minimum wage, uh, where you were able to see, like they have this beautiful data that allows you to see every single uh, establishment firm and what happened there. It looks like probably what really happens is when the minimum wage goes up, some of the smallest and lowest wage firms do go out of business. The workers then move to a different firm where they earn a higher wage. So, you could make a case that the person running that low wage, uh, uh, you know, place that went out of business is some is made worse off. But in a way, that's kind of like economic development. I grew up on a farm, very small dairy farm, uh, in the 1950s and '60s. Those farms all had to go. My father was not, you know, the most modern farmer and his methods and so on have all gone. He, you know, you could say that the modernization of the agricultural sector put a many, many thousands of less productive farmers out of business. And almost all economic development has a tone of that. And I don't, you know, I don't think you can shy away from that. That's basically what's going to happen. In the long run, we're not going to have mom and pop shops, right? We're going to have Amazons and a big huge Target or something. And we're not going to have small dairy farms with hand, you know, hand production and um artisanal cheese and stuff unless you're willing to pay a lot for that. So, there is a cost, for sure. Thank you. Sure. Thanks. So, returning to this discussion of evidence, um I'm curious maybe it's because I don't know the study well enough about evidence with immigration, but I think nowadays, especially if you listen to a Trump rally, the concern isn't narrowly about immigrants taking jobs. So much of it is, I think, a crime scare. And I don't know about other countries besides the US, uh but I know that at least in the US, uh immigrants commit fewer crimes per capita than native-born people, including undocumented migrants. Right. And I'm curious whether studies have included that in their evidence portion and whether that shrinks the part that could be attributed to tribal or less euphemistically racist concerns. Um yeah, so actually, you know, in our the analysis I did with Preston and Dusmann, they had done some work on crime and some of my PhD students had done work on crime. And most people who've studied it do find exactly what you said, and that's not just in the US, that's true in other European countries. Um and there are there might be some exceptions in certain countries, but, you know, in the US, it's it's complete it's a way out of proportion. You know, the the natives are much better at crime than the immigrants. Uh anyway, um we didn't find that people put very much weight on the crime thing. Now, that survey was done in the you know, 2006. And it's possible that in 2006 crime concerns about crime are one of those things that a a politician can elevate by bringing up. And so you could probably get responses to that these days in Hungary that would be much different than the responses we got in Hungary back then before Orbán came in because he's just keeps harping on that, right? Uh but so but I think often times some of those issues are um they're a little bit like the job thing. They keep saying that jobs are going to be destroyed even though the evidence it doesn't because it's kind of a a convenient thing to argue. They don't really want to just say, "Well, actually we only want white Catholics in our country." Yeah, but that's fundamentally what's going on. Thank you. I I wonder why you're using minimum wage when almost nobody works for minimum wage and nobody pays the minimum wage. Ah. Well, actually that's not true other places. That's true in Massachusetts. Okay, in New England, let's say. Yeah, but in in uh you know, even in the Midwest in Iowa or something, it's huge. And if you go down from Iowa and go down to Kentucky or Mississippi or whatever, there's a huge number of people working at minimum wage. So, it varies a lot by region and place. Yeah, you know, Massachusetts has always been a highly more productive place than most of the rest of the country. And they haven't had as much problem with low wage labor market. I guess probably back in the day when we had the uh you know, the women working in the lace factories, it was more of an issue, but yeah, that's much more of a regional thing. And similarly like in in Germany, when they raised the minimum wage, one of the things that would the that study was able to show was it didn't matter in most parts of Germany except in the former East Germany. Where there's still parts of Germany that are close to the Polish border, which are very run-down, and they look a little bit like Appalachia. And there the minimum wage was was binding and biting. Yeah. Oh, thanks. Good question. Hello, my question is basically about um a living wage versus minimum, but I don't think we can go there without talking about the labor market. So, I see people losing jobs to automation. I see jobs being created that people don't have the experience or the the um the technology, the learning to be able to do the training to do them. And so, like minimum wage has adjusted like a little bit at a time, and then there's this thought of living wages. Like I actually like the idea of the negative tax, cuz I think we are going to have people that can't get a job or there isn't a job available to them with the way our labor markets are are headed. So, if we were to enforce or or or supply a living wage, we also have the problem that, you know, a living wage is going to be different depending upon the region that you're living in. Right. But, that would eliminate the negative tax and that and eliminate welfare if we could provide something for people to do and supply actually a living wage. Right. I I No, I think there's an interesting um I thought about this a little bit um if you've ever been to like Denmark. Mhm. So, Denmark historically had, you know, very well-educated, very productive workforce, and their minimum wage is around 18 or 19 dollars an hour right now. And for the Danish population, that's great. But, it creates a huge problem because they are actually they had historically been fairly generous with admitting migrants. The migrants come in, they don't speak Danish, they're very poorly educated cuz often they're from like rural Afghanistan, Mhm. and there there's nothing they can do that's worth $19 an hour. And so you could make an argument that you if you're going to have a low-skilled workforce component, possibly for you know, very good reasons. In the United States, we have a large low-skilled population, not like Denmark. You know, we've got lots of parts of the country where people aren't finishing high school. Mhm. Um so in that setting, setting too high a minimum wage is going to possibly be an obstacle. And you might not want to price those people out of work. And example of that would be like what are you going to do about um disabled people, right? So, there's special dispensation if you hire somebody who's autistic or has some, you know, other issues. And in my view, that's probably worth think you know, it has to be debated and thought about and enforced carefully. But I I think there's only so much you can do with the minimum wage or a living wage. And and you've got to be very sensitive about am I going to mess up the the lives of certain people just so that we've got the right wage for other people. So, I think most economists would say what you really want to do maybe is have a combination. Try and get the very bottom of the market with the minimum wage, but be cautious. And then have some income support programs to supplement that. You know, especially for people, you know, if someone is blind or something, a lot of those people want to work. They like going to work. Um but they can't uh necessarily get a you know, survive on what they can earn. So, you know, it's a goal of society job of society to help support them. So, the living wage would in fact create unemployment. I think it could if it was too high. If it was really high enough to live on Mhm. for some fraction of workers it would be too high, I think. Yeah. Okay. Thank you. Sure. Hi, good afternoon. Thank you for a lecture that was uh very instructive also on um a lot of the decisions that you made over your career to determine how you want to perform and develop the research questions that you have. So, that was even more informative than I expected. And I'm looking at this slide, you mentioned the A&B testing and you've also mentioned your work with Amazon. So, AB testing one one way that I think a lot of late people may encounter that in their lives is that some of the big tech companies using social media software for instance their user interface. They will roll out features to only a fraction of their users and not to others and to see how that affects in real time on production the usage of the platform is something that helps them to with their operations and with their design of future planning. So, um what I'm wondering also is I'm thinking back to the '90s when you must have been probably calling and not necessarily traveling to 500 fast food restaurants in Pennsylvania and New Jersey. I'm wondering what the take-up of the the task of participation in these studies is like and how that might have changed over 30 years. Obviously, it may be an even more dynamic question, but like Facebook, Amazon, all these big companies more than just IBM in the old day. These companies have data research firms inside them. So, that's on the unlike the macro big commanding heights level how a lot of the companies do research into how their operations should work and what their policies almost should be. They almost are big enough to have what could be called policies. But also with some of these mom-and-pop as someone mentioned, you know, have you noticed any changes in their willingness to talk with you? Have you noticed any trends in in participation and and how that aspect of it works for actually gathering the data? Yeah, well I can I can say a few things about that. One in the particular case of our study back in 1990 it was in you you '92. Um we had the most amazing luck. Um I had a undergrad I was working with on a senior thesis, and uh he was the guy that collected up We didn't We didn't have the internet. So, we needed to find um where the fast-food restaurants were. And so, in Princeton, you know, the university has a huge library with all the phone books. And so, we got all of the phone books. And we Alan and I said, "Well, how are we going to get a survey done?" We were in kind of in a hurry. And we wanted to have high quality, and we really wanted to get as many people participating as possible. We didn't want the stores to blow us off. And so, Alan and I tried to do a few surveys ourselves. And we got blown off. We We were no good. But this guy said, "Well, my grandmother is a professional surveyor in Utah." And so, we called her. I talked to her on the phone for about 5 minutes, and and she sounded In all honesty, she sounded like she was the most charming 18-year-old actress you've ever met. And I said, and he she said, "Well, I I I have a very good success rate with interviewing businesses, cuz that's what I do." She got On the first round, she got a like a 80% response rate, and on the second round, 97. Her name was Susan Belden. Uh and she was just the most amazing interviewer, and persistent, but these days it's very hard to do that. The most of the fast-food restaurants, they now say, "We're not in We're not answering questions." Right? And actually, most people don't answer the phone anymore. So, these days you have to do it by internet and stuff like that, and so. And this is affecting even like the labor force survey that the the government runs. Participation in all kinds of surveys has fallen like through the floor. And getting people to answer these surveys is becoming more and more difficult. So, it's a big problem, actually, that we're facing, and that's not just in the United States, this is everywhere. So, it's a It is a big issue. Yeah, thank you. Thank you. But it was Susan Belden who saved us. Thank you so much for this really great talk. Um I'm just wondering more like a broader question. Like what is your opinion or thoughts on the frontier or the next step in economics as a discipline and like what advice would you give to a young economist? Going to computer science. Uh Okay. Uh You you know, I'm a little I'm always a little cautious about that. I mean, we had a great as economists of my generation, we had a great run. You know, we we went from a smaller field to a big field, probably more influential. We were very influential under Obama and even Bush. There were lots of economists. Maybe not today in a lot of countries. And also, we you know, we have got many many students taking our classes. Many people know a little bit of economics and we are important in these businesses. But I think going forward, it's a little less clear you know, where the field will go. Um it's not wide open anymore. It's very hard to get into graduate school. It's It's kind of hard to get a job. So, it's not like it's been I I've had 135 PhD students that I've first or second advisor for and they all have jobs. But I think if I started now and in 40 years from now, I don't know. So, I'm a little nervous about how things are going to go. Um It's going to be challenging for all fields and especially right now with the you know, threat to the university funding and so on. But more generally, just I think economics has had a build-up and probably I could say we might be having to come up with something new which is a little hard to figure out what it should be. You know, I mean, what we're basically doing is data driven in my part of it in my part of economics. Um and we've got opportunities for most amazing data. And I think as long as we can do something useful and people say, "Oh, that was useful. I didn't know how to do that." then we'll be useful. Then we'll we'll we But we've got to work We can't just be doing abstract you know, sitting in our room playing around on the computer. We've got to deliver something that people say, "Okay, that's helpful." And I think we can do it, but we're you know, it's going to be challenging. Yeah. Can I ask a really quick follow-up? Just like if it wasn't If you didn't If you weren't constrained or if people weren't constrained, like what would be kind of your thing that would draw you? Like what What do you wish economics could be? In In other words. I I wish it was a little more risk-taking. I I think it's very small-c conservative. Um and people, you know, our our uh projects are we don't do as many research papers as other fields. So, when somebody, you know, somebody works for 5 6 years on their job market paper as a PhD student, and then they come out there and that's their life, right? That's not true in sciences. You In the sciences, you work in a lab, you have four five papers with your lab guy, and then you do a postdoc. So, you they when you tell them like what we do, they're just mortified. They can't believe it. So, we we put a lot of pressure on our kids. Yeah. So, you should feel like, you know, people are aware that you know, our grad students are under a huge amount of pressure. Yeah. Thank you. Hi, thank you so much for this interesting lecture. Um my question is regarding ethical considerations when it comes to developing an RCT. So, an argument that I've heard like against RCTs is that there are certain ethical considerations that are like overlooked in the sense that um you spoke about the example of like the Salk vaccines um where there's a placebo and then a treatment. So, when developing an RCT, how do you decide who gets the treatment or who gets the thing? Say we're giving out freebies like mosquito nets. Who gets it and who doesn't? And just your overall view on the ethics involved in making an RCT. Yeah, usually what's going on is like in the Salk case um they were really struggling to make the Salk vaccine. Actually, there's a sub story about that. One of the producers of the vaccine screwed it up. And um so one of the batch because they're under such pressure to get done. So, they needed to um ration the treatment in some way. So, in a certain sense randomization is the fairest kind of rationing. So, if you've got a scarce treatment, then I think there's a good case that if you're going to allocate it, you should allocate by randomization because it has huge scientific value. Now, I'm aware of the other point that there are people in certain categories who maybe really do deserve this treatment as long as there's some chance it works. And actually, that's how they do experiments with drugs, right? The way they do a drug roll out is they got something, they've tried it on animals, and it seems to work. They'll get a bunch of extremely sick patients. And they'll just give it all to them. They'll give it all to them, and they'll see what happens. Then they'll go to the phase four experiment. So, that is in fact and I think often times if it if it looks like it worked in that first stage and you know the right doctor, you can get the treatment even if you're not in the experiment. So, I think there are bypasses to the RCTs in a lot of cases that are affecting these extremely limit limiting cases. But most of the time you know, the the fact of the mosquito nets, you know, I happen to know about that one. You know, it's something, but actually half the villagers threw it away. So, the you know, it's not like it's the greatest gift since you know, heaven or something. Yeah. Thank you. Sure. Hi. Um Um Thanks for the talk. It was really great. Um I was thinking about two things that you said. Uh Firstly, how uh a lot of times ideas that maybe become commonplace or implemented are first born in academic institutions by people thinking hard about this. Uh and I'm thinking about how that looks right now, uh especially with something like a project 2025 born in think tanks and whatnot. Um Governments, maybe they don't care about evidence, but certainly kind of pick certain ideas from very smart people. Uh and with things now under threat uh at academic institutions all over the country, how do you think people that maybe have ideas such as yours that, you know, support um evidence-based minimum wages, uh how can these continue to spread while they're kind of under direct attack? Yeah, I I mean, I I'm very sympathetic to this point. I think um you know, economics we we shouldn't worry about our our stuff is low cost. And in all honesty, you know, we can make a small contribution, but it's not as important as some of the basic science things in, you know, medicine and and health and things. So, and it what's really um you know, I think it is extremely important that somebody support the really basic science. A lot of times that is many, many years away from commercialization. And so, you can't get a firm or a large group of firms to fund it. They'll They're interested in something that can be making money within a few years. And they're not going to fund research that is highly speculative and you you know, fund five projects and one of them is going to pay off 20 years from now. But that's where we need to be, right? If we want to progress as a society because presumably in my view, the reason why we've gone from, you know, very close to the Malthusian life in in the 15 or 1600s to what the life that people have today is scientific knowledge and building up um understanding of the way the world works and how we can use our tools. You know, we're we're we're quite limited. Our humans are very limited in what we can do, but with tools we can do all kinds of things. So, I think we do need to try and convince people of the importance of this basic science stuff. And if I was giving advice, I would tell people, look, keep the economics out of it because inevitably people hate economists. Uh focus on something like, you know, Einstein or something where everybody can agree, okay, that guy's 20 times smarter than me. I would have never thought of it and it turned out to have been brilliant, right? Where you can say you need to have basic science. It's going to create long-term benefits for the whole world and some of your friends can make billions on it. You need things like that. You probably want to stay away from anything that's the least bit controversial when you're making that case. You because it's so important. I mean, in a way I'd be willing to sacrifice a little bit of growth in economics for saving the, you know, the biosciences and those other fields which I think are fundamentally in a way very important and do require these huge long-term investments. Like the the survey that Alan and I did cost nothing. I think four or five thousand dollars. And many of these really innovative studies, like these studies of the of the German minimum wage, they cost nothing. It was using administrative data. So, we can do most of our work at no cost. Those guys need money. Yeah. Um and if I can ask a follow-up, um uh I I certainly I certainly agree. Uh But, I also think that there's somewhat of a of an ideological ideological factor here of um it's also maybe specifically in social sciences where it may be a lot um harder to to um say very kind of pinpoint accurately, all right, this is exactly what's happening. But, it's certainly important to to put that out there and express that and um make sure that these ideas are are are being talked about. Um and I could see a a path where this quickly moves into science as well. Uh even though they're kind of separate uh the way we express those ideas is often the same. Um and and I'm wondering how the academic institutions that we have can can hold themselves together while being well I I I well, fundamentally, I don't know. Um fundamentally, I think, you know, a lot of I was at an event a month ago at Princeton. And I I know the president of Princeton and and um the you know, they're working on that, but they are and I think there's large groups of, you know, probably the president of the of the University of Massachusetts is in a lot of conversation with presidents of other university other big state universities, which are heavily dependent on federal grants. And so, they're all doing things, but they're doing it probably it's kind of frustrating, I think, for other people because they're doing it under the table very quietly. Partially because they're aware of the fact that there can be like very vindictive policies focused on somebody who complains, right? So, they're going to have to come up with ways to make their point as a group so that no single university gets um pointed out like Columbia recently did and Harvard now. You know, Columbia and Harvard can take care of themselves. But, you know, the state universities like I work at Berkeley, work here where at UMass, it's much more cash flow based industry. You know, if we don't get any income coming in in the next 2 years, we're in a lot of trouble. Uh people will kind of have to be let go and all kinds of things. And that's just the math, you know, it's nothing about it. So, I think people are being very cautious. I think they are working behind the scenes pretty hard. And and probably there's good people working on it, but they're not saying much and I think it's very frustrating for advocates because they don't hear anybody saying anything. That's So, maybe that'll make you feel tiny bit better. Thank you. Okay. Hi. Uh first of all, thank you for coming here. I definitely learned a lot. Um I just while you're here, I wanted to get your uh two cents on the current like issues with rolling uh tariffs. How it affects um economically. Is it as clear cut as people make it out to be like currently we're seeing uh rises raises in prices or eventually down the road the way it's being implemented will bring US jobs and if that's beneficiary or if there's like hidden variables behind the scenes that if so, can this be done using can you run studies on this similar to the minimum wage study in New Jersey? Yeah, so this is a extremely interesting question. Um So, it's quite controversial uh what the long run effects of trade are. And I would say a major failing of economics, despite the fact that I mentioned Adam Smith and David Ricardo having invented a lot of the ideas that still underlie discussions about trade. The evidence side of that field is pretty limited. You would have thought, I mentioned this at lunch today, you would have thought that Trump won when they put the the tariffs on China, somebody would have done a minimum wage kind of study and said what happened to the prices of things that came from China versus the things that came from other places? So we'd have some benchmarks, but that's not the tradition in that field. That would be considered such so bonehead simple that no one in that was a respectable PhD would do it. It'd have to be somebody like me. Uh and so one problem is there's not much evidence. Second problem is extremely subtle. So you put a tariff on China, what's going to happen? Well, actually they're going to bring stuff into Mexico. They're going to maybe touch it up a little bit or maybe they're just going to put a stamp on it says made in Mexico and then they're going to ship it in. And so it's going to look like trade from China disappeared and emerged in but actually trade between China and Mexico went way up. So in order to study it is very complicated. Um whether there's a you know, there are definitely negative effects of opening up trade. So if you think of the Mariel boatlift of trade the Mariel boatlift of trade was an episode in year 2000 when China entered the WTO. And so it went from tariffs that protected American low-wage sectors to getting rid of those tariffs. And there's a very important set of studies. Um Autor, Dorn, and Hanson are the three authors. I'm if you want to know prediction for the Nobel Prize, I would say that paper. Um and it showed that in the places where um there was more stuff being produced that would be very similar to what was now available from China a lot of jobs were destroyed, a lot of people had um very big income losses um marriage rates went down, opium addiction went up, the whole nine yards. So you did there's very clear evidence that taking away those opportunities without providing anything in the background had a negative effect. Now whether the tariff thing is going to reverse that, you know, I'm I'm deeply skeptical about myself, but uh you know, I I wouldn't say I mean, I'm the guy that showed that raising the minimum wage can kill jobs, so what am I to say? Uh but uh I think the bigger problem is going to be First of all, there's a huge problem for the sending countries. So if you saw the thing this morning, like a quarter of GNP in Vietnam is targeted to stuff that's going to sell to the United States, and they're facing like a 40 or 50% tariff. Well, we're going to kill the Japanese the the Vietnamese economy. Right. So that's a huge cost. Independent of anything it does for us, we're going to lose, you know, one of the Well, we we did screw it up from the '60s when we tried to have a war there and stuff, but you know, there's actually very good relations between the Vietnam and the United States, conditional and other things. In In the long run, that would be a great American ally, but we're going to screw that up completely. We're going to destroy their economy. We're probably going to mess up, you know, many, many economies way more than we help the United States economy, if at all. So I think people, you know, Americans don't think like that. They don't think, "Well, you know, what are we going to get out of it? Is it going to be worth it? How much how disruptive it is, you know, the rest of the world?" But I think that's pretty important actually in the long run. Um That said, I don't I don't know. I Good question. I Some people think that there's going to be a resurgence in car production, in steel and iron production in the United States. It could happen, but people would have to be convinced that the tariffs would last forever, and that there was no way to evade them by these clever things. Like right now there's no tariff against Russia, right? So why don't we sell all our aluminum to Russia? And then the Russians export it to the United States. That's my policy. See, that's why trade is so complicated. True. Yeah. Thank you. Sure. Hi. Hi. Thank you for the You're the penultimate. And thank you for the talk. My question is that as AI is growing very fast, I'm curious to know that what do you think about its effect on labor market in future? Well, I'm glad somebody asked that cuz I made a bet at lunch that we were going to have that question. It I was getting I was running out of uh Q here. I thought I'm going to lose my bet. Um Okay, so the answer to that is I don't work on that topic. So, I I I know that some of the work that's done. And I know some of the main issues. So, here's Here's the main issues. There's three main issues. One is is is there going to be like a situation where there just are no jobs? Okay, now in the past when we've had major technological innovations like the previous computer revolution in the '80s and '90s or the introduction of electricity um or mechanization of farms there was some the people who were kind of directly affected lost jobs, but overall in the economy total jobs went up. So, in the past and you'll hear economists say this, in the past there's always been some way that people some people lose for sure and never recover, but the economy as a whole recovers. So, I would say that's a good guess based on the past. Then the second question is Okay, who's going to get screwed the worst? Is it going to be the low-skilled guys, the high-skilled guys, the middle-skilled guys, who? Right? And that's all about like is AI a substitute or complement to the skills that people people are bringing to the workforce. And there's very mixed evidence on that. There's a couple of experimental studies looking at um call agents and things like that. And there, it looks like AI is most helpful for the ones who aren't very good at their job. But there's another really cool study that looks at a scientist that um um material engineering firm. And this firm has like all these scientists working PhD scientists working on um discovering new new materials. And they're the ones that had AI. The treatment group that had AI, the the biggest gains were the the most productive ones already. So, it kind of widened inequality rather than narrowed it. And so it's probably going to be a bit unusual this particular technology in that there's going to be just pockets like probably the very best attorneys are not going to be affected. The lower quality attorneys and the legal assistants and stuff who do all the grunt work and building together case they're affected. Similarly in the computer side, like you can already see the jobs for entry-level uh software development engineers is pretty pretty flat. Whereas at the high end, if you actually know the highest level stuff about AI engines, they want you. So, that's where it's going to go. How's that? Thank you so much. Okay. Hello. Uh my name's Adam. Hi Adam. I'm a senior here. I have a couple of questions about the current state of the economy. Um I'm going to be talking about the US because I live here, I know the most about it. But as you know, the working poor are being squeezed out of asset ownership. The government has taken on mass amounts of debt. And there is a very high uh high accumulation of wealth in a very small percent of rich. So, my question is first of all, what do the rich have to gain from aligning themselves with strong anti-immigration sentiment? And my second question is what are the potential policy proposals for how do we get ourselves out of this situation? Are there any evidence-based policies that you can propose or are there policies that economists should look to research and create evidence for? Right. So, I mean, fundamentally, I would say my answer to that is get them to invite Emmanuel Saez, my colleague, next year. Because he's the expert on this rep you know, on the wealth inequality and stuff. And it's it's it's pretty far outside of my area of expertise. My understanding is that we're at a state now with the level of wealth inequality that's kind of like it was at the beginning of the 19th century, 20th century, you know, like with the robber barons and all that. And um in the past, you know, when that happened, there was some major changes and it it somehow we resolved our way out of it. Partially, the Great Depression kind of wiped out some of them. Um and then institutions brought in with Roosevelt in the '30s. Whether there's going to be something like that this time, I don't know. Uh it seems a little unlikely to me, but um and what we can do about it, I I'm not entirely sure. I mean, it seems to me very clear that um there's a kind of a problem if all of the income is going to people who don't pay taxes, we've got a real big problem. And that's fundamentally, I think, very, very important. So, if the taxes tax rates going to people that are earning, you know, several millions and multi-millions of dollars a year are below the tax rates for the middle class or lower middle class or upper middle class even, then you got a problem that all the people that got all the money are not paying much taxes, and that means their wealth can accumulate even more. And so, I think Emmanuel has been thinking a lot about that. But, whether you can politically get some kind of policy going, that's a different question. There have been countries that adopted wealth taxes. Um some of them have worked and some haven't. Um, so Sweden tried it for a while and gave it up, but other countries have tried it. And have persisted. I think that's what people are going to be talking about, but whether you can get the political will to do that, I think is really good question. Okay, I have I have a quick follow-up. Sure. Um, so you talked a little bit before about how the field of economics has changed over time. And I know that a lot of economic policy is adopted based off of microeconomics. And I know that because microeconomics, um, the way I'm taught in school is based off of representative agent, um, inequality isn't usually considered to be, um, a factor that impacts that model. So, if, um, could you could that, uh, application of that type of theory be a reason why we're seeing inequality the way that we're seeing it without any, um, action really being done against it? Or is that maybe why it's not being talked about as much in, uh, places like the Hill and places where policy is talked about? Well, I would say, I mean, there is a set of macro there somewhat complicated models, they're called heterogeneous agent neo-Keynesian models, Hank models. Um, they're very cumbersome, heavily mathematical at this stage. Eventually, they're probably going to get a little bit more simplified, but they do deal with this question of different types of people in the economy. I agree that the basic one is kind of silly in some sense, unless your goal is to just maximize the income of the economy, which is historically, that's all the economists talked about. Um, so, I think there is hope for the future. I mean, the the new next generation of macroeconomists, they all learn that model, they all that's kind of state of the art. And so, I think it will move in that direction. And obviously, if everybody's concerned about inequality as they are, and you're an economist, even if you're a macroeconomist, you have to be thinking about it. So, I think the incentives are very strong, and and and will people will respond in the long run. Yeah. Okay. If you go to grad school, you're going to have to learn it. Thank you. Yep. All right. So, you're the last question. Come on. Hi. Thank you. Um I think just within the last decade I've kind of noticed like a very high increase of like skepticism, especially like in terms of research, where like nothing is really everything's like kind of questions like very severely. So, I was kind of wondering how you like in your research practices like take your data and like make sure to like extract it. And also, when you like propose it and like present your findings to other people, how do you kind of like analyze your data in a sense that like kind of limits like the ability to like question it or like kind of look for like any underlying or like co-founding variables that may kind of like either undermine or lead your like research into question? I think that's a good question. Um actually, economics is kind of a leader in this area, to tell you the truth. Um we're not usually a leader. Um and for instance, that that New Jersey minimum wage study, um we put the data for that on the internet be- before there was really Netscape or or not Netscape, um Google. And you could, you know, in the in our book, it said there's an FTP, it's called an FTP site where you could download the data. And so, it's been available for 35 years. And actually, the reason why that study is well known probably is because most undergraduates, when they take econo- econometrics, download the data and run the regressions. And it's always a competition then, you know, most advisors would say, "Can you find a specification that gives a different answer?" Um and so, you know, I think in economics there's a every journal that publishes papers requires you to deposit your data and computer code if possible. And um there's a huge enterprise of people reproducing other studies, especially ones that are a little dodgy or that's are thought to be dodgy. So, we as sort of mentioned this before, economists are fundamentally they love to smash big guy. And so, it's a it's a benefit to our field that a lot of people would like to see sort of tweak something and say, "Does that really hold up?" So, we don't Now, medicine is different. Medicine you can't get them to give you your data give them the data. They've got the right to keep it private forever. And it's really annoying. Um so, I think but we in economics have been really There's a one of my colleagues, Ted Miguel, has a um been uh leading a kind of whole group in the economics society to help make data available in replications and stuff. So, that's what we do. Yeah. I think I think we have many faults. That's not our worst one. Yeah. Thank you. Okay, thanks. Thanks everyone for staying. So, we'd like to finish up by thanking Professor Card Card Card for providing such a wonderful presentation and very informative. And I want to thank all of you for showing up. Um of course, I'd be remiss if I didn't thank Miss Sheila Gilroy for doing a lot of the work. And you know, we had so much fun doing this, so I think we'll do it again next year in 2026. All right. See you then.