PyCon India 2025 Keynote: Artificial Information: How Today's AI is Changing Information - Katharine

Thanks a lot Bhavin for the glimpse and instructions. So are we ready for our first keynote today? It's by Katherine Jaral. She is a privacy activist and an internationally recognized data scientist and lecturer who focuses her work and research on privacy and security in data science and machine learning. To know more about her, you can follow her newsletters or her published book. Uh, please have a round of applause and welcome Katherine. >> I guess this will not work. So, we'll go back. I don't think it will work. Oops, it will work. Okay. Um, hello everybody. Good morning. Thank you for being here early. Uh, my name is Katherine Darl and I'm going to be talking today about what I call artificial information. So, we'll learn about it together. I'm really happy to be here. By the way, the organizing team has been amazing. The volunteer team has been amazing. I just want to really give a round of applause mainly to them. They're the only reason that uh I could make it here. Okay, so artificial information. I come from a place called Los Angeles, California. Uh in Los Angeles, California, we're really a city of immigrants. We uh almost nobody of my age was born in California in Los Angeles because it was a relatively young city and we have immigrants from all different places in the world. I grew up with most of my friends being from uh having different languages at home, different foods and so on and so forth. And in Los Angeles right now uh there is very large-scale protests that have been happening for many months because of the changes in immigration policy and enforcement there. Uh basically rounding up immigrants everywhere and deporting them sometimes to countries that they've haven't been to in 30 40 50 60 years. Um, obviously as a as a native Angelino, as somebody that spent all of most of my life growing up there, um, this is very disturbing for me. And one of the most disturbing things that I've noticed is the use of technology to surveil and find and connect data sets and deport persons. And this has been increasing technology use that's happening by ICE, the Immigration Customs Enforcements, connecting lots of different previously desperate or unavailable data and putting it in the hands of persons that are literally just snatching people off the streets. This combined with the powers that ICE is getting and and increasing powers that they have is um is extremely troubling for me. Um but this is not just happening on the local level in places like Los Angeles. This is also happening at a federal level in the US government. So here is some persons who now are in charge of connecting data sets, of running technology, of running AI at a federal government level who have zero experience in any of these data sets whatsoever. So they're connecting disparate data sets. They don't know the encoding. They don't know the data collection. They don't know the data quality. For any of you that work in data governance, you're shrieking inside of your head because we know as governance professionals that if we don't understand the data quality, how the data was collected, the documentation related to the data, we should certainly shouldn't be connecting data to each other and then making either AI based or algorithmic based or even analytics-based presumptions. And that is what I call artificial information. We take information that we don't understand. We take information of murky origin or governance. Perhaps there's some truth in that information. Perhaps there is some real information in that information, but there's probably also poor data quality, halftruth, and then we push it through AI systems and then we call it true. And that's what we're going to talk about today. So, we're going to first back up because I think we should make sure that we all have the same understanding when I say information. And what better way to start talking about information than information theory? So, who here is familiar with Claude Shannon's information theory? Yeah. So, Claude Shannon, this is uh him at MIT during his research times and his professorship there. and Claude Shannon formed uh in a thesis that he wrote or a very large long paper that he wrote much of what we today call information theory in his era they were trying to figure out new communication channel systems. So how do we take information from one end of a channel let's say a telegram and push it to another end of a channel and at the time um the prevailing idea of information is every message is just and every part of a message is just as important as every other part we know this is not true today but that's what they thought then and what Claude Shannon introduced is uh in in his paper that I highly recommend reading if you haven't read it at is this imaginary word system. So this is our imaginary corpus and next to our corpus we have the frequency of the different words in this language and we can see that some words are quite frequent and other words are very very infrequent and Claude Shannon's thesis which led to a lot of the computing power and the communication power so the design of packets and internet protocols and all these things his thesis was that perhaps some pieces of our messages are more important than other pieces. And this entropy that shows us that there's new information or this rarity of some of these words is something that we need to explicitly preserve. So rather than if we're communicating across a noisy channel, rather than send everything twice, we should be careful about how we encode and send specific pieces that have more information. we should spend specific attention to those pieces because they do indeed contain more information. This was a huge breakthrough and it led to a lot of what we now call coding theory. How many people have studied or come across coding theory? If you if you haven't and you want to learn, I highly recommend Mary Wooter's class. She has a free class online on YouTube. She's a professor at Stanford focused on coding theory. But in coding theory, we're still thinking about this information from Claude Shannon. And we're basically trying to say if we need to put this information into some sort of space, some sort of representation. And then we need to communicate it and decode it, then perhaps we should be careful and efficient about how we encode and decode. And here's just one example where we could have the code words could be embedded in spaces within these circles. And then the yellow shows the margin of error that we can retain. So the resiliency to some degree of these encodings. So we can decode up to a certain margin of error correctly. And if we can't decode correctly, perhaps we can take a good guess. And this this type of coding theory thinking how do we efficiently encode information allow us to decode it. Not only did it lead through all the breakthroughs or a bunch of breakthroughs that we have in cryptography, it also led to breakthroughs in machine learning and artificial uh intelligence. So uh what the one of the huge breakthroughs that allowed us to do deep learning with language was word embeddings which really if you've studied coding theory word embeddings are just a really clever way to encode linguistic information into tensors into some sort of embedded space. And when we think about kind of the beauty of language and obviously in India you can think of the beauty of language in many more flavors than my English upbringing and Spanish upbringing. And we can encode then many different languages perhaps a language with itself or languages with other languages into these spaces. And by distilling the information in a much smaller space than what we used to do uh with one hot encodings we can then have also uh better uh relations between the types of encodings or embeddings that we use and then we can better learn. So when word tveet came out the big thing was if you took the embedding or the tensor that represented man and you took the distance and direction towards king and you applied that to women you would get queen and this information theory coding theory how we do embeddings leads us to computational learning theory and if you haven't studied any computational learning theory that's fine I highly recommend you look up VC dimension and pack learning probably ox approximately correct learning but essentially these are some of the theories that run the basis of how we do machine learning and guess what AI is just machine learning how we do machine learning today and if we take some sort of embedded space so we can imagine this is a multi-dimension embedded space and then we try to separate let's say sentiment let's say language let's say concepts or themes ideas right we can do this based on the VC dimension of that data space and this VC dimension alongside the theories that we get from PAC basically tells us how complex is this problem and complexity and information are very much intertwined. So the amount of information directly relates to the amount of complexity of learning a particular problem and alongside that when we have so we've been spending all this time thinking how do we efficiently take information how do we encode information how do we then learn that information right when we look at today's models with VC dimension we sometimes can review a model of what is the VC dimension of the model. That means what is the amount of complexity that a model can learn correctly? And uh when we talk about overparameterized models, which is the AI models that we have today, we're talking about extremely high parameter space related to the amount of data points that are in that data. So we have data, right? That's just some representation of information. Some data, as we learned from Claude Shana, Shannon has more information than other data. And then we have these embeddings, right, that try to distill that information from whatever space it is that they occupy in in the encoding in our computers. And when we talk about overparameterized, we now have more parameters than we even have data. So we have way more parameters than we have information. This is important to think about because it means well what exactly are we storing in the parameters if we have so many more parameters than we have information. Hold on to this slot. But this for example just to show you this is just looking at the GPT parameter size over GPT 1 2 3 and four. So uh there's a concept of overfitting that I would say is now dead with overparameterized models. With overparameterized models, we have what we experience what we call the double descent when we train. Sometimes now there's not even a double descent. But essentially in in previous machine learning concepts or in not overparameterized models, we have a limited parameter space. We have to stop learning at some point in time. Otherwise, we'll overfit and we'll essentially just memorize the training data and we won't generalize well on the test examples. But with so much parameter space, we are actually can save tons and tons of information and tons and tons of different boundaries between decision space if we increase the parameters and we keep training. And now what we do normally when we train AI models today is we actually overtrain. So we train far beyond what we used to train and far beyond when we see the test error drop away. What does this mean? What is happening? So we want to kind of go back to these fundamentals. Information theory, coding theory, overparameterization. What is happening here? We've spent decades trying to efficiently compute and now we are all of a sudden somehow inefficiently computing. What's happening? Well, there's quite a bit of research on the fact that memorization is happening. And memorization today helps us actually generalize when we're dealing with an overparameterized model and trying to do something like learn the world's information, right? And this is uh let me take you through one really interesting paper in the memorization research, but I have a whole article series on it if you'd like to read more. But memorization uh is actually helping us generalize because if we have so much parameter space that we can memorize unique examples, those highly informative examples alongside generalized common examples. We can kind of do both at once. We can do anomalies alongside common. So Shiwan Zang uh is one of the great researchers. Another one is Vital Feldman whose work you can read. And this is Zang's paper on identity crisis. What they were trying to do is they were trying to learn the identity function which basically means I give you a piece of data or I give you some sort of input. I expect to get the input input back. If you've done linear algebra, you know this all too well, right? So uh what they did is they trained different size feed forward neural networks in a highly overparameterized state because the only thing they trained on is this image of the seven to the left the whole time they just trained on the image of the seven and they did uh one layer, three layer, five layer up to 20 layer and because they were using feed forward networks we can also introspect the outputs at a layerbylayer basis based on the you know kind of typical CNN design and what they found is the small layers so small networks mainly up to five layer those were able to learn the identity function perfectly we got back the inputs so we can see the inputs across the top and we got them back as we move into further layers we get maybe edges some sort of non-learning space right and then at the end when we get to 20 layer networks it perfectly memorized the seven and this is exactly uh what we have happening in today's AI models. So, we know that we have these great models. They're also very useful. I'm a big fan of of of training models. And we know that they have a lot of information and perhaps they've also memorized some of the essential information and maybe even non-essential information that's rare or informative. Now what I want to shift to is talking about how these models are perhaps used today and to think about information flows in our societies and how they relate to power structures. So, uh, Edward Bernay is, uh, if you're not familiar with his work and you do anything in marketing or advertising, you should read some of his work. But Edward Bernay uh was a New Yorker and he was basically king or founder of modern advertising, modern PR and Bernay uh helped do a lot of really in uh informative and inspirational advertising campaigns in the US. But he also worked with the US government on promoting like the SpanishAmerican War and a few other things like this. And uh I'll read his quote for us. This is directly from his book. The conscious and intelligent manipulation of the organized habits and opinions of the masses is an important element in democratic society. Those who manipulate this unseen mechanism of society constitute an invisible government which is the true ruling power of our country. Obviously, we can see US uh advertising uh coming through. No. Um but uh the this modern advertising idea, this idea that if you got a message and that message was compelling, you could convince huge groups of people to believe something or to do something that if they didn't get that message, they might have never done. And it was his theory that this was how democracy functioned. Of course, probably many other theories exist that are also true, but this is really I want us to understand that this is the same theory. And this theory, um, Bernese himself was a a New Yorker Jew, but this theory was also used by Gerbas in Nazi Germany for the propaganda department. And that leads me back to Germany. I live in Germany now. I've been in Berlin for about 11 years. And in Germany during the rise of the Nazi party and the rise of the propaganda, there was also a really cool group of scientists studying at the Frankfforter School. And here's two of them, uh, Max Hawheima and Theodora Adorno. And they were studying and trying to figure out how does psychology work with politics, work with economics and so on and so forth. and they were publishing along with a very uh diverse multiddisciplinary group of scientists and creators and artists during the 20s and the initial part of the 30s before most of them fled Germany. They were publishing a zitra social which basically means a publication on social research and had an essay thesis called and that means uh observations about science and crisis and in this essay he was talking about of course at this point in time uh the Nazi party is rising to power there's increasing ing hate crimes, discrimination, there's increasing economic crisis and political and social crisis. And he said because of this crisis state, science is being perverted. Science is being misused. It's being taken by some people and it's being used very similar to Bernay's ideas. Bernay and Hokima were both contemporaries of Freud. So very similar to these ideas to influence and manipulate people. And they said even scientists are being manipulated now. They're being manipulated because they need to get funding. They're being manipulated because they need to get support. They need to get publication. And they are being manipulated because it's the kind of the trend. All of their peer group is being manipulated. And we can take a look obviously at Nazi science was uh pretty uninformed on many topics maybe informed on a few others but uh physicists cannot be compromised right um so this is kind of what happens with fascism and the misuse of science and now I'm moving us back to today's AI because today's AI is maybe somewhere between marketing and science and in today's AI we have lots of worldviews that are encoded. We have uh if you ask chat GPT to make you some world views. I did I think this is Chad GPT4. The first world view that it gives me is IAS the technoutopian who believes that AI, biotech and space colonization will save humanity. I don't know from what but from something and we can see these are worldviews. These are encoded in how things work. And even uh I've been an advocate for AI fairness and AI ethics. But even when we talk about fairness and ethics in AI or machine learning systems, it has to be contextual. It has to be informed. Uh history and ethics, they're intertwined. You cannot take the same ethics and apply it everywhere. And so here is an example from Google Gemini that brought a lot of uh of uproar showing obviously Nazi soldiers who were clearly never Nazi soldiers and this is again applying kind of Google's mindset of trying to do the right thing do fairness but then applying that on a global level is is perhaps uh sometimes hamfisted is not uh not useful because again ethics have to be informed by local experience and history. And finally, OpenAI uh pioneered the approach of developing a model designer. A model designer is somebody that chooses who the AI is. It writes the stories that end up defining they work closely with the research scientists that take the model away from this is what the model has learned from the data and towards this is how the model should interact with humans. somewhere this step these fine-tuning steps between pre-training mode and what we actually interact with and if you look you can look up the open AI page you can read the career page for the model designer these people are in charge of storytelling they're not in charge of information they're in charge of creating an engaging experience of course information is still there of course otherwise nobody would use it right but the main goal there isn't validity to the training data or validity to the information. It's an experience that we should have. It's entertainment to a certain degree. And this entertainment combined with the fact that we have a chat interface. That means when we log in, we're not having a communal interface. You know, Google, I could send you my Google search. Let me Google that for you. And you could have maybe the same results, maybe different results, right? But we could have we could share that experience, the chat interface. And I'm going to talk about the work of another amazing scientist, Joseph Fisenbomb, who's there on the right. Claude Shannon's on the left. They're obviously contemporaries. Um, this is them at MIT with a few other famous computer scientists. And Joseph Fisenbomb built the first AI chat therapist. Okay, it wasn't really AI because this was quite a bit of time ago. It was in the 60s, but this is an example of Eliza. And Eliza was a system that he built to kind of do Freudian style therapy to try to pick out the important words the person was typing and reflect it back to them. And this was kind of supposed to be a program that people who maybe didn't have access to therapy, couldn't afford therapy, how they could get mental health. But shortly after building Eliza, he actually became a huge advocate to never use something like AI or machine learning for therapy. And why? Because he found that the people who were using it were becoming addicted to talking to Eliza. They were becoming they were sharing things that they would normally never type into a computer into a computer program which then got saved, right? He was concerned about their privacy. He was concerned about their well-being. He said Eliza isn't actually a therapist. Uh it's not a good replacement for real therapy. It's perhaps um maybe a tool that should have never been built in his opinion. And I bring us uh to AGI. I I generated this image from chatbt, so I figure it's fair game. Um, so, uh, when we talk about AGI, like this is I feel like AGI and the way that we're talking about like AI gods and all of this stuff and these type of most significant development in human history like this type as if information theory, if information theory didn't exist, we'd all be doing something else, right? Um, but uh, when we talk about this AGI, this is like supreme marketing, right? We could maybe even call it propaganda. We have moved away from like the science of thinking scientifically about machine learning, which I love, uh, and into some sort of propaganda machine, which is then being tinkered with behind closed doors by people who are paid to make us engage and make us dulge things about us via interfaces. And so uh I've spent a lot of time about the problems. Let's talk about uh how we can actually go back to real information. So moving away from artificial information. Uh I was so excited in researching for this talk to find out about the AI I for I apologize if I mispronounce I'm very sorry. the AI for Barat Barat Barat Barat um at IIT Madras and uh if you don't know about the program or if you work in the program I want to talk to you later if you don't know about the program um they've been going around and doing consensual informed data collection for things like speechto text to speech audio translation text translation and hopefully or I know that some of their work went into uh developing uh sovereign LLMs. So uh this is really cool. This is a way to not do closed information. This is a way to do open collection of information that's informed, that's useful, that also helps uh helps build new types of models and languages. Of course, will there be some worldviews and some linguistic worldviews in there? Of course. But this is a much more open way and diverse way to collect information. They've also published their data sets. In case you're also working in any of those model types, uh please take a look. And this leads to building public models. So I think two or three weeks ago, a pair twist launched. This was from the Swiss government alongside several large Swiss institutions and it uh is one of maybe the first huge public models where they've also released all of the code for training a paper on all of the data sets that they use how to reproduce it as well as uh an analysis of the memorization properties that they found in the model which of course they did. So they have a very long detailed paper, they have a GitHub, they have the openweight model. Um you can have a look. I don't know how it how good it is on uh all languages but definitely have a look. And I think that this way of publicly showing our work, which has been a a history of science over all times, is a cool way for us to kind of build models that both have the power of overparameterization, but also are potentially less influenced by marketing. And I'm a huge fan. I'm going to now move to the local level from the public and massive distributors. Uh to the local level, this is me at Pyon Germany this year. Um myself alongside in Montari who you probably know from Spacey hosted a feminist AI land party and I brought some GPU power. So this is my gaming laptop. It's got 30 gigabytes on it. And uh we downloaded a bunch of models. We served them across a LAN, so a local area network. If any of you are old enough, maybe you remember LAN parties. Um, and we we used our own data, used our own notebooks, ran workshops, ran experiments. We also had diffusion models with Comfy UI. And we were just hacking on how can we use these models side by side, how we can we compare them, how can we experiment with them, how can we figure out which models we like, which models we don't like. And if you haven't already installed Olama or GPT for all, I highly recommend that you start trying out and playing with local first AI or local AI to be able to keep your information safe, secure, relatively private, but still play around with LLMs. And I have just an M1 machine and I can run most of the models. So you your results may vary but I'm very excited for a talk later today um on edge ML and tiny ML. So there's a talk later today I'll be there on how do we use microcomputers to do also edge machine learning. Of course, those models have to be quite smaller, right? But I think moving this information to local, then you're kind of exposing what models are good at what. You're not just using a random interface. You don't really know what model version you're working with. You have to send all of your data to it. Instead, you can try doing all that locally. And if you're practicing locally, you also can move to collaborative distributed federated learning. So I've spent most of my career in machine learning working in federated and distributed learning. So how do we take data from very disperate places and how do we exchange just gradient updates or other parts of the model to train right and so this collaborative this is a real example um taking a research group a hospital group and a government and we were able to collect data sets compare them across and develop new ways to do early preventative disease detection and to actually allow for better funding and better outreach. to communities and areas of the country that were at higher risk of preventable disease. So I think when we think of collaborative learning alongside all of these other things that we've done, we can really decide what information is going in, how are we connecting it, how are we protecting things like security and privacy, and then what are we trying to learn out of it? And I for me this is the most inspirational machine learning work that I've done. How do we learn privately? What is even possible to learn privately? Well, there's a lot of I can send you lots of learning theory papers on that if you ever want to check it out. But in private learning, and this is an image from my book, we take uh the SGDs, the stochastic gradient descent training process, and we essentially insert differential privacy into this process. What is differential privacy? We're not going to get into the details. You can come find me. I'm happy to talk with you. Maybe some of you also work in differential privacy. But differential privacy allows us to say that we cannot learn from any one point. We can only learn from groups of points. So when we think back to the embeddings, it kind of allows some fuzziness for the individual points um via uh clipping noise a few other techniques that we use and some of the best work that I was ever able to do is called private and secure aggregation. And this is how do we take federated learning systems. So this collaborative learning, how do we then encrypted send our updates? This is a field called encrypted computation which allows us to compute on encrypted data with valid mathematical answers and in that process how do we add some differential privacy so that we can guarantee individual privacy and then we can combine the gradients and decrypt. So uh this is also from my book. There's a whole chapter on encrypted computation and the work uh that of the team that I was working with helped uh power the initial Google implementation of secure aggregation. So my call for us today is to bring science back to AI. I don't think it ever left but I think the science should be the middle part of the conversation that we're having here. I think that uh we should spread scientific doubt and there was an amazing Oenheim quote from Shakti's presentation yesterday about doubt. How do we challenge each other? How do we spread doubt? And in in doing some research on what is the history of science in India, I came across and now I pron mispronounce again the Nya Sutras. Okay, which is like by the way a foundation of scientific process for also a lot of the world um is about perception, comparison, observation and how do we then learn and challenge that and in these nyaya sutras philosophy there's also samsay sams say >> thank you And this is actually what I'm talking about. So this encapsulates this scientific doubt. This being able to say I don't know exactly what is the truth but I am going to doubt and challenge and doubt and challenge and by each repetition of this I get perhaps partially closer to the truth. I also want us to be easily unimpressed. I want us to have kind of our rational mind take over our emotional minds. And this is mainly because when we were talking about Bernay's and we were talking about the Frankfurter school and all of that stuff, what they found why they were both inspired by Freud is they found that a lot of this marketing and this propaganda, it uh it plays on our own insecurities. It plays on our own feelings. It plays on our emotions. And when we're making decisions based on our emotions, I mean, that's fine to some degree, right? Emotions are there for a reason, but perhaps they're not the best decisions that we've made in our life, right? And so, when somebody is posting just some nonsense that has like nothing to do with real science, uh, I want us to just be unimpressed. I don't I don't think it's that exciting. I didn't learn anything. So, um be unoppressed, be rational, challenge central control. I I have to come back and I just have to study movement patterns in many different states of India because before I came early, this is my first trip to India. I came early. I went to Kerala for UNAM and uh it was such an amazing time. If you're from Carerala, you have a beautiful state. Um, and on Onam, we were actually at Alipe on a housebo. And I thought to myself, there is zero chance that all of these boats are going to make it safely into this small channel that we're all moving into. Guess what? I'm here. All the other boats are here. Everything made it safe. There was kayaks. There was speedboats. Uh, there was crazy amounts of awesome music going on. Um, everybody was like, "Who are those weird people? Wave to them. Do they wave back?" Yes. Okay. So, it was uh this beautiful study of challenging central control. I had this idea, you know, bringing my German mind where is the stoplight? Where is the traffic light? And Bangalore has taught me you don't really need stop lights or traffic lights. You don't need any of that. Right? Uh so, uh you have a lot to teach me about challenging central control and uh I have a lot to learn there. It's still a philosophy I very much believe in. So, how do we decentralize? How do we move things locally? How do we allow people to be the controller of their own information? How do we allow people to share private information safely? How do we do all of this? And I think we can only do this um when we bring the amazing drivers in this room into the conversation and you help teach me even more things about challenging uh this central control concept. And that leads me um again this is some photos from the Pyon Germany feminist AI land party. Uh we have a website you can look it up. We have open source kits so you can run your own. Um I built the hardware kit so I'd be very happy to get feedback on on the hardware kit of how to build your own gaming PC. And um yeah, this was just a a really great way to uh build locally both uh on our land but also in our communities to then support communal learning, communal information, exchange of ideas and workshops and and just tinkering and experimenting and build collaboratively. So um in my opinion the only way that we can fight this artificial information and to move away from the AGI marketing blah blah into real science is to do so via such initiatives and to build communal local collaborative and decentralized. Thank you very much for your time today. Um I'm very very happy to talk with you further. Uh you can find my work on uh probably private. I have a a YouTube and a newsletter. Uh that's my book if you're interested. I know that there's an India edition as well. And if you're working in collaborative information models or you drive regularly in Bangalore and want to chat uh come find me, okay? I'll be hanging out by the tea. Thank you.