FOI Fest skills session: WhatDoTheyKnow and Alaveteli

Welcome to FOI Fest 2026. >> We are from MySociety, the makers of What They Know. Right, so I wanted [music] to start us today thinking about the code that is behind What They Know, which we call [music] Alaveteli, and reminding everyone that that is an open source code base, which means that anybody around the world can run their own freedom of information [music] website, and lots of people do. Um we have around 25 to 30 freedom of information websites around the world, and it's an open source platform that has helped citizens make over 1.5 million freedom of information requests. And it was really nice to hear earlier two people talking about making requests around the world, and yeah, our software and websites mean that not only can you make those requests, but obviously crucially you can view other people's as well. One nice little feature that you might not have seen we've recently launched this new page what that we're calling the showcase, which is a kind of Pinterest style scroll where you can look through all sorts of exciting examples of uses of FOI around the world, and trying to get across that principle that there is loads and loads of exciting, interesting, positive work going on to FOI. And so yeah, you'll get these slides afterwards and can have a look through that as well. Um Again, I did just mention this, but just to run you through the basics of Alaveteli, it is a database it is at its heart a database of authorities who are subject to FOI and includes the correct contact details for those authorities. We obviously try really, really hard with our amazing team of volunteers to keep those contact details up to date. Um and yeah, we try to make a nice, simple, non-threatening interface for people to help them write their request. Um and yeah, when you're looking at public body websites, it might seem kind of daunting or there's a lot to process, and so we try to keep that more simple and give a bit of basic advice. And then yeah, once the request is sent, and the correspondence is handled through the platform, and the response is made public. So, WhatDoTheyKnow is now an archive of millions and millions of over a million requests. Um, it will also keep the disclosure and email logs, so we can prove that the correspondence was sent rather than public authorities claiming that they didn't receive them. So, we can show that that is the case. Um, and then yeah, the key benefit of this means that so many requests are published and that you can go and through and find out um, uh, what people asked for, what they got back, um, whether it was successful or what, um, rejections they received. Um, and yeah, that is our site. The code base, as I said, is used in 25, uh, different places, but I'll hand over to Gareth to talk more specifically about WhatDoTheyKnow. >> So, yeah, um, you'll all probably be familiar with our UK site WhatDoTheyKnow, which is, uh, run by a small team in MySociety and a group of, uh, really dedicated volunteers. It was launched in 2008, a couple of years after the FOI Act, and has grown to over 1.25 million public requests on the platform, and we estimate it, uh, makes up around 15% of requests made in the UK. Um, but as well as helping with the mechanics of just sending requests, uh, we really try to use WhatDoTheyKnow, um, to help improve the wider FOI system for everyone. So, for example, uh, we write guides to help people understand FOI, uh, and how to write effective requests, and understand the technical jargon around refusals. Uh, so yeah, we're always trying to help requestors make better requests, and, uh, always interested to hear how we can make, uh, the requesting experience, uh, better for public authorities as well. So, do feel, uh, uh, we do encourage you to reach out and talk to us. What you might be a little less familiar with is WhatDoTheyKnow Pro. Uh, so back in 2016, we started designing an additional set of tools aimed at journalists, campaigners, or researchers, or like anyone who uses FOI more than just occasionally. Uh, and some of the key challenges we were trying to solve with Pro are uh being able to do FOI investigations in private, uh making it easier to get comparable information from several authorities without having to like wade through uh loads of different websites to find contact details and uh use uh random array of forms um and then more more like make it easier to manage these uh requests without uh an overflowing inbox and a spreadsheet from hell. Um Pro has some really great features on top of the basic account uh that helps tackle some of these challenges uh of using FOI more professionally. Uh so I'll hand off to Julia to talk through some of these highlights. >> Cool. Thank you so much. Um so yeah, the first one is that you can make private requests. Um that's with WhatDoTheyKnow Pro. So that means that you can embargo your request. The default is 3 months, but you can go up to a year and then um if it uh is running out, you can then refresh that and continue to keep it embargoed. Or you can of course publish it, which we do encourage because we are trying to create, as we say, this public archive of requests. Um and then batch requests, another one um which I think is really really helpful. And we've heard a lot today also about the the value of kind of creating your own data set by asking the same question to multiple authorities. So that is one FOI request to multiple authorities. And yeah, it's about building those data sets that have never been curated in one place before. Um so the features within WhatDoTheyKnow Pro allow you to do this with, you know, a click of a button um which we think is really powerful. Obviously um we do, you know, um give advice to make sure that this isn't um uh used vexatiously, but I think it has created some really brilliant results that I will talk through in a little bit here. Um and yeah, one more thing just to chat through quickly is this enhanced dashboard. We've heard a lot about the power of a super um successful spreadsheet to manage your FOI requests and that's the sort of thing we are um trying to look at here. So, it's kind of a to-do list um uh and yeah, it tells you um like where your requests are at. And then, it will also tell you uh within your batches um kind of what's going on. Um you can classify them as um whether they've been successful or not. Um and it gives you kind of searching and filtering so you can see um where your requests are at. Um and yeah, that kind of at a glance view of how much information you've got where your where your total um set of information is. Um but, one of our newest features of Pro, I'm going to hand back to Gareth to walk through. >> Uh so, yeah. Um we showed earlier about like the standardization problem. So, even if you make a batch request, um uh you know, even if you ask the a group of authorities like the same for the same data, it's rare for it all to come back in the same way. Uh you know, that you might track it over like different financial year, calendar year, etc. or just uh take these um or like respond with these figures in like really different formats. So, uh it like managing the batches quite difficult uh in its own right. And then, you've got got a load of work to make that data actually uh usable. So, something we've been prototyping for a while and released fully at the end of last year and now built into Pro is our collaboration tool around FOI uh investigations. So, the idea of projects is you take a batch request and you can bring in people so either members of your team or people in your community or volunteers like uh climate emergency use to help with some key parts of that process. So, when you set up a project around uh group of requests, you can create um a set of structured questions uh for the information you were looking to pull out from the requests. And then, you get like a homepage which collects um uh all those requests together with progress uh for the key um key tasks that these contributors can help with. So, uh task one is classifying requests. So, uh you know, did they release the information? Did they refuse? Am I still waiting? And then the second uh stage is uh extraction. So, pulling out that data from the requests into a more structured data format, so then you can just download that as a CSV uh to work with and you know, actually use that data and compare in uh more consistent way. >> Yeah, absolutely. And it'd be nice to tell you some success stories from uh What Do They Know and What Do They Know Pro. Um and yeah, it's worth um noting that we have this part of the website, FOI in Action, um which allows you to add a citation to um your request or somebody else's request. And we now have them on a huge variety of topics, as you can see here. Um all sorts of important ones. And it's really important us to show how, as we were saying earlier, FOI works in practice, how people are making change using this. Um and yeah, so uh please do, if you're ever using What Do They Know or can link um a request to a story, that's really helpful for us and for people learning the value of um FOI. Um and yeah, there's a few different examples here. This top right screenshot is from uh Climate Emergency UK, from the score cards that you heard um Isaac talk about earlier. But there's all sorts of um different ones here and we do try to track them. Um uh yeah, it's we just think that there's so much power in this idea of like getting raw material but bringing that um together. And yeah, going from really what we would call fragmented data um to something really powerful. And um Isaac was talking about the impact of the score cards earlier, so that's another exciting project that that we contribute to. Um and uh just to say, as well as making What Do They Know and Alaveteli and keeping those sites running, we do what we call some of that kind of upstream work to try and make sure that FOI is working for everybody. And so, um obviously, you know, Alex has been talking today, who's written a lot of these great publications. But um we do a lot of work to make sure um that we can, yeah, monitor how things are going. Alex recently gave um evidence in the Scottish Parliament for the new FOI bill, which was exciting. And we're always trying to make sure that we are reacting to how things are working in in in practice. And yeah, taking the the sort of stories that we see through the platform and making them known. An example of this is yeah, we're not just trying to run a transactional service, but we call that bad practice when we see it. And so we have seen a pattern of some authorities refusing requests through email and instead trying to encourage requesters to use specific portals that are less accessible. And we call that out. And so yeah, there's an example here where we highlighted some of that. And you know, one of something we're really proud of is our relationship with the ICO where we can take these things and try to get guidance changed and advice so that the system works really well for everybody. I'll hand back to Gareth. >> So a little bit about what's next for WhatDoTheyKnow over the next year. We're really focused on trying to get more out of the information that's on WhatDoTheyKnow. So as you heard earlier, each request is viewed like about 160 times as a minimum. So you know, we know there's so much value already there on WhatDoTheyKnow. So what we want to do is improve search, so make it easier to actually find information that is relevant to your interests and your investigations. We're also looking at improving the statistics that FOI can generate around the FOI process. So things like response times, exemption usage, and you know, overall performance like on a quite granular level. We're also exploring topic extraction, so being able to group similar requests around a topic. So for example, um you know, green infrastructure or um uh uh being able to kind of identify that um requests about bike lanes are similar to uh requests about um uh you know bus routes like all around uh sort of greener transport. Um and then meshing that up with the statistics as well. So we'd be able to sit see like uh you know whether those requests are more successful or not for example. Um So yeah, we'd love to hear from you about what would [music] be specifically helpful in these areas uh cuz yeah we really want to make the uh the information released from WhatDoTheyKnow as helpful [music] as it can be. >> Cool. And I think we are actually out time. Yeah, please do um join our mailing list. And yeah, email Gareth or I directly. We're really [music] keen to hear people's feedback. We're also on this guy and those sorts of places. Um so yeah, thank you so much. >> Hello to our most [music] completest listeners. If you're still listening, thank you so much for sticking with this [music] right to the end. I'm Mif from MySociety and it's my job to put these podcasts together. And you know what? It would be really helpful for us to know a bit more about how and why you are listening to them. So if you have a moment, please do take a look at the show notes where you'll find a link [music] to a very short survey. It would help us so much if you could fill [music] that in. Thank you.