l10n and i18n BoF

All right. Uh Charles is going to be leading this uh buff on internationalization and localization. >> Hi people. Thanks uh thanks for coming. Uh I tried this last year. I'll try again. I'll keep trying until I have this room full of people. So let let's try. Uh so the first thing is I'm Charles. Uh I'm member of the localization team from Brazil. So Portuguese Brazilian. Uh what else? I'm a developer for a while. I contribute to Debian since 2020. And now is the time to ask you to present yourselves because this is a buff. It's not a talk. although the the room may fool you. >> Okay. Uh I can well I can go first uh before I pass the mic around. Uh Colin Watson uh I'm I've been involved in some way in internationalization for a long time despite never being a translator uh mainly in my capacity as the maintainer of the Mandb package and uh uh integrator of a whole bunch of manual page related things. Hi, I'm Carilas. So, not Charles, that's not a typo down there. I do translation between uh English to Catalan usually in the last year a lot of popp um templates. So, the questions that I ask on the um packages updates or installations uh to the users. Um, and I've been doing some other translations over the years in absim raspberry. Well, I'm I'm simply user and curious. >> Uh hello, I'm Feder and the same thing. I'm come from breast and I'm a user since most of 10 year ago I think and I use it for server at work and on my desktop computer and I'm here because I'm curious also. So, thank you for your your talk, your both [laughter] um come from China, man. Um I need before and I translate the document uh taping reference. >> Oh, okay. the the uh um deep has deep this book has very long story. The first video is almost uh 20004 years. It's the first I began take a party is translated translated to ch uh Chinese. Then several years ago the deb released the Wii U2. I also take a party translator. Uh don't only translate by myself. I also send a mail to the uh uh deb Chinese man and ask people help to the translate this book. De reference is so big. Now it take many time to translate. Um uh in my memory uh Chinese simple Chinese language is the first language translated 100%. Interesting. Okay. uh uh except the uh debin reference. I also do sign translator for PO file uh DDP uh uh package disc corrupting uh and other uh little uh simple Chinese translate in upstream or in debing or also I find s simple even stupid translator bug in upstream for example is yes no cancer It should be three meanings but sounds stupid translated for Chinese is kind of translated to low. >> All right. Uh would anybody else like to introduce themselves? Uh one newcomer. Uh do you want to as well or are you okay? >> Um hi folks. I'm James Adderson, Add Spelling out, but add N. Um, uh, I'm from Edinburgh and, uh, I guess I'm a bit interested in documentation related projects. So, Sphinx, um, and there's a bunch of internationalization, localization functionality in there. >> Okay, that's uh, it's very nice to have you here because I'm no expert in international internationalization. So you guys will help me a bit. Uh so the the first thing I I'd like to to discuss and I guess uh who here is member of a translation team uh in Debian like to that interacts with the translation team via mailing list and all that. Oh okay. Um, so the the first idea uh I had to for this buff was for us to share a bit how we work and how our our process to to do the translations uh because I I was expecting a lot more people uh about uh from from the translation teams here but since we have only K maybe he can share a bit what they do uh in the Brazilian team, we have everything uh documented in uh in the wiki, the Debian wiki uh and we tackle the many many aspects of Debon. So we do translations for the website and in there we do the mailing list uh to sync the translations. We also do the pep conf uh that's through the mailing list but anyone can use their own editor to to do it. Uh then we have none pages translation. Uh it's also from mailing list and we use the the anyone can use its preferred PO editor. Uh what else we do? Let me grab quickly uh everything that we do. Uh, so we have a space on our wiki that was recently moved. So obviously is in practice so you guys won't understand anything. Awesome. So we also do some priority setting. So every once in a while, one or twice a year, we get together to set our priorities. uh what is more important to do in this next cycle and we do the web pages, we do the dep uh models. Uh we do the package descriptions. The package description is usually the easiest one for newcomers to start. So every time a newcomer comes to our team, uh we direct them to the package description because it's a a web page you it's called DDTss. So you can log in there and you do the whole translation in the website and for younger people that that's easier for them. So that's why we do it. Uh we also do the debut installer uh num pages as I said. Uh we are also trying to translate weak pages. So we have at least the most important ones translated to Portuguese. We try to do publicity things. Uh so bits from Debian and micro news we didn't start because there I'm not sure if that will work right. Uh but uh the announcements we also translate and the Debian handbook is I guess it's almost fully translated to Brazilian Portuguese. Uh there is also the Debian EDU effort but I I don't remember seeing any movement in that front in the last year. And then we have a list of a lot of services in Debian that mainly could be translated but probably uh are are still not translated also. Okay. The the checkbox is translated by the team. uh the the circle is it's it's internationalized so it can be translated but it's not yet and we have the the cross sign for things that are not internationalized so we cannot do anything about it so so we have a lot of pages so have NM which is not currently localized we have the QA pages it's not localized ized. It's not internationalized. Uh the tracker page is also not uh it's not internationalized. Uh the mailing lists are not also but uh the BTS doesn't seem that's no >> okay. So the SSO apparently is not internationalized. Uh this was the first one that we did. No, this is the packages.demin.org. We did the mentors one. Uh it wasn't internationalized but uh the maintainers of that service did help us and then we did the translation and send it back. Uh and I think they also added the mentors in in the hosted weblate service to be translated. Uh so I mean I guess uh WNP is also not internationalized. I guess I'm just showing this to to show you that a lot of our web pages and services are not even internationalized. So there is nothing we can do as a localization team members. Uh contributors is also not internationalized. Deb mentors is the one that is uh internationalized and they set up the weblate theme for other teams to also translate it maybe it will. Yes. So this is for the Portuguese, Brazilian Portuguese, but they have for all other uh all other languages. Uh there is also de tags although I think de tags are is being sunset. So uh there's also the main pages interface for for the man page interface. Do you um there there are two dimensions there? Of course, there's the the interface itself and the contents. Does does the does the service at least give you translated man pages if they exist and it can work out what language you speak? >> I'm actually not sure. >> Let's try to see. I think try man one. Okay. or yeah, deep probably does. >> Oh yeah, other languages. It >> does. It's just the the interface that is not internationalized. So I can do Portuguese. Yeah, but this keeps in English. So I guess that's what it's meant, not internationalized the the service, the web service. snapshot uh code search Deon trends Deb and genitor the links thing dely [laughter] well that that that's a funny one right but I mean so that that's something that we eventually should be careful about uh and besides that we have uh our main lists uh the groups where to some some these are some guidances for newcomers because there is a lot of information here. We did set up some uh video tutorials for newcomers that helped quite a lot. So they would like prefer to people would prefer to watch videos instead of read documentation. But yeah, we we are trying to accommodate to to everyone. Uh and we also have a best practices uh documentation. We have a standard vocabulary like so okay this these terms always appears in a lot of places. So let's do a standard way to translate that to Portuguese. So we have this uh standard place to look at it. Uh, and we have a a style style guide for how to translate. So, so newcomers can see, read it and try to accommodate to the way we already translate. And so here when we want to make changes, we always go first to the style guide and then we try to say people, oh, okay, so we are changing the style guide. Maybe we should translate that that way. Uh, so I guess there's a general thing on how we do things in the in the Brazilian team. So Carl, if you want to maybe share what you do in Catalon or or if anyone has other experiences, it would be nice to hear. Uh first of all we had kind of a question in the part from >> oh sorry >> at these question um >> okay >> so I don't know if you know the answer >> that I not very sure if they are used >> yes I guess in in in the Brazilian team we use this Okay. So, yeah, this is kind of uh this was in the web masters uh the web masters buff early this morning. Uh they want to like move stuff other services out of the debon.org or website to like places like the INT. Is it the same? No, it's different. Uh, no, I guess they they are different. should probably take a look and answer a bit later, but there are some duplicate stuff in Debian because one person did it a few years ago and then another one joins and do and redo the things. But yeah, I Oh, so okay. So it's louder. Awesome. Uh I yeah I could send you a message and we can start working to sort this out. Uh Okay. So, just I I was just answering in the IRC, but yeah, I think we should defin definitely take a look on those pages and try to remove this duplication of stuff. Okay. So >> yeah, I can explain what happened the last year more or less in the Catalan team. Should I >> should [clears throat] there? >> Ah yeah well I was looking. Yeah. Uh so um we've been focusing specially on the pod conf templates as I said earlier and then I don't know if you've seen the workflow but it used to be or it's based the official workflow to be sending emails to a mailing list and then wait an un and specified time to know if someone is working there and then if no one is working do it. send the review then wait for the reviewer and that was a bit too synchronous and people didn't know when to start getting the things didn't know what to get next and all of this so last year in August July August they started doing kind of an experiment um that have been quite successful at the moment or so far the experiment is that I built kind of a Python script or Python system that gets all the packages from Debian that need uh podcom translation into Catalan and then it creates a list is saved into a git directory in salsa with JSON and then any user who wants to translate can get clone this check what's available for translation assign to themsel do it locally and then say well I finished this and I want to review and as a reviewer which is actually is what I usually do I review this and when I'm happy I say okay this can be now sent to the Debian package and then I figure that it's also me in this case that is the manager creates a merge request all automated with this system that I call podep confer then it creates a merge request on salsa and then if the maintainer is not um merging this in x number of days that have been changing as an experiment for to see what happens if it doesn't get maintainer create a back report which is also semi-automated saying there's a merge request maybe you haven't seen it I can provide a patch or you can merge as a maintainer and we went I wrote some stats in the part that's the >> the website of this which sadly I started having Catalan as a use case and I would like to spend some time making it more multilingual so other teams like can experiment with this if they want to uh but because we've been doing a lot of translations I haven't had time to generalize it as much as I wanted but the we went from last August 27% of strings of podep translated so is 1,600 strings and now we have 54% which is um well twice what we had. So I'm very happy even if no one else is going to use this system that we managed to increase this. We have another 1,00 strings that are waiting for the maintainers to merge them. And these I don't know if Charles or maybe Colin if any any of you have some idea on how to tackle this. The problem sometimes is that uh translator translates something uh reviews merge request gets created back and then the maintainer the package is semi-orphant or the maintainer is not responsive and the translator feels like well I've been wasting uh my time and this is demotivating for the translator. one suggestion that the translators had it was an NMU for these cases, but I don't know what's the feeling in Debian maintainers for these uh Colin wants to say something. I pointed at you, sorry, because the closer one went to me. [laughter] >> Sure, I can I can maybe try to cover it. I don't think I've got an authoritative answer or anything, but uh the it it's this is I mean there there can obviously be a bunch of causes. Uh one one other cause is that sometimes merge requests don't actually have repos. It's very easy to set up a software repository such that nobody gets notified about merge requests and that's a that's another problem. But uh the in terms of just uh dealing with ah thank you in terms of dealing with uh with sort of lost translations like that uh obviously uh pain can be worthwhile. I do find it easy as a maintainer who's uh trying to integrate these things well even even given that it can be easy to miss sometimes. Um but failing [clears throat] that uh I know that when Christian Perier was still involved in Debian he's retired some years ago now uh but he used to do he used to kind of try to keep an eye on this across the whole project and u would do NMUs as needed uh to to keep things up to date and I can't think of any instance where people got cross about that uh it was generally recognized as as a basically reasonable thing to do. Um I I it's very hard to say that anything in Debian would get some kind of push back, but I think on the whole the project would welcome NMUs for that sort of thing. >> Yeah, that's cool. I don't know if anyone has any other views. I'm happy with this. And so yeah, one thing is that we have about 1,000 I think slings. I don't know how many packages that are waiting for maintainer to clarify we opened merge request and after 30 days I'm doing now if I execute the script and I have time but between 30 and 60 days we I create a back report as a ping I'm not pinging again bugs old bug reports and the so that was cool and then we have actually the merge requests that are merged but no new upload has happened for which I don't have data because of how I have the system and I I cannot get at the moment this data easily but that's another step that some translators think like well this got merged but the final user is not um using it which is an NMU again but the thing is that I don't have the data of how many packages or slings in this state Um, so that was quite a cool thing. And then that's a question for Charles or anyone else doing translations. Are you going to add anything, Colleen? No. Another question we had is as a I think different people will have different views but as a translator would you prefer to use web late or would you prefer to use a local system with your own tools like poedit or whatever you use. Um because now we have for different reasons and diversity we have actually a web one or two web lights because Brazil has a website and Debian has another one is this? >> Yeah. Uh so so there is this hosted weblate um which if you have a open-source project or free software uh you can ask them to host for you for free uh and and it works but I've also heard experiences that if your project is not touched for some time they might delete it or something. And so we we as myself and Paulo and people from the Brazilian translation team, we decide to try to set up our own weblate instance so we could translate Debian things in there. Uh so we did set up translate.debian.net net which is our own web late instance and we've been testing it with the Brazilian stuff. So basically M pages for AP uh DPKG and we have some other stuff in here and it's it's been a very very nice experience. uh it weblate can do uh translation memory uh across the the project. So for instance we were able to to discover that we were translating some some stuff in DPKG and AP in a different way. So we said, "Oh no, we should we should standardize that because it's important to have the same translation for that stuff in both DPKG and AP." Uh so it was a very nice experience for us and we would like to uh to invite the other translation teams to to use it. Uh at the moment it's a very experimental thing. uh we do have to set up things manually the web uh to add the PO files in here and keep it up to date but if there is more interest we would like to set up automatic things to grab PO files from Debian and populate the G repo. The good thing Weblate uh can use Git as a back end. So everything that happens here uh actually gets to a g repository as commits. So if one doesn't want to use the the web late they can just download the file uh do the translation open a merge request and update on git without even touching the web interface. So I guess it can accommodate both ways local and a web- based approach. And we also have Carl working on the PEPCO manager which is a very nice tool and lets you do all locally and it also allows you to manage the state of the translation. For instance, Weblate cannot deal with oh okay this this this file is being already translated please don't don't touch it. Uh weblate does not do this management situation. it doesn't doesn't have a this file is up for review toggle button or something. Uh and that's a problem in in my sense because in the translation team we also do this through through the mailing lists but weblate doesn't have it. Uh and so PE conf manager has it. So I I I think it would be very very nice if you could integrate both things and like have it available for everyone. For newcomers, my opinion is it's way way way easier for them to use it. Uh I prefer a terminal based approach. So I I use my own stuff, but I'd like to have more newcomers and more contributors. So, I'm all up for having this as uh the translation tool in Debian. La mentioned a link on the um uh translation memories. >> That's very cool that they did know. I didn't know. You see, that's because I'm I'm too young to the project. I I didn't know about it. I will add to our week. This is this is much more internationalization than localization. But uh and it probably just reflects my special interest, but I was uh I was wondering what people's views are quite generally on the state of translation of man pages. Uh so um I've I've been dealing with the internationalization side of this, the maintainer side for for years. uh and uh the the places where I've dealt with it have been I think largely successful but it's my impression that it's still a very arduous process for for on the maintainer side uh it's obviously a lot of work on the translator side as well because you're usually talking about quite large volumes of text but on the maintainer side it's uh it's very far from automatic you have to set up uh quite complicated piles of uh of p4a scripts to to to make everything work. Uh, it often requires adjustments to the to the the structure of your man pages to get it to be translatable at all. I've had to write custom poor extensions to get it to be able to handle details of my pages. And this this all seems well outside what it's reasonable for um to expect maintainers to to support. I don't really have a view of how of what percentage of man pages are even in practice translated. Uh I don't mean of the percentage that are available for translation. I mean of of all pages in the system and so like do do people just in practice get used to reading the English pages anyway because there's nothing else they can do or what's the what's the experience for users? I think we do have a link here. Uh like this one or this one. Oh, awesome. Page not found. Okay. But uh yeah, I think this is the one for PTBR. So, Brazilian Portuguese. And we have uh the stats for >> I think all the main pages, right? So I think that's going to give if if I'm understanding that correctly that's going to be percentages of the the ones that are >> that have been internationalized in the first place. And obviously if you I I think the majority of pages are >> uh don't even have a pot file. >> Yeah. >> So yeah. >> Okay. Actually, I'm not even sure of that because an awful lot of pages are des delivered by the man pages uh central package which has like thousands of man pages and I think there is some kind of arrangement for for translating those. Okay. My opinion as a a Brazilian uh is that we should have the m pages translated because only around 10% of the population in there speaks English. So, I don't think we can be the universal operating system if it's not translated to Brazilian Portuguese. But I do also understand the um the the like the the burden on the maintainers to set up that up. So, I guess we should work on we should work on on like automation, but Sorry. There's something was somebody over here looking for ah [laughter] um that that question about sort of the I guess progress of translation it it makes me wonder if there's any measure within Debian of kind of um quality or kind of clarity of of man pages. Um because I think you know there's obviously there's kind of stage one of translating a document from one language to another. But then if if some of the details well you know sorry I'm not explaining very well. Um if you consider that just the binary yes tick that's done um that's not necessarily catching any of the kind of accuracy or comprehensibility. Uh, and so I'd wonder it could be just as true for English language things of some kind of quality scoring system. I'm I'm not aware of anything like that existing at at the moment. >> I'm also not aware of that. >> Okay. Um yeah, I guess I guess an idea that comes to mind is that that's somewhat similar to kind of Wikipedia and kind of um I think they have different classification levels for kind of quality of articles. Um I I don't know how that process kind of works. Um probably have to go digging into some of the um Wikipedian kind of guidelines for that. Okay, just a thought. >> Cool. H about about the the difficulty difficulties on the maintainer side calling. How much do you think we could automate it and make part of of maybe the helper or something to to just >> I think it's quite often a a helping streams thing rather than something that would be in in scope for a deb helper. There are obviously some Debian specific uh pages but I suspect they're already better translated than average. Um the the the the majority of cases are I think probably going to be upstream. Uh but it might well be worth uh it might be well be worth trying to write some kind of guide for for how upstream should tackle this sort of thing. Uh there's there's reference documentation in the form of poor uh or puffer documentation. If you dig through that, you can find specific uh specific instructions for dealing with man pages, but it is quite referency rather than a tutorial on how to do it. Uh so perhaps that's the sort of thing I should be trying to tackle as mant upstream. Perhaps this is a graph question. I'm not quite sure. Uh the other difficulty there is that I've to to my knowledge it is only reasonably possible to translate pages that are written using the man macro set rather than the MDO macro set. Uh I think in theory the other is possible but there'd be a lot of code to write to make it work. Uh [snorts] and this is unfortunate because in most respects the M do set is technically better. So [snorts] >> there's also the problem that upstream is generating the M pages through like help to man or something else and then you right and ideally you'd have helped to man hook up the corresponding program translations that it's generating stuff from in the first place. Uh I don't know if that's even possible. >> Okay. But but but I think I I can maybe with your help if you are uh able to set up a documentation or something about how to send this translation to upstream and maybe say hey you could implement that and we can try to to make this upstream the internationalization of the me pages. >> Yeah, I'm definitely up for helping. I don't think I can do it on my own, but >> yeah, we should have at least five minutes, right? 10. Awesome. Do you want to show P deco? Okay. Do you want to show a bit of pure devop manager or should I just link to the the presentation of the lightning talk? >> I think you oops I think you can link to the lightning talk probably >> I have it here. I will add to the buff later. So it was in mid that comes to lose. >> Yes. Uh yeah. >> Okay. So, you had also questions about copyright and so I think you should actually that's even more um maybe more useful. the I said there's a link did a link to a what I didn't link to the salsa thing. Let me add a link to the salsa thing. The main question is as far as I know in Debian we don't um specify the headers for the Debian PO so the conf translations headers and how to specify the copyright there I found different maintainers or different languages or different packages um wanting something different every time. Well, not every time, but sometimes. So, I was wondering if in the Debian inter internialization team we should specify how to move forward on that and I'll link something while you answer because I had an example. So I'm if I'm understanding it correctly, I think the the normative expectation I think is that they're under the same license as the predominant license of the package just because if they're not, everything is hideously inconvenient. Uh it's uh it it's quite it would be quite strange for them not to be under the same license as a package. But I think that the the actual headers that pub generates expect expect the package maintainer to fill this in. They so it generates uh template headers that say this this file is distributed under the same license as the all caps package package and you're meant to to fill that sort of thing out. But I think a lot of maintainers probably don't. Uh I just looked at the deb package which you might expect to do this right and it it still has the templates. It it I said I I listened I was searching but if uh Charles if you can go to the bottom because I added just a link the last link to snippet 791 just to have an example on the screen. So we have I think this is what we're using the most. So k translation of package depth messages first line then second line for many packages we had for historical reasons copyright to the uh software public interest which I don't think is right it's meant to be the my understanding was that uh the template was meant to be the packages copyright opyright holder. Uh but obviously copyright in this file is a combination of the the person who wrote the actual the the message ID is the English text and the person who wrote the message st. So it's really both. And then we have the line three I really like. I I hope that it holds the um legally thing. And then in the line four and below we are writing the each translator contribution uh as a copyright of that file which seems to be quite common. >> I think that's meant to be normally the that's sort of normally meant to express the history. You know several translators have worked on this over time then it should list all of them. And then in this one, the last translator in this case is me. Um, but I think I should have been added myself to the line number five for example to keep the history easier. Uh, so and then from 250 um packages that we uploaded, one maintainer asked one change. I was to update the Debian copyright file. Uh which I like it's a fair question but why no one else asked? I was a bit like oh uh are we missing as a Debian project updating this file too often? Uh or this only maintainers too picky if I can say or or or this I don't know if you have any thoughts. I must say I h Mark mentioned something very similar in his talk and it had never actually occurred to me. I just never thought about it one way or the other until he mentioned it. Uh and so I would not be surprised if that's true of many people. This does tie into the wider question of to what extent is the way that we're currently managing Debian copyright in fact sensible because we are in fact preserving the the copyright notices. It's just that we're not reflecting those in the summary of everything in Debian copyright, which is of course often inaccurate in many other ways. So, as you say, it's a fair point. >> The the related and related Mark Haba was having a buff, not a buff, a talk in the buff room about how to keep the Debian copyright up to date that it didn't touch this, but touched this indirectly. Our time is up. Uh if if there are any last comments then we [clears throat] should take this. >> Uh so I think uh my last comment is thank you all for joining and and helping here. Uh I plan to go through this pad and this and this notes and like send a mail afterwards to like try to come up with summary and some tasks to for us to do in this next year. and I would very very much appreciate any help that I can get. So yeah, I guess that's it. Thank you everybody.