Support for mini-debuginfo in LLDB How to read the .gnu_debugdata section.

so thank you all for being here and to my talk on mini debug info support in lrdb and i'm conrad klein i work for red hat and mostly on upstream lodb since last year before that i worked on an openshift in 2016 and until recently so the goal of this whole project was to improve ldb as a debugger for fedora and rail binaries where you mostly have uh when you have only released binaries right you don't have debug symbols installed and that means you don't have any symbols directly accessible and that means whenever a program crashes and your tool picks it up you you only see addresses and no symbol names and the approach was um to make ldb aware of uh mini debug info which is the concept we're going to talk about and could be that i use mini debug info and the gnu debug data section interchangeably so please excuse that and the mini debug info is where those symbols are stored that we are interested in so it's helpful to talk a little bit about why it was invented and how it was invented because before we go into uh how it actually looks so the um has been invented before i joined red hat long before and i only talked to colleagues recently about uh why was it invented this way not some other way and so the whole idea was to be able to generate a back trace for uh crashes when you have the automatic buck reporting tool in fedora and for that one wants to have symbol names and probably line numbers and file names and such and so those make up an l5 on their own right and the idea was uh to put them all in and eventually it all got too big and was we everything was stripped out and so you only are left with the the regular symbol table but cut down in fashion i'm going to show later but essentially just function names and that's it no variables or parameters and everything else the rest format itself remained even though maybe the information could be stuffed in different places somewhere else one thing to remember is this is nothing to do with uh debug information even though it's called mini debug info it's just symbol tables and nothing else so there's no relation to dwarf whatsoever and um yeah this is i hope you can read this it's not can you read that somewhat okay so essentially we can just talk about the bubbles here so to the left you see uh sort of an elf file in my mind uh you have this green bubble where you have the densum the dynamic symbols right and the sim tab and usually the sim dab is this is the super set of the uh the dim sim plus more and when you have a release binary right you essentially cut this out it's not no longer there and you usually cut it out and put it next to the debug packages and install it there but since we're dealing with binaries that have only released information and as for fedora and rel also the mini debug info that's this place here um that's essentially a gnu debug data section an invented section that contains data um namely an l file on its own which is essentially as you can see the reference there uh essentially the sim tab but cut out all the duplication from the dynamic symbols and the holds that looks like it's swiss cheese that's more or less we've put out everything that is not a function name or for example you put stripped out symbol names and variables and so forth sorry uh variables and parameters right and that has some implications on ldb for example um when you start the program and you read the symbols you um ldb usually tries to find um if there's a if there's a sim tab that's enough right you have everything but if there's no sim tab i'll do you would look go and say yeah i'll read the densum and that's it and here the implement whole implementation evolved around trying to combine those two to have something that is at least capable to give you symbols for you for your functions and uh symbol names i should say so um [Music] the way i i did this was focus on uh on back traces and not on crashes or so forth but make lod essentially aware of their symbols so that you can set a break point hit it and maybe dump the symbols and for that i just took an a uh whatever i found a zip binary that that is uh mostly installed in every system and i sort of blindly identify the function uh and the only um hurdles where it must not come from denson because that's what ldb can read must come from the gnu debug data section and then we're going to do a shootout of gdb versus lodb and so on this slide you can see i've dumped the symbols through this zip dot gg gdd online two that is essentially the nudibag data section that i've extracted from the zip binary and here you can see this uh promising help function uh help symbol because i i just looked at it and said okay maybe you can find that some other way but i looked at it and said yeah maybe it's promising if you call zip dash dash hell maybe that gets triggered and we can uh on this line we just online 12 we see that yeah it's it's not in the directly accessible symbols um so it must come from the gnu debug data section itself and it's no duplication there so let's be brave and try a demo it's not a fancy demo but at least it's somewhat somewhat interactive so um when i fire up gdb uh calling exactly what i did showed you before um zip dash dash help you can see that gdp tells us yeah we're i'm reading symbols from community back data section and it also tells us i don't have any debug symbols installed which means we're not cheating so uh what gd uh what right yeah so if we start the program you you get what you expect the regular uh zip help output but if we want to set a breakpoint on help right it can find the break put a break point and if you run again it holds it stops there just as you would expect so that's nice let's see how um see how ldv performs here and i'm talking about ldb9 which is what ships with fedora 31 so um doesn't tell us anything much the calling conventions looks a little bit different and here the same we run it we see the output and if we do try to set the breakpoint on help no way it finds it right it's uh just not there so that was how it worked in lb um nine in lb llw 10 which should be shipping soon i guess or packaged soon we uh there's proof right we run it and we can set the breakpoint and help it finds it it stops and it's just essentially the proof that yeah it works hey thank you okay that worked and uh the question is is ready to ship no of course not and the main part was in in testing so just as a word of warning this was my first contribution to lodb like i said i only started last year we have a bunch of tests that um i've been asked to implement it like take the community back data section find the symbol in there using image thumb sim tab and issue a warning when you have a mini debug info where you essentially try to decompress it but you don't have elodea to make support uh compiled within or when you're having compressed i mean you can read for yourself we have corrupted archive right and the last one was the tricky bit uh getting the there's a gdp manual page where it says how are you or how are you going to construct a a binary that has this mini debug info installed and i need to sort of replicate that in in lodb because that is the only way i can really set and hit a breakpoint and you might wonder that what parts were hard or what which not and actually setting a breakpoint worked more or less out of the box the the only problem was that there was some confusion upstream about how to create this elf object and turned out to be very easy but hidden hitting the break point didn't work because i essentially just uh fetched the the uh the created an object file fetched the sim tab out of it and stored it where we stored the other swim tab and thought yeah it's using it correctly i can hit the breakpoint everything's fine but actually lw has some concept of unified section lists and i needed to put it in there and then it all worked and also ldb does work with this concept of having a stripped-down l file that is not runnable but um where you can do sim tab uh dumps on or um yeah and that's not ideal to if you wanna say i can hit a breakpoint right it needs to be runnable and uh what was a pain for me was that um the tool in lvm that was used for that is yellow to object which takes those yaml files if we have enough time i can show you that um and it always takes the yaml file and produced an elf file and that made my tests go nuts and my head exploded because it always produced the sim dab and i didn't realize that at first because then llb only sees yeah i see a sim tab i'm going to read that and it was empty no symbols found so like usual you have the regular polishing for upstream making everybody happy and documentation is really an issue in uh in lsdb i'd say um yeah so uh we have more time so let's head over to some more slides um what i really liked was uh i came to love actually the lmv llvm integrated tester which consumes files like this one but it doesn't have to be a c file um here as you can see probably guess we just print the number of arguments in line eight and that's about it and we have number of tools involved here so you pass this to the uh the lvm lit tester and it double it it is interested in those requires and run comments that you can see there and it just says yeah i need a linux system algorithm support must be compiled in i need the exact compute executable and then it's just going to execute one by one the the run statements you're not supposed to do that in line two that you directly call gcc but here i'm just doing it for the sake of explanation so percent t and percentiles are uh percent assets this file percent is a temporary file just for this test and you then you compile it to the percentage call percenty give them a bunch of arguments output that to file check and you also pass the current file as an input to file check again so that then is interested in the check comments just to to check that yeah it's going to check that the number of outputs is five so uh that required a little bit of work just some semi canonization like uh whenever you set it i mean cmec is you can just say turn it on true one or whatever so that was kind of those were uh sort of start start as one or uh as i don't know at fault false i don't remember but that was that was sort of it and that's the example i have uh for um and aspires not runable elf file where you essentially just describe your l file in yaml format and then say um okay i'm like before we have those requires run and check comments and we pass it to the lid uh to the llvim lit and then essentially all it does is tries to find the multiply by four uh symbol name there that you can see at the end of line five in the content and uh as you can see notice line three um i had to manually remove the sim tab but that got fixed luckily and uh yeah that caused some problems uh yeah that's all i have so if you have any questions uh thank you for listening thank you yes please um so you said in the beginning you wanted this for a back recording tool and then later you said uh back traces were not gonna you know a goal of this thing wouldn't that be important uh yes sure but i needed to have it in inside of ldb first and uh have uh the i mean if you take ldv and then you can just use it and i i just wanted to to uh to have it understand this mini book info but don't you still need uh information from the debug data that you don't seem to have uh you mean like back traces or something or here although the data to find the previous stack frames good question i need to maybe forward that you would but you would have that or you would use some fallback and wider and the idea that now you can map the addresses in your two functions the question no thanks you