TBB-HipHop (Integration of Intel's Threading Building Blocks into Facebook's HipHop)

um here is lensgeswente here's some delay we are open parallel and then uh we will talk a little bit of intel feeding building blocks into facebook's hip hop these are our names and essentially this talk will discuss what are we aiming to do as a startup as a business so we have a model at the same time why we are working in this area multi-core and parallel computing and which are the huge opportunities that we we saw around uh that we didn't just walk up one day and say oh let's do a project with tvb into hip hop because it's 30 degrees celsius so it's a process that we had from some time and the most interesting part of what are we aiming a part of telling you the about the project it's to inviting you to join us building a more about the specific project or other projects that we have in the pipeline so thank you very much for coming welcome we have these are a couple of white papers that we are you can download it from the website but we have it in hard copies here too i already distributed a few of them and you can pick them from a chair that are on the on the door so again welcome everyone and hello to everyone who is watching on the streaming hello mom so holy one that was real that's my son okay so a little bit of intro about what is this multi-core i mean in all the years that have been around in this uh environment just to call it something you find three kind of people the ones that uh have absolutely no idea about what is this but just read one or two words then the ones that are a little bit aware and going through it and then the ones that have been playing with this for the last 40 years or more so we will give a little bit of context about what's the process that brought us here and then go straight to the specific presentation which is all in the white papers and hopefully we will have an open conversation with all of you about what we can do together and what we are aiming to do ourselves too so what's the problem the generic big problem that's happening in multi-core and why why why are we talking about this well it's simple things change things constantly change i mean you build something once and then you expect that that stuff will last forever i was planning to do a little bit of quiz that if you are able to tell me where is this i will give you something but ah okay so okay we know that certain things don't last forever now which are the options about this sort of things you can simply choose to okay i don't care i ignore i put my head in the sand it's it's an option some people just have fun ignoring that okay good luck others are saying okay i have this problem but if someone from the moor's law someone will help me so i will just sit and wait for help the standard help that i need could come i don't know from everywhere so you can wait for help not sure if the good news or bad news not sure if that help will come so you can go and discuss in a committee or have a board meeting or round table and raise your hands how many times you have been sitting in a board meeting like this and you choose who you can feel so probably the only one who is hating this photo is my daughter sitting there now where is this i will tell you later so providing that the solution of this multi-core problem is not simple it's better to aim high as high as you can in this earth so aim high you know where is this so and if you're aiming high let's think about creative things and to be honest things can in any shape from anywhere and that's uh one part of the message that we need to expect solutions of any kind from anyone and in this time i heard a lot of people i did this in 84 i did this in 77. look at these guys maybe they have solutions because parallelism is out there it's normal it's entity it's anywhere so it's even there this is the 101 in the bay area so we come from the new zealand the south island or new zealand we have one lane bridges which means that if someone doesn't stop then the other we have problems so parallelism should solve a lot of problems but parallelism has some problems too you need to take care with it because you have problems in the same 101 so you learn it here if you don't take care you have problems and has happened what is this wanaka so before you go for help somewhere else before you go asking for people that you think that they have the right options or even big money which could be in this address has anyone been there okay so just for you i'm sorry not sure oops all these names are not dentists our venture capital sequoia is there everyone so my message is that this multi-core and parallel problem won't solve if we do not join forces so this is not a property solution that will solve or just one community there are different options and these options can be good for anyone what is this okay so for the other class of people who already has certain awareness i recommend a couple of papers that are freely available if you just browse for this title from david peterson is it's reasonable download and you don't even need to pay for it but what i love is the synthesis of the problem and i'm glad that i can introduce myself by definition i'm an entrepreneur i've been on many areas in the industry my first venture was a mathematics academy i had a.com certain stage i even have a venture capital firm but the point is that as an entrepreneur there is a gap in the market there is a problem it's not only a technical problem there is not only a call it's a business opportunity so this paper gives some hints on many directions and even a more technical paper maybe some of you already heard about these guys these are classic hits so uh this paper tells about a number of solutions that have been already there for a few decades so this is what i call my introduction about the problem so let me tell you a little bit about who are we we are just a startup but we are formally incorporated which means that we pay taxes uh and i use the word ecosystem just to talk about all the friends that surround us which are in the industry which are in academia which are in the community which are developers which are in the investment area so because you don't have a simple single market that you need to point to you play with different friends and we started some time ago that was a company that we started around niagara the netra division the netra data plane system and we did a lot of things with sun microsystems and the otago university in dunedin where lca was in 2006 uh becomes thanks to our and there was the first open spar center of excellence outside the united states so the the chancellor was very happy that you you had a banner of otago university with stamp for carnegie mellon but it's not a huge merit it's just very few people doing this we went to santa clara and with the gore expo a couple of times so those were the days and we even were invited from time to time to go to the nba because you know so and uh there is a subtle basketball place there the funny thing is that this guy was a director of major architecture division of sun so the rest of history so when a big ship implodes the small ones are a little bit solid or just imploded with them so later we used that expertise with some help of new zealand government another project was a good opportunity to create more awareness about what are we aiming to do around multi-core with the community in new zealand and australia and in the last in this couple of lcas we are organizing this mini conf which is a very very good way to be in touch with this community so i wanted just to do this intro about as i say before we didn't wake up a week ago and decided let's do something on tvb and hip-hop but if we have time after lent's talk maybe we can discuss openly so i will pass the lance and he will give the story about what is what are we talking about thank you so we heard a lot about parallel programming and i wanna take you to something entirely different for a while and then we come back um hip-hop is a project that started out of facebook because of the need that they had a couple servers probably a couple hundred maybe a couple of thousand um that ran php and tried to render facebook.com and they had some issues with page loading times and page rendering times and all those kind of things and they are probably the the largest caching farm worldwide in the moment but they really really tried to get the page rendering time down so what they tried to do is they looked around the market a bit and saw that there are a couple of guys actually doing php to c plus compiling and looked into that a bit further and took a couple of those ideas and took a couple of other ideas and came up with this thing that um compiles a subset of php down to c plus and that thing is called hip-hop very hard to google as i heard earlier try hip-hop php that probably gives you more hints so hip-hop is really a php to c plus plus compiler that takes your php code spits out z-plus plus that you can then compile with gcc and run um the the thing that is a bit different about um hip-hop is um that you can't just take your php code one and and drop it onto the hip-hop engine and hope that it compiles they've skipped a couple of core language features just to make it easier for them to to come up with a compiler that is performant enough and has not this amount of side effects um or things that are really really hard to implement in c plus because the concepts are simply not there um and they stuck with um php 5.2 and didn't really make the transition to 5.3 yet but um the subset that is there compiles really really nicely it's just important to know that there's not a drop in replacement that you can just take it throw in and expect that everything works so why why why is hip-hop interesting or why should we bother um the amount of resource consumption that you can save by by compiling your code down your normal php code down to c plus is around depends on the on the region you're applying it to obviously there is code that compiles better than 2c plus plus and optimizes better than others but it's around the area of 50 to 70 resources that you can save um facebook got rid of a lot of or not read but could use a lot of servers for other things suddenly because they started to use hip-hop more um what that means also is obviously less power consumption less administrative overhead and all those kind of things um if you're running a larger shop power consumption and things like that is suddenly a problem if you run one server yeah probably the benefits another aspect um so the the performance um aspect is one of the big aspects another aspect that we were more interested in is suddenly threatening starts to work because hip-hop can do threading whereas php's threading support not really so we try to to come up with um some way of playing with hip hop and trying to get familiar with hip-hop and one of the popular projects that you often pick is wordpress because it's readily available it's widely used um so we started um taking wordpress and apply a couple of patches and see if we can compile it on hip-hop and we were not the only ones there were a couple of other ones and in the meantime there is a really really well-maintained patch set um for wordpress and there is even not all of the patches but a lot of the patches went into upstream so wordpress more or less runs out of the box on on hip-hop in the meantime so hip-hop wordpress is really really interested in getting full hip-hop support for for it so they're actually constraining themselves to to a subset of the functionality in php just to get it running on hip-hop out of the box um an interesting thing is hip-hop has two components either you compile it down statically and and run it as as a binary or there is a just in time compiler called hp hpi and for development especially hp hpi is really really nice because you don't have to compile see where it breaks go back fix your code it's a very very iterative way of or well the php way really to develop and it makes it way faster to just test out stuff our patch set is on github as well as other patch sets that are in github for wordpress and if you're searching around either in our repository or just on wordpress for wordpress on github there are a couple of folks of the same core patch set that and kind of went further we try to measure what what uh hip-hop can can give us in in terms of running it on a standard apache setup and then running it on um on hip-hop and the the standard lamp stack that we used was just a virtual machine running in virtualbox um on a standard laptop with two cores in it um and we we just tried to to push it as fast or as hard as we could we used um sorry as the the stress testing tool is really really cool if you want to do some sort of flashlight effect things or dig attack things where you have a load that is slowly building up and then a peak and then a load that is happening out again so it can really flood um flood a web server very very nicely um and not only with with kind of hitting it and hoping that you get a response but with really defining a path through the website that a user would actually take so if you if you're doing stress testing on your website song is it's really cool written in urlang another passion of mine so we set up a zongtask that had about 250 users stressing the system from one user arriving at the second to every quarter of a second a new user arriving and so on so it really built up a load very fast and then kind of maintained it and um hold it for for a couple of minutes and um we we thought like 250 users is probably a fair amount of of users we can probably push it that far and sure enough we pushed it against our vms and they just swapped to death and nothing really happened anymore and we pushed it a bit too far so our expectations of um how how far we can push um the chest really out of the box it was not optimized it was out of the box lamp stack yeah broke it horribly we we tested it with lower numbers but um that was that was kind of the interesting number for the next experiments then so we had actually obviously working tests as well but um yeah the the hip-hop stack outperformed the the lamp stack in that respect that it was still responsive it didn't crash um it went through and um we actually got results which were the graphs we are showing in the white paper as well this is roughly how it looks like if you run a dig attack against against the virtual machine that runs hp hp hp hp so the hip-hop engine really the compiled version um but we also uh really only measured uh the the page rendering so the task that we set up uh didn't request all the static files like css um images all those kind of things um so we really only uh focused around um testing the performance of the of the php engine itself slight digression tbb we've been talking a lot about parallel programming before and tbb is a library that aims to make parallel programming really really straightforward we all know it isn't but at least it makes it a bit more a bit easier to to deal with memory management with locking with programming towards more than one cpu with a couple of constructs that you can use it's an intel library it's open source and it's not only running on intel hardware so it is written in a way that you can actually define different backends for for different processing architectures so it is written in a modular way um and it supports all major architectures in the moment i think they're still working on the arm port but all other major architectures should be working in a moment so our idea was to put tbb into hip-hop or extend the tbb functionality in hip-hop because it offers those wonderful constructs for parallel programming and it was already used in hip-hop for memory management and to get the to get the threading synchronization right already so our our first kind of aim was um to look which commands we could pick out of tbb to to impr uh to improve the performance of the parallel performance of hip-hop and ultimately of wordpress and we picked parallel 4 for a for a quite straightforward way to map for loops in php onto something that runs on several threats on several cpus in in the real on the real machine and it turned out it is it's not been that hard to to implement a new language new language constructs into the hip-hop engine it is not that hard to extend the the language set that hip-hop understands and and therefore the the php language that that hip hop can compile uh so that first thing getting tbb into getting this parallel forward tbb construct into hipaa was actually pretty straightforward and the next thing was a bit more work to get this parallel full construct into the tbb source so we hacked a couple of for loops that had no side effects that could be parallelized um and um yeah i ran a bit into problems with the fact that we can only use php two five two so we couldn't use a couple of the php five three features that we would have loved but um all in all it turned out to be a workable thing then we got patched wordpress compiled with our new language constructs and with our new for loops and turns out it was worth the effort we the we did a lot of testing and we stressed it from from different angles but we we found of that we got um about between 100 and 200 megabytes of less memory consumption roughly we got the whole test suit running in uh in considerably less time i think it was in the end around between the the normal run was about seven minutes and the our optimized one was about five or six minutes in on average so we really got got a considerable speed up we got um a definite um definitely better result in the page surf pages served in parallel we had on the unoptimized version about six pages served in parallel on the optimized version we went up to seven so there was a considerable difference between the the normal standard hip-hop version and our hip-hop version with the parallel full construct this is how the graphs look like the thing we we kind of found out in that whole process was that tbb is really easy to to use for for writing parallel stuff whatever it is it it makes it really really easy for for cnc plus plus programmers to to actually get stuff done on on multiple cpus in in a multi-threaded environment and it was a really really good fit for hip-hop it was in there already for memory management so we we really only had to extend it in terms of um hooking our custom commands in it and the the speed up we got out of it compared to the standard php running on a lamp stack to hip-hop and then to our optimized version of hip-hop was a really really big gap and if you look at that for deploying a larger setup where um things like power consumption things like um resource consumption in terms of having 50 servers or having 100 servers makes a real difference um i think hip-hop is a is a really strong contender to to jump in there and kind of get those resources down and um the other thing we we um found out is that hip-hop is a very very interesting framework that can be hacked in various ways and can really be adopted to the needs that your specific application has so it doesn't fit everything obviously there is corner cases where hip-hop really just scales minorly better than than a standard php stack but um for a lot of um purposes hip-hop can definitely speeds things up and can be hacked in ways that it even speeds it more up for for the for the code base you have so i'm roughly ready for questions roughly ready for no but there is maybe you can clarify that some people believe that hip hop for it's just for size of the size of the size of facebook but we are for example working on how we can whatever side but not whatever side but the slow php is like and we yeah yeah that's right that's that's something i mean everyone you can run hip-hop on your on your own server but probably it makes more sense on larger sites but it doesn't have to be it doesn't have to be facebook size it it makes sense for for smaller smaller applications as well a a lot of packages like wordpress are pretty useless without their plug-in architectures how does that fit in with using compiled wordpress and then are you still able to use wordpress plugins and that sort of thing or is that kind of um you can choose beforehand and then there are actually well obviously code that is not compiling on hip-hop can't be used but there are already quite some plugins that work with hip-hop and more and more as wordpress especially pushes more and more towards compatibility of uh towards the hip-hop engine um i like hope at least a lot of plug-in developers pushing to that direction as well i know that wordpress itself is really really interested in hip-hop and they they're definitely from the feedback we've seen they're definitely moving into into that direction so i guess the plug-in developers simply either have to catch up or go stale on this platform at least hi there um the parallel four extensions that you made um do you know if anyone outside of your companies using them or whether like facebook for example are using them at all um we have published them on github um i'm i don't know if everyone anyone's using them in production but i know that a couple of guys have played around with them and looked into how we've done them we obviously uh fed back our findings and things we did to facebook and they looked into tbb in more detail so i'm not sure how much of the work that we've done is really going back into hip-hop but we definitely saw some interest in the stuff we've done okay so the follow-up question to that is if if you're not really seeing it going back in what is facebook's attitude towards these sort of community contributions are they merging them back into hip-hop themselves and then using them or this is just sort of like a bit of fragmentation that's happening all throughout the hip-hop community um they take in uh community contributions and they i think they're really interested in growing a community around hip-hop um how much of that really always goes back straight in um if it is not really touching one of their problems i'm not sure but they're quite happy for community um contributions that fix problems you see the sorry the mailing list you see a lot of activity on the on the facebook developers about the contribution they're going back so it's the guys who presented facebook sorry hip-hop less than a year ago you just follow them and they are every week they are constantly coming and they are presenting all over the world so it's a really active uh you can see even for their commercial interest so it's not only for the good of the community they uh i mean i don't know how big is facebook in terms of service i guess that it's really big therefore they have a purely commercial interest also why not compile straight to machine language you've got you've got a parser i assume you've got all of the gcc back end stuff why the intermediate c plus plus um it's it gives you a bit of a couple of a couple of advantages if things go wrong in the compiler you still have a reason uh in the in the parser you still have a reasonable chance to actually read the source and can actually tweak the source if you have to um and you can compile it to different architectures so you're not if you're really going straight to assembly um you're restricted to the architecture you're you're designing for and if you if you change the architecture you throw away your parser okay thanks um yeah gcc has does have that system for doing for making languages inside it um well of course the project was started by facebook um so it's kind of their architectural decision to sort of take the simpler path of well they can understand changing php to c plus and and that sort of that means that they don't sort of have to get involved in the compiler internals that you know how much time it would save them and how much faster it would be um yeah that's just the way they started the project so we'll work with that just a question is any facebook developer here once you've hacked your wordpress php code to use parallel 4 can you execute it back on lamp and what effect does that have um not in a moment we have been looking into feeding back a couple of those um language constructs into standard php but um we are still working on that one and i'm not sure when we'll get there the problem with um with that is that core php um threading is is not really where it could be i know sam would probably know more about it but it's the the problem is really that um threading core php including all the libraries that hang off it is really a mission and it is it isn't it is relatively threat safe ish but not quite there that you can just take tvb hook it into it and and and run it so fundamentally if you want to start using hip hop you're not riding php anymore no if you're right if you if you run on the on the on the standard hip-hop machine engine everything is fine if you run on our hacked engine you can you have additional syntactic sugar to express different things and um if you want to make a portable patch set out of it you basically have some if death equivalent that says if i run on hip-hop i have i can run a full loop like this otherwise i have to run a full loop like this we have plenty more time folks we're 10 minutes ahead of schedule whoa um blogging doesn't seem like a inherently like parallel problem so how how could you get uh much of a speed up from parallelizing wordpress logging right so before before um before these extensions you weren't able to serve more than one request at a time was it is that the bottom line what do you mean you have obviously more requests more than one request at a time if you have a busy web server you have thousands of requests yeah exactly yeah but and the logging how the logging works then in parallel or you can blogging syslog blogging um i'm i'm still struggling a bit the the thing you have is something that runs in parallel you you push you you use one thread for for the the thing that you're executing and another thread is working in parallel on i don't know loading your modules to render your page and then there's another thread that loads another module for your page or another plugin for your page so one of the one of the for loops that we optimized early on was loading all the plugins that render your site and this can absolutely be done in parallel because there are no side effects you only have to load all the code and then you end up with the code in memory um so the that alone that alone sped up the the page rendering considerably because it's normally a sequential thing where you go through and throw in through so you're still rendering one page it's just happening in parallel any comments on the prospects for niagara under the new ownership the only comment that i can mention it's what i see on the back page of business week that advertising of oracle about how good they are against ibm but no we just had a nice relation at the time which opened a lot of doors uh you know i mean you organize a mini conf with vince sir for paul mckinney or these guys just because you are creating a nice brotherhood out there so that was the original part but that won't happen that's why i put the board meetings that won't happen when you have stones so things are happening in a very isolated place i mean isolated this is a change that in one way time it's on our side it's one we won't leave this room and say suddenly oh wow i need to change my religion i need to forget all what i knew it's it's a process so and yeah so deployment-wise um how how different is hip-hop uh deploying a hip-hop instance of wordpress to just doing it on normal php like obviously things like apache rewrites and whatnot aren't going to work just the same yeah does it work again yeah probably that's easy the deployment process is obviously different um hip-hop is uh has a built-in web server thing that serves the binary basically so you don't have to it's a bit like fast it's i think it is a fast cgi protocol actually no it's great http sorry it just um what what we've done is we hooked it directly behind nginx and just ran it behind nginx as we would any other non-php thing that's more my world and with with that um the the the running of the website is really restricted to run on that on that engine um and all the statics for example were in in in our example uh served directly by nginx and not by by hip-hop there is some support for render for pushing static stuff but it is i i wouldn't see any benefit in doing it um the other thing that you have to think about in in terms of deployment is obviously that you're actually deploying a binary now you're not just deploying source codes um you can deploy source code if you run hp hpi um the the trust in time compiler which is still a speed up we still got a speed up out of just running hp hpi um but uh if you're if you're compiling you actually have to think about if you're compiling in a big scale you have to think about build forms you have to think about how to actually deploy the binaries um to the different boxes if you have different architectures you have to compile for different architectures things like that so that becomes slightly more complex if you go into a larger scale but on a smaller scale um to tinker around play with hp hpi just throw your sources at it and if that becomes stable compile it and push your compiles to to the amount of notes you have how long are you talking in terms of compile times like you you're saying it takes a while you need to build farm how long does it take to be a wordpress for example the compiling compiling uh hip-hop itself takes forever you don't want to recompile it often it really takes forever but compiling the website depends on how big it is i don't know if i have any numbers on how long it really took to compile but it was in the in the amount of minutes it wasn't like days so it is not huge it is just if you have if you have 20 uh 20 servers out there you don't want to compile it on each server you really want to compile it on one server and then push it out to the 20 as a binary um have you with parallel 4 did you do much in the way of adding synchronization to other stuff so can parallel ford jobs say atomically echo stuff and it's not going to interfere with each other or are you are you just not echoing in parallel four jobs we're in the moment not echoing in parallel for jobs but you can it is simply threats that run in parallel um we didn't have any uh any outputs of those things it was really all things that were side effect free that didn't really have any output that only returned stuff that wasn't necessary for the next cycle of the of the for loop and this is how we could run that in parallel no other questions one more when you translated it to c plus did you see that process introduce any bugs um we haven't found any so it seems relatively stable but also we've only uh really used hip-hop in a couple of small test things that we made up so we haven't compiled a huge amount of php sources and just uh looked how how well it performs but um there the hip-hop guys are pretty onto it to make it bug free facebook's pretty big too yes that code base alone probably pushed it pretty far already well thank you very much that's been very informative i very much apologize for my complete ignorance of the topic and we've enjoyed the talk so thank you thanks