Drupal Project Estimation, for Fun and Profit

everybody and thank you very much for our presentation called Drupal project estimation for fun and profit here at ripple north we've got myself a Leesburg job my colleagues friends and Ken France is a Drupal architect Solutions Architect and Kevin is our very senior project manager who was we would not has done the figure projects then then Drupal and so I already mentioned a little bit that we were evolving office based in Montreal we love event and we love hosting cool people and we work with fancy looking logos so here here is the three of us yeah and I've been doing Drupal since pretty much the start of my career after after me go for over over 10 years France I think almost as long yeah he comes from ffw a small Drupal shop ba all over the world and Kevin Kyne has been doing agency agency work for a better part of a decade as well maybe more and most pummelling at twisting much newer man making and lege market so a little bit about software estimation we cribbed about half this talk from from this book that i that i picked up it's kind of a classic and software engineering circles if you have a software engineering degree they'll probably talk about either this book or Steve McConnell's a book quote complete and it's got a picture that Microsoft keyboard on it and so these are really like at the Bible I didn't read all of it but at least two-thirds and it has a lot of great examples and the first the first concept that you have to take away from from reading this book is that when people have what software estimation there's actually several distinct senses when somebody says hey I need an estimate for how long this project is going to take so an estimate could be a business target you know the sales guy says well the client is going to go for it if we can do it in you know 50,000 baht for five hundred thousand bucks because that's what the client wants to hear another thing an estimate is an unbiased definition of you know sorry it's a biased idea well how long do we think it's gonna take actually if you ask a dev who's gonna do it and then a third one is more on the project manager is it's a commitment it's a it's a commitment to to do this thing it's a promise to the team and everyone declines that it's gonna take that one now it's very clear that the same thing an estimate often a single number when it has these three conflict and goals and stakeholders it's quite tricky to figure out what are we actually talking about and what are we producing and and this duality or whatever multi-faceted nature what an estimate is is often responsible for why projects go over-budget you know we said it would take a thousand hours but it's not what we wanted it to take rather than what we thought it was actually gonna take so this is this is an idea that you need to be mindful so to make things a little bit smoother it's important to to to realize every time a preliminary estimate or are you talking about a commitment saying I'm gonna do this and and it also really helps it both the client and and the person who is giving the estimate are mature about the process and about what they're talking about and what they expect from each other it also is very important they have to trust each other like no I presume that I want to make as much money as possible right but I don't want to do it on this project I want to do it over the course of my career as a vendor so I want to build a successful project I want a happy client I want I want decline to to come away feeling that they got what they came in for so it shouldn't be seen as as a negotiation that's winner and loser it should be seen as both parties are trying to get sophisticated arrangement that respects the goals and constraints of both parties so I think I think that's a that's a really big part of it so another idea that we that we try to to do here is come up with a few ways of defining an estimate this book talks about top-down versus bottom-up and this is something that we practice it evolve all the time so for for the top down it's it's kind of saying well what's the project budget what do you think it is you know what kind of projects do we do what does it look like does it look similar to other projects that we've done how many people are going to work out like you know like you know it seems like a three deaf project and it seems like it's gonna have be at least a year long so this is this is the top-down approach and and it's very it's very useful because you actually when you're doing projects that are similar to what you've done before you can just kind of compare and then there's the bottom-up approach which is that you start listing all of the deliverables all of the all the phases of the project which includes discovery content strategy UI design UX design however many meetings are gonna have for that which reports you're going to produce her slideshows you're gonna do going to include a kickoff meeting and how many people we will be present and then of course you're gonna start breaking things down the development which is I would be my intuition for Drupal projects about 60 to 70 percent of the of the of the budget goes to that but in some projects especially a design oriented wants it may be lower but you're gonna break that up into Sprint's you're gonna break it up into phases and you're going to say well in this phase we're gonna tackle these things and then we take this phase should have three Sprint's of two weeks each and you know the deliverables and the first sprint will be this the second sprint will be this the third sprint sprint will meet it and then you account for the QA documentation and demo project management that going to every sprint so that's more than that though the bottom bottom-up approach do you guys know which is better what's what's the better approach for us to mission it depends that's that's one answer a better one is both are necessary so it's kind of like in math when I City math they talk when we're doing logical and mathematical proofs they talk about forward backwards technique you you going to start with the assumptions and then you try to derive some knowledge from it but very often you already have an intuition that you want to prove like I think you try to prove so you actually start with the end and and then you work backwards and say well if you if it was not the case for example is it a logical tautology so that's proof reduction of syrup and that's a very important class of proof set for things that cannot be proved otherwise so you in math mathematical proves you have to work forwards and backwards and see which one leads you to the proof that's that's how mathematicians do it and it's the same thing with estimation you have to do both a list you know a list of everything that you know about but then you also have to say well based on what this project feels like and what I've done before and what this client is used to doing before historically what is it what does it feel like and then you do both and then you reconcile them and then you can sigh like to the to the salespersons goals and then you can reconcile it to the clients budget and all of those things will end up being a single magical number and Eska it looks very easy yeah yeah another another important technique that goes hand in hand with top-down or bottom-up is is is if these three estimation approaches that Steve McConnell talks about count compute and judge so this is this is judge obviously is the gut it's the intuition it's like ah I got I often do this like people my co-workers hated what I say but I've been doing this for 11 years of I've done like a hundred doodle project now some bigger some smaller and this feels like this size and and people like Wyatt why are you doing that but actually I'm uh I'd like to think I'm pretty good but I know so so Steve McConnell says that you should we always use judging all the time but you shouldn't in fact you should use it as a last resort or as a way to check things and if possible you should resort to more scientific and systematic approaches so a one let's count one count that I do for every new project that comes in is I go to on Google and I go to site for it if they have an existing website that is I go to the site my client domain.com and I did you see how many results Google returns and then that's not working very good the recent clean they've changed it so that now it's more of an estimate at home instead of account but but nonetheless it's still a very useful index a it will tell you if it's a 50-page site a 500-page site out of 50,000 page site although beyond a couple of thousand it stops being accurate so that you could be factored more we also have a crawler that we sometimes use for these things and there's other tools but but but this Google thing it takes one second and there's no excuse not to do it another another thing that you can do often even without access to the backend of a site is you can you can say how many content X do you guys have how many languages how many modules contributed and custom we ask this to our clients for me we do estimation even if they won't give us the code I will say well okay how many lines of custom code do you have or I can I can ask them okay you want us to rebuild a Drupal 7 site up to Drupal 8 this is very common these days how how much of a budget and time did it take you guys to do it often they don't trust you especially if they don't know you so they won't tell you their budget under time but you can ask them proxies for these things you can say which agency worked on it how many months did it take how many members of your team worked on this thing how many admins do you have we're gonna enter the content how many new pieces of content that you that you're gonna get did you notice that drank a cup of coffee so so this is the kind of counts that we're talking about that when you do all of these things for for a given project than you're doing for the previous projects and you'll know by this metric but by that metric is a bigger is it smaller is it roughly the same and and this allows you to to index and have a common language with your co-workers as well with your client and explain why why tactics compute computing refers to using these counts and with historical data so on evolving web buzz of about a year and a half ago we've gotten fairly religious for logging the time in our time tracker which is toggle and then red line it's automatically synched thanks to a work of a co-worker and and so we get to look at the previous projects that seem the same according to those indexes and then say well that one took 1,500 hours even though we only estimated 1,200 or that one took 400 it was fine so this this is what it means to check your check your judgment using metrics and another another thing that I want to tell you we do I do all the time is value based pricing I didn't do it explicitly before because when we were in the smaller budget range and the clients would come to us asking for many many things basically like I was just saying why if I have to do all those things it will cost way more than I know what their budget is so I was really sticking with a bottom-up approach and then working backwards to say this is what's included this is what's excluded because I know that's now that we are a brand is a little bit better and we kind of have a target target market and target price range that we that we already know we make exceptions for but you know what kind of context we're good at everything and are successful financially for us we sort of stick to that when it's a good fit for the client you know we look for clients that are a good fit for our deliverables and then we talk to our potential clients and say here's what we did or this other client that that was great and and here's roughly what that looks like in terms of price and and actually a lot of my competitors are doing that you know when like I'm a developer first and have become like estimator and and sales person but many many organizations have just sales people who know nothing about Drupal development who don't know what a content type is so how did it have a holiday price it well they pick a number that's the most that they think the client will pay and what they know is the clients budget a lot of nonprofits actually publish their their last five years of annual reports which if you look carefully you can glean between the lines what their budget was on IT spending marketing spending and also the oh who the competitors are and how much they charge for something like this so value-based pricing is a really big part of part of that and when you come back to estimation as those three different things this makes a lot of sense I have I have a slide here oh yeah yeah so we want to point out that in addition to top-down and bottom-up there's different ways of doing bottom-up right there could be a list of deliverables but equally valuable to me is my staffing profile for the schedule and how how many people will be working per week per task per category of people so you know include your major like deliverables and milestones but then for each one category of resource I'm sort of user word and then don't forget to include project management of course and and just say for all of those things here is what this giving person will be doing that week and we typically throw in 20% for project management some sometimes it's less but usually it's more it's something we usually run over on but that's a good compromise and I also want to point out something that you probably know so has anyone here worked on a 500-dollar website top to bottom has anyone here worked in a $50,000 project okay getting more look at this ain't gonna work for a $500,000 project okay about the room anyone worked here for five million dollar project okay just one guy so I mean we've done all of these for the five million it wasn't involving up who's the prime but we're definitely important part of that project and I can tell you that sometimes if you look at the end deliver bones it's hard to tell if it cost five million or five thousand in fact the five million dollar projects look like they you wouldn't wanna pay $500 for it because it looks like garbage and because it's political right so the same nominal deliverable if you budget it differently for different organizations and their goals and expectations it'll be structured differently it's not the same deliverable because people might spend like a thousand hours per per month in meetings I mean this happens at government projects all the time and it's normal because that's how I've got needs to do business or I have a friend who just took a job at the Veterans Affairs uh and I think he spent the last four weeks writing a utility to run unit tests on every commit you know something like we get for free with circles yeah and they have an open source solution they didn't have that so they were super happy with him billing I don't know like a couple hundred hours just on this little utility that's going to make their whole multi-year project more efficient that'll ever fly for most of our clients so this is important too to be on the same page and another concept from from estimation I'd like to touch on is this idea of a cone of uncertainty so when you when you just get an RFP or you get a lead voicemail that says hey France call me back I might need a new website you know you might do a little bit the name of the guy or the phone or the woman and the phone number to see what organization he's from you look at the existing site and and you'll make assumptions as to what they want he'll talk to them to know a little bit more then Aarthi sometimes yeah or they are p it's the same thing and you have a list of questions in the air P then you're gonna go sit down and you can make assumptions around those list of questions then even put that in your in your proposal and those assumptions may half line up with what the client had in mind and half not then the project starts and you all those assumptions after a price has often been settled and the schedule is often good set of them if you start a discovery phase where you start questioning those assumptions and it turns out at least one-third well will be discarded and replaced by something else but hopefully it was an unbiased estimate so you're not too far off and then so you're getting warmer and then once you start actually writing the specs and you wrote the specs and you did the design then you kind of maybe know you think you know 75% of what this project is going to consist on but then your developer it starts working and starts building out your content types and reusing all the existing code that already exists and it turns out that no all that code that you're gonna assume you're going to reuse is not reusable at all because I was Drupal seven and this is your belayer this is multilingual or so so then when you start development you kind of get visibility in that process and then obviously as development goes on and you're into QA then it starts tightening down but at the same time when you hit QA you the client will come back and say oh great my boss now that the site looks it's done my boss finally took a look at it which he never really saw the design despite the fact that he signed off on them and now we want design changes and we want all these extra features that we don't have on our existing site anymore put in the RFP but we cannot launch without them so so this is this is the idea of this : cone of uncertainty and so and so it's okay it's actually paid to do estimation when you don't know a lot of developers who are like logically minded robotic creatures you know feel very uncomfortable with this uncertain world but this is the reality that we're in we all don't know I haven't even time to find out more and more and more and we just do our best to estimate based on any information we have and how do we in in real world deal with this cone of uncertainty we actually put the most defined easy reusable non-negotiable tasks in in that phase 1 phase 2 is the stuff that we the client would like to have but they haven't quite given us enough information and phase 3 or whatever it is slice up like how you want they screw the things that thank you later next next project maintenance will do as part of ongoing maidens after launch and and so then we jiggle things so we actually the way we hit our estimates is it's it's not because we're so amazing at estimating but because the estimation process doesn't stop when the estimate is done we redefine the project scope at every single step to make sure that still fits all great project managers do it I mean then clients are happy when you do it and they're very unhappy when you don't so that's that's a life lesson and then there's a very important rule in software estimation which is the first 90% of the coal accounts for the first 90% of development time and the last 90% sorry 10 10% the code accounts for the last 90% develop them so it adds up to 180 that's so I have I have some software estimation checklist but I think I'm running long time we'll come back to it ok and there's another book I don't have it here because I lent it to a friend like years ago and I guess it's not a friendly thing give it back it's called the mythical man-month and you want anyone heard of it no it's a it's a very classic book in software engineering it's from the 70s or something by the by Brooks who was IBM system to developments their operating system and and so here's he had like five hundred a thousand developers under under his team and there was a huge mother monsters project for for one of the main major enterprise operating systems and he he formulated what he calls the Brooks law which is adding manpower to a late software project will only make it later and he has charts about people communicating and meetings and number like email of CCS and bcc's he also talks about the fact that when you have a small team that's what clear responsibility there's gonna be a visionary who was really like the quality control the architect like the designer like who owns the functionality whereas when a project grows that gets diluted there was conflicting visions and so even he saw it in his real life work he defines the term of man month which was very popular back in the 70s planning days and he says it's not very useful but hence the mythical man-month and yeah I also remember him talking about a second system effect which I already mentioned you know how he said post-launch all those lovely things you want we're gonna do the post-launch so it turns out that in his experience the first phase that was a beta prototype always every project and that one fine one over budget but it went fine we launched it but actually the big failures in this career where the second system it's the time when you get to that okay and now we're gonna do everything we wanted and so that's the one that like you cannot use this trick of let's stay focused and just do the bare minimum and see what we get so that's that really killed so with that I will hand it over okay thank you very much I haven't had coffee so I'm gonna slow down bit so we talked about estimation and and alex has given us a bunch of different techniques and I think what friends and I want to focus on is this idea of the range and so the range is where you're going to provide a high and a low estimate for a particular project or a particular task and hopefully you feel might be present confident that it's gonna fall within that within that range [Music] so we're gonna we're not there spent too much time but just basically just is just to throw this out there these are like just an idea of ten different things where you have absolutely no idea how to estimate them and so usually if we had a little more time we would say as an example Annika you could try what do you think is the surface temperature of the Sun you're gonna try to come up with some different maybe or some logic or something same come up with a rate how about we did something like if everyone pick one you could then take a note of what would be the range that you think this is it great take one or two anything like it okay wait guys Table one remember table 2 table 3 table 5 Table six Table seven people eight people nine Table ten you guys in the back you gotta pass so people off Table one can you promote yourself and answer your question no Google please remember to use a range [Music] - what - how much great question - same height 3000 table 3 you did nice to get it right but I think someone 3 I think yeah okay okay okay that's a trick you can Oh Table six is on the cake okay Table six Table six is the total volume of the Great Lakes the leader is to like 550 meters okay okay okay that's what happens when you have one person that's a good lesson guys Table seven is the Titanic box-office receipts noble kyng consensus in Germany but I'll say what I think I thought it was like 40 to 50 no that's another valuable lesson guys read the RFP fantatic please how much how much how much ticket sales next one table was at table 1707 length of lines at Table eight okay Emily Pacific Ocean Pacific Ocean okay okay Table nine is the number of book titles published in the US this is seventeen seventy six million okay minimum up to okay okay area of the Asian continent the answer is 17 million square miles or 44 square kilometers and we said 40 to 50 million square close to 0 and E we got a great job 100 million for their currencies are a billion okay next is 50k - nobody softly we're 150 million two hundred fifty million we're a little bit no judge 30 to 50 kilometers 30 is 50k was quite off and then we said a hundred million to a billion quite off and then heavy is blue whale we said five to ten tonnes okay so you got you right got you right to rights out of ten okay right back so I hope that teaches you all a lesson about how we're all expert estimators exactly so so Kent please continue I'm actually just just to say that because this comes from this this book and so they had given this survey to 600 people and only 2% were able to score eight eight or more correct answers so we're just average guys and so most of the people were between one and three correct answers and the sort of sort of the takeaway from this is that even though you feel 90% confident you're actually 30% right exactly but the point is we're getting your range so you could have played your range bigger to encompass that that's the idea yeah of course like you can have more research you can decrease your range but it's the same but that's the same thing if you have research and information you can try to make it what smaller ranges do yeah I think the point of this exercise is also to be mindful of without doing sufficient research and without being cognizant of how low you know you're gonna pleasant surprises at the end of your project Thanks very quickly we'll just touch on this very quickly it's another estimation technique called planning poker and so basically let's say you have a project that you want to estimate you have a series of user stories in your backlog and then you'll have a team of estimators and hopefully one representing each department so let's say one representing UX design development project measurement q8 each one of them will get the stack of cards with these different types of numbers and then what you're going to do is the product owner is going to read a user story to you and describe to you all the requirements and the business requirements and then each person on the team will choose one of these numbers let's say these are just for now is helping story points is the point system and then you each estimator will choose one card based on what they feel is the level of complexity for that particular user story and they don't show anybody else so each estimator will take the card and then when they shoot their card they put it facedown and then when everybody's ready the product will say go and then everyone reveals their card at the same time and what you're trying to do is you're trying to get consensus among your whole estimation team for a particular that's a pretty particular range or an exact figure and then you have a discussion we took us to let's say for instance let's say fourth you picked a three but one unit that picked of 13 then it's up to you to have a discussion with everybody and good to say well why did you feel that this you just don't raise much more complex whereas everyone else so it was kind of like low nice teeth and so the idea here is that the estimation group reaches of consensus using these cards and then you take another code again until like until everybody's sensitivity is gonna be more towards of three or four to five or the 13th and so the advantage of not showing your cards is that you don't bias anybody because if the first person to show their card first you might be influenced by their opinion and then maybe change your estimate based on that so that's just very quickly to talk about planning poker and unless another there is that it's good not to estimated hours and because it's not realistic that it's hours it won't be so at least this is more honest that it's relative complexity of this task versus another task and then you could say what would take three big tasks and ten small tasks that's a more realistic approach than trying to say well according to of this thing it says it's gonna add up to 20 my dad's do this to me all the time this is gonna all adds up to 24 hours so one of me tough clients may take 24 hours oh my god to me it seems like 200 our project guys so let's not do that so when you have points rather than hours it's a little bit more honest that way okay we're going to do this very quickly Alex but basically we just wanted to do a little exercise again with you guys so now we're not talking about oceans or Titanic or whatever and this is like an actual request an actual request from the client so basically if you can imagine a product catalog page and a standard one and of course Drupal 8 and it's going to have all of these requirements given to us by the client so imagine that you're gonna have a page with a bunch of products card style each card must list name description tags product height there's gonna be multiple product types that to choose from the different fields and then in a search you're going to be able to either match it exactly with the SKU number or one of these search filters and then the filters are going to beat by main business units my chronic type or tank words and then when you first arrive at the product catalog page we want to show you all the results by default but then as you start clicking on the different filters we're gonna use Ajax to refresh the results on the spot and the other thing there the technology piece is to use solar so we're just going to give you maybe like one minute since most of you have experience with this to maybe just give us an estimate a range for front-end development and back-end development and you can use the tables to to discuss them on your stoves okay so yeah so basically we spent was a twenty eight hours front end and seventy-five back end yeah for that so doesn't include TM does include QA designer UX but that's just a straight-up development work for that so the total that we log in our system was a 105 not in five the thing that we should be estimating there for - if the client asked us for a single commitment number that means we should have probably given a range don't know if you asked for a single number which is often the case probably want to go to 125 to 150 for the dev and then you start adding all of the PM design QA deployment revisions and so on and then you probably will tell them 200 to 300 and and this is with the benefit of hindsight and then you might even give them some extra work if you come in under which issues happen some of that okay so one last thing before I move to inner that's the biggest problem that you encounter in this project in this search is the solar the sort of integration the solar configuration was probably the most right for this part is the biggie there wasn't back and forth not understanding exactly what we needed to do for the solar and then we had to go some back and forth with the client to get that right it was not super complex but there was some misunderstandings on the reach of ours originally estimated forty to eighty four beckoned and then forty to sixty to fountain right well the whole back end was 75 exactly once you make your search do you go you have a link to go to see a detail page you did this to get this out no that was not part of it it's just a page there's no suggestion there was another suggest was there's less don't somebody miss it very much like though that should have been listed though because that adds extra complexity to test and extra modulus can stop yeah so we did exactly match search for sq okay but doesn't make sure that autosuggest on the on the search field okay it's okay any any question guys yeah in case you're just like we do estimation next year yeah pretty sure there's so many ways that yeah so we basically estimate based on value delivery so this is more on like feature delivery and what you know solar implementation number number of this right which aligns nicely to fixed price fixed open right we're much more like agile in terms of how we estimate so we extend agile not just a software development but also to the estimation process where we'll look at initially you don't let go of uncertainty right under this whole stage look it like the level and look at the epic level and then will estimate the value associated with the effort level based on heuristics of doing similar things in the past and also like uncertainty on those levels right so that I'll call the estimations our basis I value deliveries right because an ethic is always statement of value delivery rather than the specific how long it takes that yeah so we work off of a backlog that is low fidelity and then we were find that over a period of time with the client you get a higher fidelity but then the scope always has to be variable but the end of the day you know from practicing agile the idea is that although the scope is very much the product is going to be better even if you don't complete everything so it's just a different way of course or the estimation process or an agile yeah and I and I would say that we do this implicitly we don't have a process for it this is the start because it was just like a base of what things cost us almost right and then we implicitly will validate well does this make sense do you even propose to the client this many hours for this kind of work yeah so we always do that analysis that you're describing anyway and it's not absolutely necessary let me sure creative process but we haven't yet maybe you will go to your table next to me yeah any any other questions or comments and they fit with it I know reduction pushing my group terror or every warranty period that comes out next I think that's always important thing is you always put it as down without the problem yeah we often put like a hundred hours of something for post-launch warranty and goodwill budget most images percentages yeah I think we're using and that's that's to a particular it was really 15 for QA and then 25 for p.m. okay last question folks okay good thank you very much [Applause]