Deconstructing the framework, by Gary Bernhardt

[Music] you guys ready um i'm gary bernhardt i look like this on the internet i look like this in real life sometimes i run a company called destroy all software that produces screencasts on various advanced development topics and the title of this talk is deconstructing the framework it's about web frameworks i use rails as an example it's not rails specific and it has five different parts the first of which is design in the small so the design of small fine-grained pieces and i'm going to use one very specific example i'm sure you've seen a code like this in rails apps before i have a comment model active record model it has an after create callback that notifies all commenters so this is the behavior that github just has for example you comment on a gist and for for the rest of time you will receive notifications and you can't turn them off sadly you guys should fix that the the problem with designing things in this way is that you end up with a system where all your tests are testing some system under test the the systems under test are using model objects and the model objects want to send email or want to hit an external system of some kind whatever that system is rails happens to turn email off for you in tests so email's not a big deal but other external systems are a problem so you end up first of all having to stub the external system to create your model objects which seems extremely wrong to have to do that and you also end up with this situation which i'm sure you guys have seen before you break the external system and suddenly all your tests break because they all depend on this thing that really wasn't relevant to the test or to the model object the fundamental problem here is that the record is coupled directly to the api the active record object is coupled to the api and the obvious solution as with anything in software is to add a layer of indirection so if we create a service that aggregates the the process of of creating a user or creating a whatever it can make the record it can hit the api it contains that logic and here's the same example done that way so this service posts comments it creates it in the database it then for each of the commenters in the thread it emails them and now the activerecord object has no coupling to the external system to the email server this is a small sort of obvious example but it's going to motivate the rest of the talk starting with services wrappers and records this is sort of the minimal web application architecture that i think makes sense and can scale you to what we would probably consider a medium large rails app which of course in the grand scheme of things is not that big the idea is that you have services things like i just showed you stateless objects with behavior but no data in them the services will talk to records these are the things in the database and the records are very simple and they're they're very thin layers over the actual database itself so for example here's what i'd like an active record record to look like this is an example taken from the destroy all software sucks rocks series of screencasts it's a cash score this is a system where you type in a word and it tells you how good it is based on what the internet says so if you type ruby in it will google for ruby rocks and ruby sucks so very english centric and it will compute a score from from the relative numbers of results i have two methods in it only the first of which saves a new score it's a very thin wrapper around create as you can see but it narrows the interface so it takes exactly two arguments they have to be there there's no ambiguity it also gives me a point that i can stub in my tests that i control so i'm not afraid to stub it or i'm less afraid to stop it and it hides any changes in rails that might come now create bang probably is not going to change but if you think about rails 2 to rails 3 the whole query api changed a lot and basically everyone was screwed if you had a rails app at that time i'm sure you remember it sucked because you probably had controllers that were calling the querying api query query api changed bad times so this prevents that as well in the future and the the second method there is uh the query method speaking of queries it finds a single term and returns the score and the interesting about this interesting thing about this is that it's a function from a string to a floating point it does not return any model objects in fact the rest of the system never actually sees a cache score it sees the numbers it sees the the terms that it asks for but it never sees the actual active record object in general i want my active record objects to load things save things update themselves so like change password might be a method i would put on there it might have to change multiple fields at the database level but it provides a slightly higher level update and then very simple aggregate functions like full name is first name plus last name so those are records and the third thing is wrappers around third party apis so in that same system that i built for those screencasts i had a search engine class and this is how i search for those phrases it's an extremely simple wrapper around our bang which is a ruby library two lines of code as you can see but these two lines of code give me so many benefits for the design and especially the evolution of the system it gives me a place to stub in my tests if i want to it protects me from changes in our bing it also allows me to swap the entire search engine out if i want to for it for google or whatever so there's all kinds of advantages to writing even these two small lines of code and finally in this very simple application architecture services can call other services so they can be aggregates and you have a tree of services is what ends up happening so this is the simplest way i know to design an application that will grow reasonably over time it won't turn into a ball of mud with your thousand line controllers or if you read skinny controller fat model your thousand line model which has exactly the same design problems which speaking of thousand line things brings me to part three the squishing rule which i'm going to motivate with everyone's favorite principle the srp often quoted rarely understood a clash should have one and only one reason to change so i thought we would go through very briefly and look at the reasons to change that your typical rails controller has they tend to do authentication so that's a reason to change they tend to do authorization which of course is different that's a reason to change they tend to wrap http in both directions so coming in you have params and headers going out you have status code and headers reason to change they tend to manipulate the model calling methods on activerecord objects that you defined that's your model they tend to manipulate the database which is actually a different thing if you call the reload method in a controller that's not model your application conceptual model doesn't have reload in it reload is a database idea and it's an idea that not all databases even have so the controller if it calls reload has knowledge about the nature of the underlying data store so another reason to change query they often query the database controllers do which means they understand the structure of the database and and the way the data is organized they often present models and that could be something as simple as computing aggregate values from single columns or it could be something like group by where you're aggregating several objects they often contain view content so a flash is content pushed into the view to be rendered they often contain response content which is for example rendering json that is actual literal values that will be output to the client they often do routing not on the way in hopefully but on the way out conditional redirect is routing they often choose the content type and they often contain model logic which is the most sinful of all of these you should never have model logic in a controller in my opinion so the single responsibility principle says the class should have one and only one reason to change and this is the one reason for a controller to change which doesn't seem right to me that does not seem like good design and i'm not a fan of treating the srp or any principle as an absolute rule but as a guiding principle it seems seems to indicate something is wrong so the squishing rule which i propose here to you as a certainly a much simpler to an easier to understand idea than the srp is that if you take a big thing and you just cut it in half the small squares are exactly half the area of the big one don't worry if you take a 400 line class and split it into two 200 line classes probably the 200 line classes are better designed even if you split it poorly even if you choose the wrong line to draw they're probably still going to be better designed so that's part three of the talk we've so far had this minimal architecture for rails app and we've had this rule that i basically argue by argue by i don't know my own my own intelligence that if you squish something those pieces will be better so now i want to put these two together and consider uh the decomposition of a controller especially in rails app but really just the whole rails app what is it actually doing if you look at the things rails gives you this is basically what you have putting things like helpers aside things come into the route things go out of the template and in between you have a controller and a model and this makes it very obvious uh why there have only been two architectures that were popular originally there was fat controller skinny model and then it got inverted but now what there's no other option if you only have controllers and models so let's start decomposing and figuring out what is actually going on in here well presenters have become popular they're really just object decorators rails culture has this weird thing where it uses the wrong words for everything but i like the idea it's cool so let's put it in there that was something the controller used to do in 2006 let's say it was very common and now we pull it out into a separate object the model here is really we can really separate the reads and the rights and suddenly we've exposed boundaries in the system that were not obvious before reading is actually a boundary and writing is a boundary and furthermore you can actually do the reads in many cases based purely on the route so based on the action you route to before you call the action you could theoretically pull all the data out of the database that it needs give it to it let it do its thing and then when it's done write it all back it's a perfectly reasonable thing to do and it's sort of what the unit of work pattern is meant to do of course you need entities so let's put the entities in between those are the active record objects you can do the same exact thing for http if if a route is matched you can in almost every circumstance immediately pull out all the headers it's going to need the params it's going to need and pass them into the controller the controller doesn't need to know about them because they're for for 99 of controller actions the headers and params request are used are the same in all cases and then after the controller is done we could do a write a high level write so a high level http write would be something like redirect 2. that's a pretty high level idea it translates into a status code 304 i'm guessing and a location header but you don't say 304 and location header you say redirect 2. continuing on to in decomposition i've become confused what changed oh yes so when you pull out http stuff and database stuff from the controller you really just have logic left that's sort of what you have it's like a service class or it's like an interactor if you've seen uncle bob's talks about this kind of stuff so continuing to decompose uh there's a value being communicated between this logic this controller and the presentation so let's put that in here just to be clear that it's there and uh the http write can come out of that it's sort of straightforward obvious kind of thing and finally this is the really weird one but i have this thing that i'm kind of obsessed with which is routing is not just about verbs and urls it's about the state of the world relative to the user making the request so i really think that authorization is a routing concern i think that the router should i should be able to say to my router this url or this route matches verb x url url y and state z of the world and state z might be users and admin or user is authenticated so that's another thing we can pull out of the controller and you can see this is a heck of a lot more complicated than what we started with or at least it looks that way there's a lot more pieces but all these pieces are there these are all things you're doing it's just that if i color this differently the dark parts are what you normally put in a controller in traditional rails design if you do a skinny controller fat model it changes slightly but not that much so all these different things are going into the controller which is sort of unfortunate and people when i tell people this they have a hard time imagining how i would fix it what what what would the code look like if it weren't like this um so i made that code i made this thing with my friend tom crayford called the raptor web framework it is an experimental framework it's not designed for building applications it's designed for proving that you don't need things to be in giant ugly controllers or giant ugly models so here's a raptor application this is actually a complete application assuming there's a database layer for it to talk to this is sort of like saying um what would the rails equivalent be resources users only create yes only create it's like saying that underneath the hood it's translating into this this is what raptor's actually doing creating a route named create it accepts posts to slash user because it's in the slash user path the second line is what it's routing to this is a path to an arbitrary method in the system so there's no idea of controllers it just routes to some method and line three says if it succeeds redirect to show that's what you normally want to do on create right you make the thing you redirect to show if it succeeds if it throws a validation error then you should re-render new because there are errors and you want to re-render the form so in other words in raptor by saying the word create you get the standard rails create behavior that you would have coded by hand in normal cases usually you're going to override the target of the route so this is uh sort of like specifying the action you would run in a rails controller or in a rails route and it's just an arbitrary method path and then the method might look something like this it's in a plain old object no base class and it takes the params as an argument and then does whatever it needs to do in this case it's creating users so it makes the database record and then hits braintree which is a payment system to put a record in there as well now there's an important question that this should be raising in your mind and i hope it is but before i get to it so far we have routes we have routes to delegate to services and the services hit some kind of database layer but the route the the services in the database layer are your problem raptor doesn't doesn't tell you how to do those the question that should have uh come up in your mind though is what's with the params controllers need more than params sometimes you need headers sometimes you need current users sometimes they need cookies or session state they need all kinds of stuff there's like 800 methods i think currently on action controller base and raptor doesn't have action controller base it doesn't it actually doesn't have any inheritance in it whatsoever instead you get arguments but the arguments can vary so your one service the top service there wants params the bottom service wants current user and params and the way that this works is that raptor does in fact have an injector but is not a container it's not an ioc container there's no configuration and i don't mean there's no xml i mean you can't configure it there's no configuration to do it's purely name based so when i have a method that takes current user and params raptor knows params out of the box current user is actually one that you will define using a fairly simple method telling raptor what current user means and then whenever it sees an argument current user it'll pass the current user in raptor also has requirements on routes which are like the guards that i mentioned earlier things like authorization and they look like this i have a route update it requires an administrator only an administrator can do it what does that mean though raptor doesn't know administrator doesn't have auth built in well i define a class my app whatever your app name is double colon requirements double colon admin raptor will look that up based on the fact that you're requiring admin the injector will inject the current user argument because it happens to take it and then the match method will return true or false for whether the route should match or not so much like you can hook into the injector you can hook into the router and specify arbitrary high level requirements with this sort of conventional name-based interface and there's lots of other stuff about raptor i don't want to go on and on about raptor because i don't think it's very interesting and this talk isn't about raptor but raptor doesn't have controllers and it actually works as a web framework i mean we've not built production applications in it or anything but you can you can look at pretty much any rails controller and figure out how to decompose it into raptor and then it makes much more sense all the concerns are separated routing is in one place logic is in one place and the injectors tend to be sort of a junk drawer but they tend to have single responsibilities of course the real graph looks like this the injectors everywhere every time raptor calls your code it injects and uh it doesn't inject arbitrary method boundaries but whenever it calls you it injects if you want anything you don't have to use it if you don't want to so raptor is a real thing you can go to github and find it and download it's been there for over a year but once again the point my point is not raptor raptor is an existence proof that you can decompose applications into pieces that make more sense than just well the request comes in and there's a controller action that's my architecture i have a function it's not it's not much of a design the the point the the more immediate point that i hope you take away from this is that stateless services will help you to decouple your persistence from other bits of the system and you want that and simple third-party wrappers will help protect you from changes in other ruby gems or services ruby has a culture of rapid change and not a lot of regard for backwards compatibility so you really want to insulate yourself from that chaos in in the ecosystem and these two things i think should be on one side of a very important line in your application which is the rails line and on the other side are very simple active record objects and then rails and you want those things away from your services and your third-party wrappers because you want to be able to test without rails and you want to insulate yourself from the chaotic nature of rails over time to be honest uh the two to three transition demonstrates that things don't always go so smoothly when you upgrade so that that is the end of the talk i realize it's sort of all over the place but i hope you get something useful out of it as far as thinking about what an application does what are the pieces of it what especially what's a controller doing and what should a controller be doing so once again i'm gary bernhardt i own destroyal software which produces screencasts on things like oo design tdd and also all of this stuff basically uh this there's a series called sucks rocks that i took the examples directly from and it talks about exactly these things so thank you guys very much for for listening to me for 20 minutes any any questions hey hey um so basically i've been trying to use some of those uh paradigms to do it falls in in software um and i find that for some some things it works really well especially when you deal with pretty simple uh input output uh scale up values strings and so forth but oftentimes in rails we really do create a lot of qrad applications and it doesn't feel like it scales to that uh really because oftentimes you use you add a validation to your model which really just is a view concern often uh like putting in an error message and that just works and the just works part is the bit that drives rails people right and i feel that it's it's difficult to get there yeah it's um the problem with rails is that it is so opinionated that the more design you impose the harder it becomes and so um uncle bob for example has been talking about design and web apps and the design the sort of architectural vision that he he describes is quite complex there are a lot of moving parts and he advocates complete separation of your basically your application from the web framework like rails and in rails i don't think that makes sense i think that is that is far too much the basically pulling out services which is the core of what i advocate uh i think does make sense not all the time for example that validation case yes it is a view concern yes it's these sometimes it's a view concern yes it's easier to put it in the model and that's probably what you should do because if you fight that you're going to end up writing 20 lines of code to avoid writing the one line in the active record class so i i try not to take it to an extreme but as long as it doesn't hurt oftentimes it won't hurt like the case as you mentioned sometimes it really is nice and in those cases it'll be obvious that it's nice when you when you do it and that's when i think it can be really beneficial even if 75 percent of your app was built with services and the other 25 was plain old ball mud rail style it's a huge improvement we have one in the middle here yeah as pointed out prior about the validations a lot of the beauty of rails lies in the meta programming and dsls um was it a conscious this decision i mean of course it was probably um to go the same route or even drive that further because i thought the decomposition aspects were really clear and going into a layered architecture having some kind of domain driven design input in that um but i thought kind of that having a more clear like using the basic apis of ruby itself approach would also be beneficial for for yeah the basic api is like the core types yeah exactly oh okay um i'm not entirely sure can you give an example oh you've lost the microphone i gave it away um yeah um when you when you use the um dsls to define your your actions and you have the the injector which is a cool thing i guess um but um where you where you actually have everything in these dsl calls what what you basically do where you could rely basically on the libraries that provide output and stuff so make it more explicitly in the car so that you just use what's basically there and not define like a layer of dsl in between i'm yeah i i i'm not impressed by by the extreme level of sort of dsl obsession that that rails took as a design i mean if i were building a web app today i would probably use rail once the right so it's not as if i think it's awful or anything but um i think it takes it very far and in many cases hides intent uh there are a lot of cases in activerecord especially where if you just define a method it's like one more line and then you know exactly what it's doing and so that's why that's one of the reasons wrapper doesn't actually touch the database layer because uh it's hard to do that tersely without introducing lots of confusing stuff the only actual dsl in raptor is the router and it's a pretty complex one but other than that there's there's nothing it's just all methods all the way down methods with arguments i don't know if that answers your question at all but are we on time still good i guess so there's one over here far away from all the microphones everyone's waiting so nicely during my questions i don't know why why i'm special uh yeah i'm also concerned a bit about their magic router stuff uh what's about uh connecting it will be web machine instead that was machined by strong crips i know it has some issues with track and so on but like going this direction it wouldn't be the nicest way you think um they make sure i understood you're asking about um integrating something like raptor into yeah into like heroku deploying it um actually no to web machine but oh web machine oh i'm sorry i misunderstood um i have no idea uh um i'm not super into the whole async thing so i haven't paid a lot of attention to web machine yeah because i i really like how raptor is is designed by all the all the constraints and right and nice nice design but yeah the router is like very magic thing in there and yeah i didn't like it yeah it is very it is very magical and that's the thing the router had the most design put into it and it's the part i'm the least happy with i think it's real it's a i have a lot of respect for rails after writing that router um and ours is i mean the router and raptors i would guess uh 150 to 200 lines it's extremely simple actually i was playing with router and after that i don't have respect for routers for me it's useless thing i have respect for one machine right now yeah it's awesome um when we were so the the thing the thing that uh makes me want to design something like web machine is the fact that it bakes this the protocol in right it's the protocol is part it is web machine web machine is the protocol it's like one or wait am i confusing web machine with um no web machine is is like a direct implementation of http where state transitions are yeah um and we thought about doing that but we just could not reconcile it with what we wanted an application to look like especially the router and because raptor's router raptor's router routes on everything from like a verb an http verb all the way into model level stuff like you're an admin or not and we could not figure out how to fit that into a web machine style model it's it's like two fundamentally different ways of slicing the problem is encode the protocol directly or build your own thing that uses the protocol okay thanks yeah great presentation by you thank you i think we i guess we still have time no one's kicking me off one more i can't see very well i don't think we have one more all right thanks guys [Music] you