EmberConf 2024 - Closing Keynote "Our Engineering Values" with Ed Faulkner

so it's been a long day of talks I'm going to give your brains a little bit of relaxation and change we're going to do a little puzzle um it's going to feed into the theme so first just take a look at the puzzle think about the puzzle and we'll get back to this puzzle so um my actual title side is this so I'm going to talk about the engineering values that make our Community our community and uh you know how they're applying where we've done them well where we're where we need to do them even harder keep theming harder um so that previous puzzle is an example of a Sudoku and sidoku is a very popular game um I'm guessing most people have probably heard of sidoku and seen the rules of sidoku but um the rules are simple easy to super simple to State I place the digits one through nine exactly once in every row column and box right very simple bunch of constraints but it leads to a lot of complexities and honestly at first I didn't really get the appeal of Sudoku because I felt like as a programmer my mind jumps immediately to I could brute force that solution like with program why would I spend my time trying to solve that puzzle um but after after a little bit later exposure to seeing the clever ways people solve the puzzles I got a bit hooked um because what you find is there's a lot of really elegant patterns and chains and Fs that people can build and it's just kind of you know puzzles are fun I think a lot of programmers like solving puzzles um so the appeal is you don't have to bruteforce it and um you know you get to find clever connections uh the kind of them kind of tickles the same thing in my brain that I get when I'm solving hard programming puzzles but um one of the things that annoys me about Sudoku is that when newspaper editors tend to say like hey here's our soku they're really careful to tell people don't worry you don't need to do any math I and I that pretty ironic because I mean they mean you don't need to do any arithmetic but solving p is actually the most mathy thing a lot of people get to do because a Sudoku solution is really a proof right you're constructing a bunch of inferences through a bunch of steps and as long as every step is sound you know you got to a good answer um so getting back to um the big puzzle I started with so people who get really into Sudoku might get sucked into the world of variant Sudoku and this is an example of that and these puzzles can appear absolutely baffling the first time you see them there's just nothing to grab on to um but there there is a guiding principle here these puzzles are not randomly generated they're lovingly handcrafted and they usually leave a trail of things for you to find um so there's like a hidden method to the madness and there's a given take between the puzzle makers and the puzzle solving communities as they like trade techniques and get familiar with patterns and then have to one up each other and it's it's a fun thing so um so in this case the this is called a zipper line puzzle and the rule is very simple right it's that as you can read up there um I'm actually going to let's see let's just actually to show you the quick Breakin of this puzzle let's do it together I think it's easier to do this live all right so how would you start this puzzle so we're going to pay attention to uh oh I'm not looking at the same screen as you all right I'm going to do the pre-made version then I have a little screencast I'll just have to narrate at the right speed okay so we notice this Square Right Here Right Now by normal Sudoku rules we want to ask where can it go in this box and it can't go there because it's going to collide in the column and by the zipper line rules right we know that the squares on the zipper line add up to that Central Square so it can't go on its own zipper line so when we look at that box we realize it can't go there it can't go there it can't go there so it has to go there right so we've identified two places where the same number goes and then we do now we're going to move on to the next number here the blue one and we can use that same logic up here we say it can't go there it can't go on its own zipper line it must be somewhere up here we don't know exactly where and that's enough now to rule it out of that whole column in the bottom box so we know it's not there we know it's not there it must be somewhere here and then how about this third colored number call it yellow now we know that yellow has to be bigger than blue because blue plus something equals yellow we know that uh and our blue was already bigger than the red because red plus something equals blue now we're doing the same trick we're repeating it one more time and we're going to decide that yellow has to be somewhere up there so if yellow is on this final zipper line that we're calling green now we know that green has to be bigger than yellow yellow has to be bigger than than blue blue has to be bigger than red and what is the small what is the smallest thing red can be um well red is the sum red is is the sum of each of these different pairs of digits in that box all right so we've got five digits and what's the very smallest thing you could put in those five digits they have to be unique because they're in one box be 1 plus 2 plus 3 plus four plus 5 the fifth triangular number that's 15 we got one more digit on the end to add in so it's at least 16 and that's three pairs adding up to that same zipper number so it's got to be at least six and so therefore we know this thing's at least six but it's got three numbers higher there's only one answer it has to be six come on screencast explain one more time this is why I wanted to do it live we're almost there come on give me the payoff I was too slow when I recorded it okay yes so we know the red is six we know the blue is seven we know the the yellow is eight and we get the nine right so like each of those individual steps when they're explained to you when somebody explains them to you they um each step by itself seems pretty simple right like once somebody shows you you can follow along but knowing where to look is where all of the art in this thing kind kind of thing happens right the art isn't knowing where to look and that really just comes from practice and learning from other people and uh training techniques so I think the analogy to the kind of work we do is quite strong uh like well-designed software abstractions they look really obvious once somebody shows you uh but making them can seem much more mysterious especially when you're starting out right and it really just takes practice and immersion in a community of people who are solving the same kind of problems and practice in the sense of practice makes perfect right and exposure I want to make I want to draw one last analogy from the world of Sudoku because I think it's also interesting so even Sudoku has controversies um and one is called the uniqueness controversy right so the idea here is that the rules of Sudoku don't explicitly say that there's only that there's a unique solution to the puzzle now there effectively always is because it's just bad style for the puzzle maker to give you an ambiguous puzzle but people disagree about whether you're allowed to use that knowledge to solve the puzzle and because of the thing about unique so in this example um these little blue digits are supposed to show you the possibilities of those spaces it's a note saying one of those three numbers fits in the box that says 137 but if you picked a three right that's going to force the other two corners of the square to be sevens and the last to be a three and you'll have two sevens and two threes and the same would go if you pick the seven but in either case that whole thing is so self-contained it doesn't affect the rest of the Grid it's becomes an ambiguous pattern and so knowing guessing that the puzzle maker made a um a sing solution puzzle we can actually just jump directly to the solution and say that square has got to be a one that's uniqueness now some people don't like uniqueness because um it basically gives you if you don't assume uniqueness and you still solve the puzzle you get a stronger confidence you get a stronger more stronger solution and people like the Aesthetics of that you basically proved there was one solution and had there not been you could have actually caught it instead of just like using that extra knowledge um so I drawing here I'm drawing to your attention here is the idea that sometimes um it can be beneficial to not rely on some of your knowledge right sometimes compartmentalizing away something that you know and acting as if you don't know it can let you get a stronger result than you'd get if you rely on knowing the details that you know and that feeds directly into what we do as programmers and it feeds directly into what I see as like part of the core engineering values of this community so one of the hardest things in programming is knowing when to not know things right some of the most effective implementers struggle with this because I've met people who have brilliant memories and like encyclopedic knowledge of how things work and sometimes those people actually struggle because like other normal people can't keep up with them we don't know how everything works we need abstractions and uh like they actually can people who see too much of how everything works can struggle with that sometimes it takes real practice and um of course it's useful to know how things work inside but knowing when not to rely on that knowledge is really important and it takes a lot of practice and you need frequent you need to frequently exercise the mental gymnastics of jumping over the abstraction barrier while and back again to to put yourself in the mind of someone who doesn't know how it works inside but also jump back over to being the person who does know how it works inside so you can go fix the bug inside right so the alternative to knowing how things work when you're in the other mode is knowing what they mean right um so ha you thought this was a programming talk but now that I'm between you and happy hour we're going to do philosophy all right but stick with me because this really is actually about why we as the Ember Community are here right where does meaning come from right to oversimplify thousands of years of philosophy right an eternalist is somebody who thinks meanings are fixed and clear and permanent and a nihilist is somebody who thinks there just are no meanings and everything is meaningless and and they're both wrong uh I I think meaning emerges from a consensus among a community and I think for an example of what I mean by that is just consider our our words right words do exist they do convey meaning uh and new ones get created all the time but you can't just make up when whenever you want and expect it to do anything right it really is an emergent process between not just one person it's it's a it's a community effort when we construct meaning all right so our abstractions are like that right what does this code mean we have a shared understanding of what this means right in in Ember this would be a component invocation right how does it work some people here will know and some people won't and that's that's good actually right we have that whole Continuum and there's multiple levels to understand how it works but even when you don't know you can have that shared understanding of what it is right so this is a very small and simple example but the general value here pervades how we try to think about software right what does that do for us what do you get by focusing on what it is and not what it does for one important example like what's a breaking change versus a bug fix and if anybody ever argued about that right so the nihilist here would say that every change is a breaking change because there's just the code and you change the code and it doesn't do the same thing I love that xkd CD uh comic about breaking changes there's nothing you can touch that won't affect somebody right but when you've built a community consensus around meanings around interfaces around what things are and not just what they do right now um you can have consensus on what is a breaking change because a breaking change is the one that changes the meaning sometimes you want to change the meaning but some a bug fix is one that leaves the meanings alone right I meant it to be the thing that it still is but it had a bug before if you're looking only at the code there's no objectively measurable way to reliably detect the difference between a breaking change and a bug fix right the computer only ever sees the code where's the meaning the code the code feels solid and concrete and knowable and the meaning can sometimes feel more squishy and nebulous and human but meaning is very real and it lives in standards it lives in docs it lives in rfc's and it's passed from person to person all that other stuff that's not just the code is where we do some of our most valuable work right it's it's the clarity of our meanings that makes things like non-breaking changes more possible and it's a central value of our community that we care about that work and we do our best to we aspire to do that work really well right um because all of our power and software comes from Leverage the ratio between that interface that meaning and the implementation right to build ambitious applications you need to control very sophisticated implementations with the smallest simplest interfaces possible for the highest Leverage that really requires a high degree of trust in your abstractions and that's only possible when you have durable and precise and consensus-driven shared meanings so even when you decide to phase out an abstraction that you've been relying on that migration is going to be far easier if everybody has a crisp understanding of what that thing is right if you have a a spec for it and not just an implementation often the shortest path to removing something that is underspecified is actually to do the work to make the spec for it that's how you know how to replace it how to Cod mod it how what is the replacement strategy so many of the successes that our community's been a part of are examples of when we've done this well right and so the Ember Community was there in the middle of things helping standardize Promises End classes and Es modules and decorators which are really nearly there this time and signals which we got to hear about from Dan um these are all examples of these are high degree of success examples this is where we didn't just make an abstract for our community but we were able to participate and make something that is now like broad enough that permeates all JavaScript and uh people can actually just take it for granted which is the highest complement of an abstraction is like people don't even have to think about it anymore use yeah and we've also got examples that are more of the within framework abstract like the durability of our templating language even as we've evolved it in its syntax has been you know a really strong constant um our tra reactivity primitive is just extremely nice building block for all of the things we've talked about in terms of reactivity and also I want to highlight our manager apis which um have really enabled big leaps and functionality without breaking compatibility and I I'm going to zoom in slightly on manager apis because this is a low-level thing that you probably don't have to deal with and that's a good thing but the fact that they exist has been a really big win and it's a pattern that um we've used successfully and want and should use even more so the idea of manage the examples of manager apis we have is component manager helper manager modifier manager this is the low-level but public and thoroughly specked out API for how like what do the framework consider the what is a component what is a helper what is a modifier um the reason it's worth designing those things is that because they're low-level apis you get very different design constraints um because they're lowlevel apis that are really only called by the internals of the framework like day-to-day ergonomics of them is not nearly as important as their flexibility and stability right so you can design something that's verbose but really thorough you could design something that has room to uh like version itself Express capabilities you can do you can focus on the flexibility and for example our modif our manager API are very functional programming esque which is like ideal for flexibility and sometimes has trade-offs in terms of ergonomics but that's perfect for these internal apis right we maximize the flexibility of them um and don't have to worry about the ergonomics because it's not what you're typing day-to-day in your application right but it's a critical API that it exists because it's what lets us um first experiment right it lets us Implement new versions of core things um in ways that are stable you could use them in a big application and not worry that you're going to break because it's all built on uh stable public apis even if they're lowlevel stable public apis and then once the community does get consensus around the new thing that you've been experimenting with it means that you can roll it out very incrementally it will it's going to get guaranteed in operability so for example like when we rolled out glimmer components and redid the component model those interoperate with Ember components right you didn't have to worry about doing them all at once um or for example that you can just use plain fun functions as helpers in templates that's a great feature the helper manager API makes that possible you don't have to worry that it's not going to compose like if you put two helpers together and one is the old way and one is the new way they're going to work because they're both sharing that common low-level API um so this has been this is a good example of a pattern where sometimes you do you didn't have a spec for a thing you just did the first version It's implementation defined but you can capture a spec after the fact codify it as a manager API and then now that you have a spec you can actually iterate as a community against that and do the new highle feature um so um so those these are examples of successes where we've we've risen to the aspirations of of actually building interfaces shared meanings that are not just implementation defined we also have cases of course where we don't always succeed hitting the level we want to and we've got things that are implementation defined and don't have clear interfaces and clear specs so some examples of those would be addons as historically defined V1 add-ons as we're like calling them now that we have V2 addons is an example of that the idea here is that this they're extremely overpowered they can do almost anything there was not really a clear spec for them uh they have a ton of extra hooks that just got added over time so that they're very implementation defined to know everything an add-on can do you really just have to read all the source of Ember CLI um so that's an example of an area where we're uh not hitting the goal yet right I think engines is another example and this one was interesting to me because I went back and looked at the RFC that introduced engines it was only the 10th RFC we ever did so we were still getting good at this process and in the RFC actually it says explicitly it would be really good to make public API defining how these things get built but we're going to do that after we get the implementation in Ember C stabilized right and I think I think knowing what we know now about our process we would we wouldn't do it that way this time we would actually work together on first having the stable lowl apis and then worrying about letting people iterate on top of them to make the high level apis and not the other way around and so we've what we've ended up with is something that's very implementation defined and so it's slower to evolve and harder to change without breakage um the the boot process that emberli uses to boot your application is another example very implementation defined similar to the other things I've talked about um we already heard earlier today about things like the Legacy patterns of adapters and serializers never data these are ones where a lot of features were added over time and they were kind of like the what I I would use a shorthand to say as like the the bad example of O inheritance right it's just like there's a lot of surface area people wanted features people added them you're just supposed to know when to call Super on which things right um our AMD loader leaking so this is the idea that require and Define are things that you can see later around an EMP application even though we've been we helped write the module spec we've been on modules for a long long time we leak the fact that we don't have modules and in a way that was never really like specked out and and and clear when we could stick to the very good es module spec um and I'll kind of put under a bucket pre octane router patterns as well like a bunch of things that the router does that if you were redoing it post Octane and post tract and all you would do it differently right with different lifetimes and more interact more uh use of tract and all that right and so these are examples of places where we haven't yet risen to the bar that we aspire to as a community right and so the wow that was supposed to be overlaid on the previous slide that did not work oh it's going to build in again isn't it haha this is what I get for adding to my slides in the back of the room huh no keep coming so the takeaway of this next one is going to be all the things you've heard about today are things that uh map directly into these areas all right so embroider initiative no I hit the button too many times all right embroider initiative is going to help us is addressing the add-ons right um and engines because we are we have several folks who are working very hard on that who care a lot about that part of the problem we're making sure that Maps into the new world and the modules First World um the boot process of Ember CLI exposing it stabilizing it codifying it um of course we heard about warp drive and how that is you know taking taking care of those Legacy apis on the am data side um our AMD loader as a leaky API that is not fully specked as a stable thing partly embroider takes care of that but also the upcoming strict es modules feature which is currently a draft RFC that is going to that takes care of it even in classic builds if you're not on embroider yet we're going to be offering a fully no AMD loader everything's yes modules the cookie there that uh the incentive is that JavaScript top level of weight would been work everywhere in everybody's apps and um and finally the preac router patterns um the solution here I think is really all about doing more of what we've done successfully like the manager pattern for so here I'm s suggesting route manager and router manager as the implementation strategies that are going to unlock that stuff this this is recent discussions that came out of this week this conference's Associated discussions with the core teams as we're as we were looking at what is the like what is the quick wins that we can ship on the routing part of the story for Polaris um the idea of Route manager is just like component manager let us do iteration of what is a component you can use route manager to iterate what is a route and once you've stabilized that now you can iterate on it and everything at the individual route level would interoperate so you don't have to Port them all at once if you have a thousand routes do one new one and try it out or do 10 new ones and leave the old ones router manager would be at the next higher level to swap a whole router system which would be a later you know the followon step um so these are examples of you know Reasons I'm very optimistic about our community's path over the past year we've seen a huge burst of investment from a lot of different Ember supporting companies and individuals and a lot of that effort is pouring into these really high value areas where like we're really making strategic choices to invest in the really high value stuff which is you know establishing those shared shared abstractions uh shared understandings and trying to do that work right so um I'm really happy with all the progress we we're seeing it is moving very quickly so the takeaway is that our guiding principles they work and we just need to keep embering harder right so uh that's the end of the prepared slides I know some people sent some questions I'm going to try to use some time for that um so we got asked what's the status of app V2 so the blueprint that Chris Manson uh mentioned is effectively that um if the question is when is it RFC as like the official app it's like going to be a question of more people have to use it first and stabilize it um but that's that would be where to look and participate and try it and give feedback on it um the um so the the idea would be that AB V2 is really where similar to how we've already had V2 add-ons for a while and that was intentional because add-ons are really where you you know if add-ons are are holding you back you have a lot less control than things your apps are doing it's where ecosystem wide concerns are really high we really have to hold libraries and add-ons to a more a stricter standard than what you do with your own apps because if your app is is using some feature that turns out not to be great like of course you want to avoid that but it's a bigger problem when it's spread across 15 different packages that you don't control right so we did add-ons first uh but we never really did V2 apps and so this is the idea of like committing to some changes not gigantic changes uh to the app format itself I would summarize them if you heard my Ember talk last year I talked about like categorizing the cost of change these are 01 changes o or cost or one changes in that even a big app is going to have a fixed cost for it so it's about having different dependencies in your package Json a slightly different format for your index.html um and not much else it's not that and there's nothing you'll have to touch in every file it's very important to us that it doesn't scale at that cost um so that that's the the V2 stuff um what are the motivations between between behind hinted at routing changes so I think um a lot of folks can probably think of things that are irksome or um or foot guns in the router that we have for example um had we done it now if we if we' built it now something like your your routes class there would be something with the right lifetime an object where if you used ember's register Destructor lowlevel apis on it it would actually you know come to come to life when you enter a route and then when you finally leave that route get destructed there's no such object with that lifetime like routes have a global life time that's not quite correct that's an example a foot gun that people have had for a long time is model hooks don't always run you got to know about that did you transition to a model or or an ID right so there we have a list of things that we know over the years we would like to change and it's been an area slow to evolve partly just because of prioritization and stuff like it's an area I want to get to but I think inverter is more important for me finish right now um there's a lot of other little things like people have a big wish list of what they want routers to be and that's also partly why like it's easy to make a really long to-do list and then make it take a long time that's partly why I'm excited about you again using our manager pattern to unlock that stuff to make it to stabilize here's how here's how routes hook in show us what you think routes should be uh let us iterate on them together and then ship them so they'll all interoperate um so that's examples of that um had a question about uh life Hearts glimmer next I I think it's great technology demo I think um I think there's definitely stuff in there that could be upstreamed and stabilized but I I would site back also to the what I was saying earlier about it's not just the code that we need right we have to also go through all the other steps to figure out how do you do it together as a community because if it if it works for 80% of the cases but then it then the other 20% is going to tell everybody to rewrite their apps we're not going to be excited about bifurcating the community so I'm sure we could do the work or I think it's likely you could do the work to bring some of those features across or all those features across but the important thing isn't just that like it can run it's that we do all the other steps as a community to make sure we really have stable shared specs and stuff like that um which features in Polaris are fully ready for adoption today as they would ship in a coherent addition I think that that is subjective and people disagree um I can tell you that I am very much using template tag and glint and Embers built in typescript all together in multiple production applications and I've seen teams doing it very successfully um not without some early adopter hiccups and certainly you have to accept that like the docs and guides aren't telling you to do it yet because those are quite are correctly waiting for those features to be at a higher degree of Polish before we say call them recommended um so if you're accepting those caveats those are all very adoptable but definitely there's polish issues uh to go through I think they're rapidly rapidly getting better particularly template tag like in the last um six to eight weeks like has fixed a bunch of the final things that I knew about that were pain super painful um and uh same goes for like V2 add-ons we just closed a long-standing we've been knocking down the dominoes of things that were blocking people Ember data for example going to V2 which just just became possible like last week we knocked the last Domino and they did it right away so it's like lots of good progress on that um let's see Yehuda hinted at server side data loading instead of serers side rendering is there something being cooked up allaha fast to provide this so the idea behind that um I don't have a lot of detail on that but the the general idea would be um it's not fast boot because fast boot is actually the full rendering layer and the theory is that you get a you get most of the value with a lot less headache to not necessarily do full service side rendering but to run just the data fetching part server side or and by server side it could also mean you know Edge compute or one of those things um by separating those and our our data load is already pretty good at being separated from the rendering layer that's part of why routes are not just components they are distinct things with distinct phases that's a good thing to preserve and to improve um so that's the general theory it's not it's not a replacement for server side rendering it's an alternative to it that might be a sweeter spot in terms of the benefit to cost ratio um with that I am over time so I'm not going to do more questions thanks everybody this has been a wonderful emerc comp I'm so excited to be here with you very optimistic for all the energy in this community thank you very much