EmberConf 2024 - Standardizing Autotracking Via TC39 Signals with Daniel Ehrenberg

hi thanks thanks everyone I'm very very happy to be here it's my first Ember conf and it's great to meet so many people who I've interacted with online before and uh so I'm little Den I work at Bloomberg uh previously at at uh other companies but working right now on the JavaScript infrastructure and tools team which supports the Bloomberg terminal Bloomberg terminal is a platform for uh viewing and analyzing financial information so I've been contributing to tc9 since 2016 I've worked on features like big in and the hashtag based private fields and methods uh I'm currently vice president of ecma that's like less impressive than it sounds it's it's more like a board position and I I mean it's still it's a small board I live so I live here in in New York uh I used to live in the coast of Catalonia which you could see here the town has like very tiny Coastline kind of funnel shaped uh and that's sort of the whole Coast the town's all Inland uh so yeah I mentioned that Bloomberg's main product is the terminal you've probably heard of Bloomberg News but the the biggest product is this financial information terminal and this is basically a big electron app actually it's our own build of chromium together with some native code especially for uh Network and processing and we have so many problems that are in common with web applications because it is it is built on web Technologies and that's why we've been investing in fundamental Technologies like JavaScript and CSS so Matthew is asking me uh just yesterday at dinner why is it that Bloomberg invests all this and really the only difference from other other companies is because we've been around for 4 years because we've been maintaining this continuously available system and evolving it over time incrementally we really see the see the need to make sure that in five years we have a good basis that continues the the life of our code so um pretty aligned with what we've been hearing about Ember Evolution I think so we're we're talking about UI Frameworks that's why we're all here and I think the key job of UI Frameworks is is to keep the Dom up to date with the data or in another terms the view is a pure function of the model and words like View and model they're kind of considered obsolete that's like 90s o but it's just referring to the same Concepts that we're all getting at right now the view being what's on the screen and the model being the underlying data um so in embra this works out really well we have our templating syntax and we have our our JavaScript so the JavaScript is able to represent with Ember autot tracking it's able to represent the this language and then when the language changes it might change with this update language which is based on the the select then the greeting will change and this gets somehow rendered on the screen automatically so even if you have the same name uh the the greeting will change and Ember is smart enough to only update the part that actually changed so this is due to fine Grand pull-based reactivity so you consistently update only what's needed only if it's needed and autot tracking means there's no need to declare dependencies so that that works based on when Ember is rendering it just watches for whenever you read something this attracts fields and then takes note of those and then it kind of builds a graph so that when you need to uh so that when some data changes it can kind of Follow That graph in the reverse Direction and figure out what needs to be written back again to the screen actually it doesn't work like that it works in this opposite way so Ember ember's autot tracking is based on these clocks so there's a current revision number of what um like how up toate the data is and Ember uses this this mechanism to pull that and update when needed but saving work because of kind of smart ways of maintaining those numbers this is described really well in a series of blog posts by uh by Christen hu Garrett um who's formerly you know working a lot on Ember and another algorithm is more similar to what I described it's kind of you could call it like the push dirtiness pull values algorithm so in either case when you update a signal the not calculation is done immediately so imagine you have a as a tracked field and then maybe B and C are other cached Getters eventually f is like another cached getter when you change a how does that lead to updates happening later one way is that you increment the you know you take a new clock number and then uh when you read F then it will uh see okay yeah F's clock numberers lower than a is so I'd better go back and and calculate things that's completely lazy the semi laaz way is when you write to a then you get into this first part where you dirty B and C you color them red and then when you read F later then it starts to go through these later stages of okay uh I'm sort sorting things in topological order uh it's a slightly complicated algorithm but it's not that complicated uh the whole point is to make sure that we have consistent calculations when sometimes this is referred to as the glitch-free property and although there are multiple algorithms they have the same observable behavior when I say observable I mean uh the JavaScript program that's running you know your application can't tell whether it's using the clock algorithm or the dirtiness pushing algorithm because they just do the same thing and we so to me this reminds me a little bit of uh this process of carcinization so many different crustations have independently evolved to have a crab shaped body because it turns out that just works out well for some Lifestyles so uh you know porcelain crabs pictured here have came back to that to that same idea even though they're not very closely related to Common crabs and okay here's my heterodox opinion to be analogous to what Yuda said I think all the Frameworks are becoming pretty similar they're becoming the same and that's a good thing because really how complicated can reactive rendering really be we're just we're just kind of figuring out the answer uh so this a mix of convergent evolution like the crabs and sharing ideas directly there there's sort of both happening uh convergent evolution really was a you know demonstrably a part of this because over the past 10 years you could see as different Frameworks updated their reactivity model they got different parts of the answer right at different times it's not like solid came up with signals and then everyone figured out how to copy them and they took time to come around I mean Ember was coming up with this stuff at the same time just with different algorithms and some of the things emper got right earlier like about signals computed signals being garbage collectible but um don't have time to get into that right now so a signal been using a word without defining it is basically a tracked cell so uh this is like when you have a a class in Ember and you have a field that's tracked it it did take some time to to figure out but all these different Frameworks ended up coming to basically the same conclusion now we're trying to share our common answer answer so here's how we here's how we put it into an API that's now proposed for standardization in tc39 this is you know pseudo type scripts because you can't do class signal. state but whatever uh there's a class that's just a signal so this is just like a trct field and you can have an initial value and you can get and set it and computer which is like a cashed getter which takes as an argument a call back uh which doesn't take any arguments and you can you can get it there's a number of other apis that we ended up needing uh this is the subtle part so when when you're programming you're usually either making a track field or making a a getter which might or might not be cashed but Frameworks need some more subtle things to be able to integrate this and into their rendering to figure out when to render and which parts are the the relevant parts so we have some of apis for for that purpose that we're iterating on trying to be minimal so to to visualize you know the app developer will use maybe some nicer skin over the computed and state apis then for when things render into the Dom or when other side effects are supposed to happen uh Frameworks would provide some effect API or some template syntax that would end up using these other subtle apis and this allows us to build framework independent reactive models so you know kind of not going with exactly what Yehuda said before what you know I like classes but if we want to present this to the whole rest of the world we have to we have to live with you know we can do this with objects also this works uh this is actually how yehud and I together with Json from from the Google whiz team were presenting signals to tc39 if you have a little data structure which might have Getters for um so imagine this is a counter and you have a button to increment the counter and you could also see what the current value is and whether it's even or odd so this gives you an object that represents that with signals for this reactivity this could be used for example in preactor view this isn't this isn't Ember code but it it looks so similar I mean you just have a template you have things filled into the template you have an event handler um and they both you know this template is where kind of effects come in so when when the when preact is is rendering this it will call this function which will you know read parody. count and that'll access this state so that'll be you know autot tracked it'll also call parody which calls these even and uh and parody computed causing them to be evaluated uh initially they're they're uninitialized so this is represented as kind of red but then once they uh once they all evaluate which Happ happens you know immediately when they're read everything about signals is synchronous uh then they um they have this dependency graph that they're able to construct and they have their values when you click on the button the counter increments its state so now these these diagrams are colored more according to the solid algorithm than the Ember algorithm it's kind of easier to visualize um so um pract can see and it sees this in kind of an atomic way in the pract case because it it ends up having to reevaluate the whole show counter function because of how it uses the standard jsx transform anyway it sees okay last time I ran show counter I was reading counter. count and counter. parity and those are both uh stale so so counter becomes uh stale and then walking the dependency graph backwards uh all those other things become stale and then so you have more than enough reason to render show counter uh but then it can re-evaluate them all and and get to a steady state but a cool thing about this is that invalidation can be granular uh so this because of the way rendering Works in preact it doesn't actually allow the re-rendering to be granular but it does allow us to not have to calculate this super expensive turnery operator so in this case uh you know you click plus two so these things all become red uh but the um I guess we kind of skipped some steps but uh is even in parody because it looks like they're stale but then when we get to calculate is even it gets the same answer again so then there's no particular reason then parody can become revalidated by the system and this actually works in Vue just the same because view uses a declarative template syntax like Ember uh it's able to break this down further and only rerun the the individual parts that um that change so imagine that count or are passed into some helpers it would only have to rerun those helpers if uh if it's relating to the the the part that changed so when you do plus two becomes initially it looks like it's invalid but then you know it didn't really have to do the parody again so let's talk about why it bother with standardization or what does standardization even mean so at a at a social level what we want this to have an impact on the the ecosystem and this really needs to start with the ecosystem so one model from 10 years ago now is the promises A+ project this was a community-driven effort where several different promise libraries uh got together and said how can we unify our apis and able enable them to be compatible with each other so they came up with a common then method and and catch and and Define what they should mean this became integrated eventually into the tc39 promises standard and was added to Native JavaScript uh so we want to take some things from this but also learn from some of the lessons uh the Ember communi is and the only one that had trouble upgrading fully to Native promises many people did including inside of blumberg uh and so we're trying to what what we're really doing is finding this common base of reactive semantics but then enabling different ergonomic skins on top of it from different Frameworks uh the other thing that we're doing is first very similar to promises A+ UniFi trying to make sure that different actually used libraries and Frameworks can adopt this system before trying to standardize it um but because of this second part of different systems having different skins on top there's absolutely no changes that applications need to do to adopt signals so signals would be something that underlies ember's reactivity mechanisms but would not be necessarily how you program in terms of it I mean of course if it exists I'm sure people will reach for it but there's uh there's nicer ways programming models that I'll that I'll get to soon that you can use interoperability is a major thing so I showed how this model can be reused and maybe you even have a model that you use in multiple ways on the same page so um if you have a you know micr front ends with a common data layer then maybe they're accessing the same uh the same structures and they they become invalidated hopefully this could allow embedding widgets inside of each other and I think uh standards could give a common reference point for this kind of sharing one thing about autot tracking in particular it relies on having a global variable for what is the current computed signal or what is the current uh you know cach getter so that it can track what the uses are and Associate them to that cash getter and uh with with the way the JavaScript ecosystem works it's a little bit hard to compose different things it could be nice to have the JavaScript language be this common reference point so that the so that you can make sure you don't have different Notions of what the current uh computed signal is in different points in the program implementations that are native to JavaScript engines might also be higher performance I don't expect this to be magic they're not going to have algorithmic differences I believe but inside of C++ you can get a constant Factor Improvement by using more efficient data structures uh and for now we're working on an efficient polyfill in javascripts or polyfill just an implementation it's just a JavaScript library of signals and um trying to make this efficient not not there yet though signals can also help us pull effort on important things that span uh multiple Frameworks for example Dev tools with reactivity it can be hard to debug kind of causality what what caused what to happen uh the uis that you might want to make in Dev tools are are kind of complicated and right now each framework has to come up with its own way of visualizing this but really we're all doing the same thing so it would be great if we could work together on this whether it's through an external Plug-In or in Native Dev tools so how does standardizing a JavaScript Feature work this is done through ecma tc39 so Emma International this Geneva um organization it used to be the European computer manufacturing association but now it's like KFC it doesn't stand for anything in particular uh and tc39 which is originally the 39 technical Committee of Emma we have our our website we're focusing on speci javascripts and that's just another website so the the standard for JavaScript is just a a web page which we develop in our um own kind of markdown dialect called which we call E markup and um it's just a very very long web page don't print it out it would be over a thousand pages long uh and it it describes everything in JavaScript in in thorough detail so it's unambiguous actually um One engine author I don't know if you've heard of lib Js from the serenity OS Project a newer JavaScript engine he was uh lenus grow my cooworker at Bloomberg who who's um you know worked a lot on lijs was saying that compared to other language standards the JavaScript one is really unambiguous really tells you everything whereas with python or Ruby it's not quite as much the case anyway that uh that preceded me there have been a lot of editors over the years who have done a really great job making the the standard precise and we we work by a process that's kind of similar to the Ember RFC process uh we have our tc39 process and there's actually been cross pollination in both directions uh the tc39 process was via Yehuda inspired to take some elements from the previous Ember RFC process and then Ember had its reform of the RFC process based on some of the things we figured out in tc39 and we have four stages for how standards work at stage one an ideas just brought brought on the table to be under discussion at stage two we collectively decide we want to do this we have a first draft we have some concrete ideas uh and as a committee we agree that this is a good idea for the language at stage three we have a pretty final draft we have tests so that when you implement it you can just run the test to see if your implementation works and we think it's kind of ready to go maybe some browsers will ship it at this point and at stage four we have multiple implementations and at that point it's standard uh when I say standard then we land the pr it's in the editor's draft and then it'll be in the next year's es 2025 whatever addition so some examples of this signals are currently at stage one ASN context which is a a feature that I think would be really useful to underly things like the the Ember context initiative um is at stage two decorators as we saw in the the Ember data talk are at stage three and uh private fields and methods they're at stage four they're already part of the standard so decorators we have the complete spec that's why they're shipping in typescript um asent context people people think it should happen we have we have a strong idea but signals they're just an idea hope hopefully it'll happen I hope it'll happen but uh we haven't yet agreed as a committee that it's going to happen and tc39 works by consensus we don't votee on what the language should be uh we try to work together to meet everyone's goals so we ask for objections to consensus and uh this this really helps maintain a sense of equality within the committee which is important for our values and it also helps maintain a conservative default of we don't want to add things that are not ready yet because we have to maintain a very strong sense of compatibility over time also because there are many JavaScript implementations and it's a lot of work to implement something so we don't want to ask implementers to do something that's a bad idea so uh couple months ago Yuda and I together with with jatin from Google uh presented signals for stage one at tc39 and we got consensus on stage one it is on the table that's that's all we could say at this point from here from here uh there's a lot of work that we could do towards prototyping signals I want to make sure that before we consider adding signals to the browser that we make sure that we're doing the right thing and we could do that by work at the ecosystem level in particular at the framework level so the first step is having an implementation of the you know the signal Library itself which uh I worked on with some co-workers at at Bloomberg uh Dolan Mew and Eric yans so we have the the Bloomberg copyright header on the the rapper but actually uh this is just like a 200 some line wrapper around angular signals angular and and whiz were already doing the the right thing so we were able to implement this just by reusing their their thing I think there are many signal implementations that we could have used so this is on npm you can use it but it's not ready for production it has it has bugs the API is going to change but I encourage people to play around with it if they're interested but another thing is just having these atomic cells is not really the programming model you want to be using all the time and Ember and uh figured this out with the track built-ins and track toolbox and um also present in starbeam so noox populi right here in the back uh ported these to signals with signal utils which include an at signal decorator which because they're using stage three tc39 decorators you can use it even on a private field uh as well as signal object which is like capped objects so also on npm uh should get that download count up it's had a spike now almost nothing uh so integrating signals into Frameworks is also good again this is something to do in development branches it's not time yet to to ship all this stuff to production but uh I'm really grateful that Alex lifeart has uh done this sport on a branch of glimmer next uh and found immediately that the part of the code that I wrote this Watcher unwatch method is really really slow uh so if anybody wants a contribution opportunity you can PR PR is welcome to fix this uh also found a bug where just some simple thing crashes uh so thank you so much Alex for filing these issues uh I'm also really excited to work with Chris thurn on uh integrating signals into warp drive another possible contribution opportunity uh so just want to thank the the many contributors from the Ember Community um and um excited to continue working together so if you're interested we actually need need your help there's a there's just a lot to do so please join the effort we have a Discord we have uh calls every couple weeks that you could join if you want to chat with me and the other signal co-champions that's what we call the the people pushing forward the effort I mentioned like 10 Frameworks and there's basically somebody from each of those who's you know taking responsibility together for the proposal happening so it wasn't wasn't just Ember it was also people in solid spelt uh view everything working together to make this happen uh there are polyfill bugs you could write tests there aren't enough tests integrating signals into applications would really help uh prove them out and filing issues you could even join Emma in tc39 so that's um please just get in touch with me if you're interested in doing any of those and U I'll I'll be around afterwards also I'm on the Ember Discord so you could just write me there and uh thank you very much than