D3 with Canvas - Kai Chang

thanks Irene see my talks are on plugins on the blocks page is d3 in canvas here I'm gonna go fast so I don't know if you'll be able to read everything in this slide my talk is a little bit of a split personality it's a little bit of tips and tricks and some about learning d3 and programming in general we're talking here about pixels and sub pixels which are I'll know have you ever seen one of these since the iPhone 4 came out because they are so small I think a child nowadays may not even be aware that you know displays are made of pixels I mean nowadays there are probably billions of pixels in this room and think about all the computational power that goes into driving those pixels this is a picture of this display here it's a macro picture and let me zoom in just a little more there's a real photo of this computer this morning you know this is what pixels look like they are red green and blue usually they can come in a couple different layouts this is it IPS is a kind of standard layout and you can control each of these you know red green and blue values individually and once you have enough of them and they're small enough you don't see red green and blue anymore you get color mixing effects you know it happens on the surface of your eye and your retina this is an image if you just pull it out my eye and slice it in half sorry this way slice it in half you know the retina are these cones and cones and rods cones are the ones they're really a sort of peak in the center of the eye they're sensitive to color color and shapes and then we have this sort of layer of processing bipolar cells ganglion cells that actually produce a lot of do a lot of visual processing right there on the eye before the information gets passed through the optic nerve through the optic chiasm and the visual area of the thalamus visual cortex I mean our eyes are directly connected to our brains and the images that go into our eyes you know they just reverberate through your whole brain so we want to control these pixels and we want to draw things on the screen you know they give us ideas and in order to do that we're going to have to deal with computers and programming languages and all this stuff that we've created I mean there's this I mean it's really that you know the photons in this reaction that's happening on the eye and the ideas in our brain that's what we care about but we have to pick something this fog is about canvas it is it's a API a JavaScript API developed by Apple in 2004 it was there's a specification created by the what WG a couple years later it's basically in all browsers nowadays it's been that way for about a decade at least it works on mobile phones pretty much every device I think it could drive most of the billions of the pixels in this room so maybe like your your camera viewfinder or something in order to program on canvas we're going to need to use JavaScript which means we're going to use numbers strings boolean's arrays which are a great data structure for if you're interested in visualization they have a scary name array functions also have a scary name arrays functions don't worry about it they're cool and then method on arrays like map filter and for each there's a talk given last year to open discount by marco kazakh there's a wonderful talk on canvas especially image the image data object which i will not really discuss basically the image data object is is the bitmap that it is actually a canvas which the RGB red green blue and actually there's an alpha value at every pixel on the canvas and there's just a sort of a flat array and that's that's what the canvas data is and marco says she really really likes to race she likes mapping them she reduces them but she does it in the physical world and so these are the metaphors you know that we're going to have that that they're useful when you're learning a programming language especially for the first time because you know something real is happening in the computer the words don't necessarily correspond to exactly what you you know is happening in there and actually sometimes it's so complex what's happening here I mean it's overwhelming to think about so having ideas like about mapping which is basically transforming a set into something else or reducing which is you know taking a set and sort of turning into something smaller generally it's useful we need some boilerplate boilerplate sucks especially for beginners because you can't do anything until you have it the nice thing about canvas is it's not too bad you need one canvas element and then one script element to put your JavaScript in and then you need to do this scary stuff some Dom selection and you need to get the canvas and will render on the 2d context so almost everything I talked about for the rest of the talk in terms of rendering is going to happen on this 2d context so here's some basic drawing with canvas just filling Rex the context campus context has a bunch of properties like fill style stroke style line width they're all camelcase you can put in all the styles they're basically take colors you can put any HTML color in there RGB HSL dark orchid fill rect just immediately you have passed it an X Y width and height have some arguments and it immediately renders on to the canvas it's a medium load rendering doesn't keep any memory of this rect this is how you do circles circles are a little different there is no canvas method for a circle you have to begin a path and then you have to draw an arc which takes an X a Y this is actually radius and then the start angle and end angle so you don't you don't have to draw a circle you could drink for pie charts generate the arcs of a pie chart but this is a circle code like this now this is imperative code and I think imperative code is actually great for beginners because it's line by line something is happening and that's kind of the way it's good to learn to read code that way and see things happening eventually we want to sort of create words which are more expressive ways of talking about things here is a word for sort of a metaphor for a circle what will do is it will pass in X or Y or radius and a color and will immediately fill the circle on the screen we can create the exact same image it showed I'm going to go with our new circle function and it looks a lot smaller so don't be afraid to like define functions and play with functions also once you are making a visualization you may find that this particular circle as it's defined here doesn't really fit what you need to do you may be annoyed that it sets the fill style because you know after I've drawn this rich steel blue circle anything else you feel in the canvas unless you update this pill style again will render a steel blue because canvas once you set something it kind of stayed away unless you go back and change it I have a sort of sequence here exploring the lab color space it's a perceptual color space it's available with d3 so the first d3 method we're going to look at it takes three arguments Ella a and C so I have defined them here as variables and we're just filling a rect that fills the screen this is just picking a single color in lab space and so you can just take the return value of d3 lab and set it to a fill style we can make a 1d gradients out of this by using d3 range to create creates an array creates an array the same width as the canvas and then we iterate through this array using for each so we're doing something on every pixel of this canvas and the this here this little kind of scary masse bit doing as it ends up going from negative to 100 which are the valid values for this a parameter in lab space so for each one of these it draws a slice full height all the way across the canvas we can take a 2d section of the lab space by nesting yet another loop and doing the same thing on the D parameter and then we can trip out with d3 timer here oh no I changed the order of my slides I made this code not run on this slide all right let's let's get that side if I have time at the end it was cool it was cool here's another example of d3 timer this is a gif it's not a live example on the page turned out to be a mistake so d few timer what the free timer does is it takes a function and it just will keep running that function internally it uses something called request nation frame requestanimationframe sounds scary what it is is basically requestanimationframe will go to the browser say hey are you doing anything right now and the browser will say why no I'm not a requestanimationframe we'll say run this function and then the browser is like okay and then requestanimationframe like 16 milliseconds later will be like a are you doing anything now and they're just like why you use me that and he's like but no I'm not and then animation frame it's like we'll run that same function again at least that well that's how requestanimationframe is used in d3 time it just keeps running this function over and over again and the function that it runs it passes in the elapsed millisecond since you started so whoops I'm using it here too with satellite AF to just plot satellites so the way this code works is it creates a date gets the current time adds on 150 times the zaps time so it's basically running time at 150 1x speed and then this data object I'm not going to talk about loading data this is more talk about rendering so I'm loading data they're actually TLDs from celeste track which is the orbital parameters of the satellite and I'm running it through my plot satellite function passing in the data for that satellite in time inside the plot satellite function au satellite J s and this is kind of pseudocode here the structure is right but the some of the methods and arguments are a little different it propagates out that satellite to a time in the future so it's running it through its orbit takes the position of the satellite at that time and now it's a position in space because remember the earth is also rotating underneath the satellite so we also need to test at time and the position in and you can pass it straight into saddle ideas to get the geodetic coordinates which is sort of the where the satellite is directly above and then draws it on the ground obviously there's also a map underneath I'll get some maps a little bit later this will talk about canvas this is another astronomy dataset it's the exoplanet this is parallel coordinates what I just wanted to show you is the only demo I'll load externally my first open viscom talking about this this uses canvas move to in line to to render these lines across the chart there is a double loop so there's a loop iterating through every every data every exoplanet and then there's a second loop which loops over all of these dimensions the discovery method the planet letter number of planets in the system each of these dimensions have their own scale and so line two as it's rendering across the chart just goes and gets that scale and it knows where on this axis it needs to hit also when any interaction happens on this canvas I do kind of a trick I have a utility called render queue uses d3 timer as well to basically render the data slowly why would you run want to render the data slowly well the reason is because there's like 3500 exoplanets almost we tried to render every single exoplanet across all these dimensions every frame it takes a couple seconds maybe you know a second or two which means any inner action that you would have to do in that time scrolling brushing anything almost anything else on your computer your CPU would just be like you know totally maxed out for that moment we'll just trying to get everything on the display so ideas render queue to render things slowly so that the interactions are quick even if it's rendering you can interrupt these interactions the code for that is a little tricky so I'm not going to go through it the thing to know is that if once you draw something on the canvas there's no cost to keeping it around you can just keep rendering on to the canvas so if you have a way to stream in data and put on like campus I mean you can render gigabytes of data onto the canvas another trick is particle trails you can create the sort of whimsical effect when canvas very easily every frame of in the canvas usually you clear the whole thing every frame and render it anew because it's not that expensive to render things in canvas and if you want a particle trail instead of clearing everything you just kind of put a transparent rect over it so fade it out a little bit and every frame you just do that and you get these trails almost for free so I also want to talk about learning in canvas and SVG I found this develop a road map saying which kind of bugs me because you know it's cut it this is like the path to become a friend and develop really HTML CSS cs6 javascript you play with typescript wag pet angular react CSS you learn flexbox gradients at some point down at the bottom here you may play with SVG and then finally you get to learn d3 you know who will never do this is like ninety-nine point eight percent of the population I mean this is a it's an incredibly complex like the ecosystem right now FEG itself I think has some challenges to learning this looks like a boring slide it's very attractive trust me it's very inflammatory in the d3 community this slide there's a lot of complexity involved in managing a Dom hierarchy and in the relationship between SVG and HTML and CSS and adders and styles path strings have their own language you have to do string concatenation if you've ever tried to learn d3 you've probably struggled with exactly what is happening between these methods select all data enter a pendent merge and transition and then it gets slow with a large number of elements about a thousand elements but there's tons of tutorials out there we've invested I mean as a community maybe tens of thousands of hours of developing beginners materials using SVG I think a lot of people who learn d3 they struggle and we get frustrated and I've even I mean I've known a lot of people who've learned d3 I think even some people struggle with like a minor mild depression at some point trying to learn it and these people laugh because they know it's true I'm not I'm not being hyperbolic here I think it's because it's different than other technical tools you may learn because they think you learn d3 because you want to express something and you have a vision of what you want to express you see the amazing work by the New York Times and you want to do this so you want to learn this technology because you have a story you want to tell and you have a unique perspective and all this technical stuff gets in the way and then all of a sudden you know you're stuck somewhere on some bug if no idea why this thing is not happening or this line is being drawn this way or you know something it happens to a lot of us when we're learning I think almost everyone has this experience even people who program and come into it pay no attention to this slide canvas is perfect one thing I do want to mention is that the really one of the big things you lose I'll show you how to get some of these things back but I don't have a solution here for an accessibility fallback with SVG you get for free or with a little bit of effort support for screen readers other accessibility tools you cannot get that from canvas people they can't see they can't experience your work this is the New York Times graphic epidemic of drug overdose ripples across America it's about sort of the rise of opioids and it's kind of related to these others diseases of despair these are choropleth maps she have data comes from I don't know if that particular data comes from cdc-wonder but if you're interested in doing work like this with mortality cdc-wonder is a great place to start it's got data by county you can break out you can get breakdowns by year of death by age group race gender cause of death where they died if they died in hospice facility Quorra plus in d3 with canvas are actually pretty easy because geo paths actually supports just taking a canvas context and so whenever you pass in a Geo JSON into the path and I know now that if you haven't learned d3 you're going to get lost in terms of the technical examples for the rest of this talk I apologize about that and all you need to do you do all the same canvas stuff and when you're ready to render a path you just pass in the Geo JSON into the path element path function and it will just run it and you get to pick your projection you can do everything almost everything you do in gt3 geo almost everything you can do in d3 you can do with canvas you can get retina support with this little thing the device pixel ratio basically like this display has twice as many pixels in in both directions as what it sort of reveals for the canvas and sort of basically by default things look fuzzy because it's a retina high density display a lot of displays are like this now so that means for every pixel that it looks like it has when you say font size 13 pixels there's actually two pixels in both directions so four pixels for every one so if you want to render get that level of detail on every single actual pixel on this display you need to put this this little junk of line of code here's another advanced trick if you're beginner I mean this is hidden canvas lookups basically common complaint with canvas is that you can't select objects on the canvas because there's nothing there it's just a bitmap this is the trick that just want to put it out there because it works with a lot of visualization you can create a second canvas which you use this will look up basically you put a unique color for every object that you've rendered and then you save that mapping from the colors to the original data into a JSON file and then whenever a canvas mouth event happens on the screen you use get image data to get the color off your hitting canvas and then you can look up your original data file and then you can add tooltips or highlighting or things like that Yanik from a book who actually has a great blog post on how to implement this but i just want to give you a visual that it is possible to do it a couple other ways to do interactions you could use d3 quadtree this is especially useful a lot of people use Voronoi diagrams though add a Voronoi layer of SVG which sort of gives you sort of maximizes the area that you can interact with a point on a visualization if you use a quadtree and just search for the nearest point you get the exact same interaction service so in this sort of example running in the background it's actually a bunch of Voronoi is being drawn on a canvas and I use quadtree to figure out which Voronoi is being drawn and highlighted and as you can see they are the same it creates the exact same interaction surface who views it that way for simulation has a new method in version 4 called simulation client so pretty easy actually to do there's a lot of examples of the force layout in canvas with interactions and the force layout is actually pretty fast if you've only used the foresail with SVG the biggest bottleneck is SVG the force layout really can support tens of thousands of nodes and up to about ten thousand almost no problem sorry tens of thousands of objects so I'm talking about nodes and edges so here I think there's about 6,000 if you add up all the nodes and nodes and edges d3 paths in version 4 I think it's a new utility I forget actually but it makes available these canvas method so these are all methods that are available on the canvas and if you use them in d3 path instead it can still render it to the canvas and basically do that if you're only using canvas there's a big reason maybe to use it at least I can't think of one off the top of my head but the cool thing is actually you can generate SVG path strings so and it's actually used for all of the v4 shape generators so arts pies lines areas curve symbol stacks all these things are now built on d3 paths which means they too can render to canvas so all of these shape generators now can do both in version 4 of d3 we have a huge problem I think in terms of educating new people on how to use d3 I think the learning curve is extremely steep I think a lot of people get discouraged because it may be their first time programming and I think it's a critical moment when someone says you know I'm ready to take this active relationship you know with the computer and actually learn how to program the computer to make images because I mean for whatever reason that they they decide to do that and if they hit this wall of complexity and you know fail to really get to a point where they feel comfortable you know they're less likely to try it again and if they do try it again and they fail again they don't they're not able to get that place where they feel comfortable and they feel like they can actually express themselves they probably won't try a third time and I think if we have this complexity like in the learning process sitting there we're going to lose a lot of pee and just a sort of attrition of people who have worthwhile stories to tell I've never met someone who won't sorry I've met a lot of people who wanting to learn d3 and are struggling to learn d3 and they'll tell me why they're trying to learn when d3 and it's just a worthy story that sometimes they're trying to tell and often a complex story and a story that I've never heard before I had the privilege to work for several years at stamen design one of my favorite habits at stamen and is that when we were making prototypes and creating stuff sometimes you just have an idea and people will put their laptops on the floor like like this we kind of do it just I guess it's the height of the couches but it's also because it sort of minimizes the area that the laptops take and we're at that point you know an idea is consuming us and we're walking around the room and we're you know thinking and talking and brainstorming and the data and the ideas and the computer have left off the page and are now in the room and we're thinking about them then we're more brainstorming and I think that's really the goal of visualization I mean I'm talking about pixels and things but pixels you know it's just a mechanism to get these ideas into our minds to be able to think about it and have this relationship with data especially you know there's no way to have a relationship with a gigabyte of data unless you're actually using computers somehow some tool to internalize it I was inspired to learn data visualization by this wonderful talk by Hans Rosling at some point he explains that what you've just seen is like 40,000 50,000 points of data and I was like wow I feel like you know at no point was I over whelmed I mean I you know I know I could go back and look at all the details of this visualization we have this incredible capacity to consume a large quantity of data especially when we have the right person to sort of guide us through the data and tell that story and so I think we need more people in our community who are that person who have that vision of the story that they want to tell so I hope you will rise to the challenge of thinking about this problem of how to make learning d3 and learning programming more accessible to a broader audience because it would really benefit us and and the stories we're capable of tell to tell by I am by having that diversity that's what I have thank you very much [Applause] [Music] [Applause]