Behind the Curtain: Inside the Hidden Tech of Live Theatre and Events - EMF 2026

Welcome, welcome. This is Behind the Curtain. I am John. I'm a production manager, technical director, special effects supervisor. Um, for the last uh 3 years of EMF, I came on in 22 to run the music team. Since then, I've been one of the co-leads for the stage team, and this year, I have been the rigging, fire, and pyro lead, which essentially means that anything to create or destroy fire is unfortunately in my wheelhouse. Um, many, many moons ago, I used to do a lot of data center engineering. Uh, now I mainly get, uh, paid to do things mostly on Broadway and West End. Um, I am licensed in an unreasonable number of jurisdictions. Um, and I am here to explain pretty much all of that to you in various forms. Uh, we're going to be talking specifically about a lot of the in-depth technical details here in terms of how these types of systems tend to work. Uh, so quick disclaimer up front. Uh, this is a talk, not training. Uh, I have an unreasonable amount of public liability insurance. Please do not try anything that you are about to see at home. Um, so first up, um, quick show of hands. How many of you have at some point gone and poked around the front of house or back of a theater or tech events venue? Maybe even here. Excellent. Everybody here. Um, you will probably have seen a number of different boxes in one of those stage racks. And those will be doing a number of different things ranging from networking, lighting, audio, show control, video, power distribution, and even coms. Um, and this stack is generally relatively simple in terms of each section of it. But the complexities always tend to come in when you start integrating across them because we have various different segments in each one that all have to interconnect and all have to do various different things in order to make sure that we have lights and sound and video and all of the wonderful things that we use to not only bring you talks but also to bring you a myriad of different live events and theater in all its wonderful forms. So, first up, um, every single one of the lights that you currently see illuminating me or anything else in this room is controlled by a protocol from 1986 that most of the world stage lighting still runs on. Um, it was developed originally to replace a collection of proprietary incompatible control systems, a problem that will never happen again. Um, and the key limitations with DMX as a protocol is that it is RS485. It has 512 available channels per universe. A channel being a single parameter. So, red, green, blue, brightness, intensity, etc. Um, and no error detection whatsoever. Um, it is completely one way only. Um, which is extremely fun. um except if you're doing RDM which I'm not going to talk about here because that's complicated but technically you can do two ways if you start doing things that make you hate yourself. Um and it's interesting particularly because uh the standard actually says verbatim that it is not an appropriate control protocol for devices that could cause injury. Um, which is interesting because in most people here will know at some point that you can just go off onto eBay or AliExpress and buy a DMX controlled flamethrower from a remarkably unscrupulous website for about2 to300. Um, which more on that when we start getting to pyro control, but uh please don't do that. Um, and yes, specifically this is a very dense um, like like you have a lot of stuff going on here, but often that's still not enough. Uh, because you may have several hundred different universes, a universe being 512 channels, um, based on the sheer quantity of lights that you have, especially if you're dealing with things like LED pixels or any of the larger, more complicated, uh, fixtures that we get in the modern era. Um, and along comes to solve that problem, ArtNet, uh, which actually comes in in 1998 from a British company called Artistic License. Um, and it's their solution for blasting DMX over pretty much any IP network. Um, it is UDP broadcast, which means that every device gets every single piece of data. Um, it won essentially on sheer adoption because the practical difference is that streaming ACN is multiccast. The good news being that your subscribed devices only get the data for that specific universe. The downside being that you need managed switches and good network hygiene and a million other things. Um, and ArtNet will run on any off-the-shelf unmanaged Ethernet switch. Um, so you can tell which one succeeded on winning the adoption war there. Um the flip side is that um most modern installations that are using considerable quantities of DMX over IP or anything that requires high high redundancy will be using streaming ACN. Um and especially if they happen to be using any form of addressable LED pixel. Um because a subtle pro problem here is that with streaming ACN and to a lesser extent with artnet sometimes um every universe of DMX is its own network packet being sent independently right because each one has to go to a different place. Um, so on a big rig that can be hundreds of packets per frame and without any kind of synchronization, each receiver is then going to go and act on that data as soon as it receives it. And then suddenly all of your pixels are like a fraction out from each other, which can look really, really weird. Um, lighting, you might not notice if you're mapping pixel wall or like big color bumps across the whole stage, you get like visible tearing across it. Um, so the good news is that we fixed this. Uh, in the 2016 revision of E131, which is the streaming ACN spec, they added a thing called universe synchronization, which is a data packet that carries a synchronization address, and then the source sends out all the data packets for that frame. Every receiver then waits for them all to arrive and then it sends out a single synchronization packet on the sync universe, which then causes all of those receivers to then update at the exact point that they receive it. um which is great. Uh the catch is that it has to be multiccast which means that you again need a network with proper IGMP snooping and not every receiver implements the spec meaning that if you happen to get one that that is older or potentially like less implemented it will just ignore it and free run. So always worth checking whether that's actually going to work. Um yes. So show control um almost everything in every single one of these stages has some form of integrated show control. um especially the video side is running off of a wonderful combination of a thing called bit focus companion and a few other things. Um commonly when it comes to technical theater we are using a piece of software called QAB which is probably the most common piece of show control that people here will have heard of. It's a really easy entry point. What it does is it takes a bunch of different cues, whether that's network messages, MSC commands, MIDI, show control, OSC, um different form of show control, uh time code, all of those in a defined order and then lets you fire those repeatably and reliably. Um because for most theatrical cues, we're fine with tens of milliseconds worth of precision. The value is all in repeatability. You want to be able to know that the thing fires the same every time each night, that your lighting board op doesn't have to remember to hit go every time your audio board does in order to make sure that your lightning actually syncs with your lightning sound. Um, that can get really really silly. Um, and yeah, so this is this is a zoo. Like the we have all of these different protocols and they all exist fundamentally because all of these different problems have been solved by different industries at different periods of time and then we had to sort of all push all of them together and all of their transports and figure out how to make them all talk. Um so let's talk through a few of those now. Um so first upcoms. uh the system that nobody in the audience knows exists and the one that everybody running a show cares about immensely. Um this is how your crew talk to each other. So think of this essentially as little radio packs um single single like group party lines um and that gives you the ability to have everyone on that channel hear and talk at once like a big conference call. Um so there's a few different major names in this. you will most commonly with analog party lines here. Clear common RTS being the two big names. Uh most of the tech of which still dates from the roughly late 1980s 1990s. Um and the idea of that in most cases is that there's either one or two or three or four shared lines. Everyone hears everybody else. You do not have the ability to directly dial people. Um, and it's almost exclusively wired headsets with a few exceptions. Um, for the uh wireless intercom systems that run wirelessly on analog. Um, and then nowadays we have much newer, more consistent stuff in the form of things like the Redel Bolero series, any of the modern forms of clear matrix intercom, which basically allows you to set up as many different groups as you want, configure different belt packs to be able to talk to those groups in whatever combination you want. Um, and then basically just fire off and and have as many combinations as as you wish, which is what a lot of the larger TV and broadcast folks tend to do. Um, and the the other fun with this as well is that a lot of the older packs are also for wireless are specifically 2.4 GHz, uh, which will definitely fight with your Wi-Fi. Um, Comm's quirks. Um, analog party lines areing weird. Um, cuz they they are all running generally across a standard 3 pin XLR mic cable. Uh, the difference is that it's duplex. So, you are talking and listening down the same line and then on the other line is 30 volts of DC that is then running all of the belt packs. And that also actually carries the call signal, which is the thing where if you click the call button on your on your belt pack, that will then flash the call button for everybody else on the same loop. Uh because your SMS and automation folks and some other people can't always have a headset on their ear. So you hit call and their belt pack then lights up to say, "Hey, look at me. Pick up. There's something you need to see." Um Clearcom does one channel per XLR like that. RTS splits it two channels across pins two and three. Um and the trap here is that this is all on three pin XLR. Um, and it's an identical plug and there's 30 volts of DC sitting on the comm's line. So, if you patch your coms into a mic input or a DMX node that takes three pin, congratulations, you've just bunged 30 volts of DC directly into it, and that is now most likely toast. Um, and Clearcom and RTS also use XLR3 again, but with different pinouts. So, even the two comm systems aren't interoperable with each other. Um, which is an additional thing. Um, all of the wireless systems are decked. They're all 1.9 GHz. Like various different additional fuzz on top of them. But like Bolero, most of the modern Clearon stuff, it's all it's all beening decked. Like why? Um, other than the fact that it is reliable as we all know being in this field and definitely not having picked up a phone call that has just become like garbled. Um, lovely. So let's jump on from coms to very quickly talk a little bit about Dante. Um that's the elephant in the room for audio networking. It is the direct uh is the direct protocol that is currently taking my voice from this belt pack through the air to the receiver at the back of the room and then through into the Yamaha DM3 sitting at the back. Um it is a horribly proprietary system from an Australian company called Odinate. Um, and it has quietly become the absolute default for how audio moves around modern show environments. Um, the clever thing about it is that it does all the hard bits for you. Runs over gigabit Ethernet, same switches, same cable you normally use, but it deals with all of your clocking, PTP clock discovery, routing, all of that basically for you. Um, and what you do is you go into a piece of software called Dante controller and you say, I want this input to go to this output, which is called a subscription. Um, and at that point you can then go off and select your latency offsets anywhere from half a millisecond to five. Uh, and the trade-off there is basically you're trading delay for tolerance on a more busy network because obviously more packet drop means that you end up with a situation where potential loss there means that you actually lose the data. Um, and obviously that then means that you have one network cable that can carry hundreds or thousands of channels in both directions which can then replace the horrific multi-core XLR snakes that we've been using for years. Um, the good news is that you don't have to use this anymore because we now have a wonderful thing called AES-67, which is a common mode mostly open-source protocol built on around sort of the Dante spec, which pretty much all of the modern um inter interoperable folks that use Dante support. Um, and it lets you effectively um interoperate between multiple different consoles that don't necessarily talk the same control protocols. They should all be able to in theory talk AES67. Um, as long as they're all set to that mode. Um, and as with all things in life, uh, the eternal cursed nightmare of this is clock sync. Um because you always have to like elect a single grandmaster PTP clock for every single like basically PTP works by having the most accurate clock in the room. Um and the practical thing to know here is that if you are in if you're in an AS67 or a Dante network um as a general rule every device is a potential PTP participant. That is to say that if something boots up and then it decides that it is your PTP grandmaster while another device has already decided that it is your PTP grandmaster then congratulations everything is going to start unpleasantly clicking and dropping out. Uh which is a great way to find out especially mid show. Um now the fix here is three-fold which is one enable IGMP snooping on your switches. B get some proper manage switches and C um get a dedicated grandmaster that you actually trust which usually if you're doing this should be your sound desk um or actually no e either your sound desk or your Rio or equivalent stage box. There's a bit of yeah people have strong opinions on this that I don't really care about. Um but it's yeah it's a diplomatic problem as much as a technical one. Um so um next up we will touch a little bit on rigging which is the corner that I have mostly had my hands on for EMF this year. Um I would like to make it very clear first that like rigging is a structural engineering problem first and an electrical and software one second. Um the most elegant show control systems you will find will not help you if your halfton motor is attached to a beam that can't make the that can't take the load. Um so an example of this here um we have a uh truss directly above your heads that is run up to a couple of span sets sitting on some halfton chain hoists uh that I rigged a week and a half ago. Dear god. Um and that is then connected directly to a pair of steel safety chains which then ensure that even if those directly uh even if the underlying um the underlying hoists fail that the truss then cannot fail on top of that. Um so couple of terms that you'll hear often when it comes to rigging. Uh working load limit uh working load limit is not safety factor. This is probably the single most misunderstood thing about uh entertainment rigging. Um, so you will generally see most things rated at a safety factor of 5 to one, which means that if you have a halfton shackle, that is the 500 kilo working load limit. That means that it has been tested to break at 2,500 kilos. That does not mean that you can safely load it to 2,500 kilos. you load it to 500 because the factor exists specifically to deal with things like dynamic loading, shock loads, and the fact that everything does eventually degrade over time. Um, the other piece of this uh load cells are your friend. If you are ever doing anything that is particularly complicated, uh, I'm specifically talking here to the folks out there who are doing circus rigging, which uh, congratulations. I salute you and I'm sorry. um you have lots of wonderful complicated dynamic forces that will be going on there. And often the easiest way to solve for that particular problem is putting a load cell on your motor attachment points which will then give you the exact ratios for how much things are actually getting loaded. Um there's a few different companies that make these, specifically wireless ones, which are excellent. Um, and it allows you to basically get a full live picture of what all of your points are actually doing and then adjust them if necessary to make sure that everything is truly balanced. Um, ooh, that's fun. Interesting. Please hold. We appear to have lost slide. No, we haven't. There we go. Excellent. Um, lovely. uh small brief vocabulary of uh things that you will potentially hear in terms of terminology when it comes to this um structures. So this is box truss. You will see some triangle or trirust around places. Um I mentioned earlier we have what we call spans sets or uh soft shackles or gak flex which is basically steel aircraft wire encased within a much softer outer sheath which we're able to use to set specific rigging points for certain circumstances. Um clamps various forms doy cheesebor different US and American terms for those um and then chain hoists and also chain motors. Um, you will hear those specifically referred to in the context of single or double braded motors. That effectively means in the form of a double braded motor that it has a built-in backup of some kind, which effectively means that the the motor itself is not capable of just freef falling the load in the event that the first um that the first break itself fails. Okay. Um, moving on, uh, from the main fundamentals of this, uh, let's talk about actually flying things in and out. Um, so fortunately, we do not have any counterweight flying on site as far as I'm aware of. If you happen to find any, please let me know. Um, but as a general rule, when it comes to most theater, you will find one of two specific um, flying systems in use pretty much everywhere across this country and and reasonably the world. Um the first of which counterweight flying um all muscle all rope. You just have a single um pulley set up loft bar head blocks over in the grid. Um a batn or bar which is what actually hangs your load set on a set of wire ropes. And then an arbor which is a cradle that you then load with a specific amount of weight in order to effectively offset the weight that you have then put on the bar. The idea being that your flyman then only handles the uh amount of effort required to move it out of balance. Um, as a general rule, you should always be very careful to wait your bars before you fly them. Otherwise, they will have a habit of running away into your grid. Ask me how I know that. Um, and as a general rule, uh, you have relatively little feedback unless you're operating in a theater that happens to have, uh, invested in some kind of a front of house or top- down camera system run to their fly floor, which some of them have, but for the most part, it is you are relying on everybody else on the ground, your stage managers, everybody else to act as your eyes and ears in terms of safety systems, and then you're putting a couple of marks on the rope to be able to take it from one point to the other. Um we have in fact managed to move past this um though not without the addition of a quite considerable quantity of budget. Um so we now have a lot of automated flying which is motorized winches um all computer control closed loop very very very fun systems and this lets you fly people and set and vehicles and whatever the hell you really want uh reliably to roughly to the millimeter. um often sometimes beyond that depending on the system. Um a lot of the systems that you'll hear for this is Kinesis which I believe has now been bought by Tate. Um Stage Technologies which also do a lot of stuff in that corner. Um yeah, bunch of different uh CM Loadar also do some hoists that specifically have these kinds of features built in. Now um and this all runs to the MCE 143 spec. Um, and the critical number that you're looking for here is safety integrity level three, which is a formal standard for how often these things are allowed to fail. Um, the critical safety function here is 1 in 10,000 demands. Um, which I know seems low, but realistically that tends to be more than enough in a in a practical reality for for what we're doing here. Um, and the key here is position is position control, not speed control. like you're going to tell it to go to position X and stop there repeatedly and it will do the identical move every night because it's all being watched by encoders and many other fun things. Um, usually redundant ones. And the safety heart of it is essentially that you will have pretty much two encoders on every single axis. And if either of those two encoders disagree about where something is, the system isn't going to guess. It's going to estop. um which is a very similar fundamental to a lot of what you'll get when it comes to aviation engineering where you have like two pretty much two sensors on everything unless you're Boeing. Um yeah. Um and you get to define all of the motion envelopes in the software in terms of how quickly those things will will bounce between certain points. Um quick thing on flying people. um everything on the automated flying side still applies but the moment that a payload becomes a living person rather than a piece of scenery the overall extra layer of engineering comes in and the design factor goes up. Uh generally if you're dealing with lifting scenery or vehicles or whatever else you're dealing with a 5:1 sometimes an 8:1 factor for human flying you do a minimum of 10 to one on the entire loadbearing path which means that all components are rated at least 10 times the full working load. Um and on top of that you do not rely on any single thing. You should duplicate every single load path. Dual suspension wires, dual brakes, dual um secondary safeties that are completely independent of the prime of the primary and can fall arrest if required. Um and all of this is wrapped in a lot of very specific standards. In the US it's ance 143. Um in the UK you're looking at the BS7905 lifting equipment for performance standards. And all of that then sits under the ABTT and DSA flying a performance guidance. And then all of that sits under Lola um which basically is the mandated inspection regimen for all of this. Um and also worth noting none of this is worth jack without a rescue plan because if your performer ends up stuck in your grid, you need a rehearsed practiced way to get them down again. Um your performers are signing on fundamentally to your trust and you need to not only earn that but also back it up. Um yeah, a a a a point on this. Um often these we we do a lot of complicated often very dangerous things. Um most often what you need to focus on is competence rather than having the right equipment. Um, the best example of this, um, the one that basically every stage pyro course will teach you is the station nightclub fire in Rhode Island where in 2003, um, we was a band's tour manager showed up with a bunch of jerbs, untrained, unqualified, um, set them off. Um, it then ignited the acoustic fire in the walls and 100 people 100 people died, 230 people were severely injured. The venue was burned to the ground in roughly 6 minutes. I say that not to be grim, but because that is the clearest possible example of what happens when pyrochnics especially, but any form of critical safety system is used without proper planning, proper materials or safety assessments and without coordination with the venue. Um like every especially for pyro, every single control we have in modern proximate pyro practice exists because at some point something went wrong with it. Um, speaking of, let's talk about fire alarms for a second. Um, so fire alarm systems. Uh, as soon as you start putting, uh, haze or smoke or pyro into a building that happens to have fire alarms, uh, congratulations, you are now interacting with the fire alarm system. Um, you should know how it works. For one, there is a number of different detectors built within this. You have optical detectors, which deals specifically with light scatter, uh, which is most likely what you end up with in most theaters. You have ionization detectors which specifically focus on the combustion particles. You have heat and rate of rise detectors and you have multiriteria ones which is most likely what you will have in most normal theatrical venues. The idea of this is specifically that in almost all cases a theater is capable of being dropped into what we call a zone isolation mode. And what that means is that you can turn off or turn off certain facets of the fire alarm systems within that area of the stage. Meaning that, for example, you can turn off the heat or ionization criteria on a specific sensor and only look at optical light scatter, for example, so that haze might not set it off, but much darker, thicker smoke that could come from an actual fire will. Um, there are specific requirements on this. Again, normally if a system is in an isolated or a silent alarm mode, then you have I if if that alarm then trips in most commercial venues, this is not always true, you will usually have a roughly 15-second period in order to go and acknowledge that alarm on the panel, at which point you then have exactly 1 minute and 30 seconds in order to identify the alarm and cancel it out. Otherwise, it turns into a fullscale alarm and you then have to deal with the joy of explaining to your set people why all of your flats are now very wet. Um, yes, HVAC often always has its own sensors. So, if you happen to have any smoke that is getting pulled up and through vents, always keep an eye on that. Um, and if you happen to be capping smoke detectors, which is a thing to be very clear that you should only do if you are qualified and know which ones you're capping, make sure that they come off the moment that your like show window closes out. Um, and also always know who your fire marshall is, know who has the reset keys, all that jazz. Excellent. Uh, we are going to jump through a couple of things here because we have uh relatively little time. Um let's talk briefly about LED wall. Um so for anyone who has dealt with LED wall at some point, you will know that it is a bunch of different LED panels uh blocked all together with power running in between them and what looks like Ethernet running in between them. It is not Ethernet. It is a horrifying combination of serial and just bitbanged JPEGs just firing their way across the ether for some reason. Um, additionally, all of the uh receiver cards are not always, but at least in the case of stage D, for whatever reason, very volatile, meaning that if you reboot them, they then proceed to lose their config. Um, or if the flash on them tends to get a little bit screwed over, then they also lose their config. Um, and at that point you end up with fun things like firmware mismatch or if anyone has seen the weird avanguard art project that keeps showing up on stage D's info beamer. All of them turning 90° sideways for some reason. Um, lovely. We have run through the specifics on that. Um, lovely. I am going to finish off with this. Um, 8 seconds. That is roughly how long you have before any audience will notice that something has gone wrong. Not 8 seconds to fix it. eight seconds before they notice. You may have longer to fix it if you can fill the time, but clocks on awareness are very, very quick. And the best crews that I've ever worked with all have a few very specific things in common. They've thought very carefully about what they would do if something went wrong. They have thought what would happen, but what would what they would do before that thing went wrong. And they have clear decisions that they have already made, dry runs, familiarization runs, abort procedures, and very clear human layers. And at the end of the day, the principle across every discipline, it should be, especially if you're dealing with these kinds of systems, if you're not sure, stop. Think about what you're doing, figure out the parameters, and then go back and and and do it the way that you have now figured out. And if it's a safety risk, you can always call the show stop. House lights up. Everything holds as is. Lovely. Um, thank you so so much for coming through to this. Um, none of this is magic. If you would like me to explain any other specific details or anything that I happened to jump through on account of time as part of this, I will be in the Q&A tent immediately after this. Other than that, thank you so much. Thank you for showing up to this wonderful, wonderful thing that we have all created together. And yeah, thank you. [applause]