Vectors, Vector Spaces and Inner product spaces - Episode 14 - Maths

Right now, um, somewhere in a server farm, a machine learning algorithm is looking directly at you. >> Yeah. But it doesn't see your face. >> Exactly. It doesn't see a photograph. Yeah. And it, uh, it definitely doesn't see a neat little row on an Excel spreadsheet. >> No, it sees you as a single highly specific arrow just floating suspended in a 50-dimensional void, >> which is wild to think about. And depending on the precise like mathematical trajectory of where that arrow is pointing, your insurance premium is about to double >> or you're about to be flagged for fraud, >> right? It is a profound shift in perspective. >> If you are an aspiring actuarial student or a financial analyst or you know just a fiercely curious learner trying to decode the underlying architecture of risk in the modern world, >> you have to realize that the fundamental nature of your work is no longer just arithmetic. No, it isn't. It is geometry. >> Absolutely. >> In our last deep dive, we spent a lot of time talking about matrices. >> We did. Yeah. >> We looked at those massive interlocking grids of numbers, right? The coariance matrices, the transition matrices, >> and we examine how they act as like transformation engines in our mathematical world. >> Exactly. And if you follow the actuarial sciences, you are already intimately familiar with that matrix algebra. You spend your life buried in those flat spreadsheets. >> Right. Because the illusion of the spreadsheet is that risk is something contained in a single cell >> like a single flat scalar value just a number on a page. >> Exactly. But today our mission is to step completely off that flat plane. >> We are jumping into the source text the geometry of risk vectors and multi-dimensional spaces and we are going uh basically inside the matrix. We're looking at the objects being transformed because true risk, the kind of systemic cascading risk that brings down pension funds or exposes a multi-million dollar fraud ring doesn't live on a flat piece of paper. >> It doesn't. >> No, it is a massive intersecting multi-dimensional reality. >> So for everyone listening, today is not about just memorizing a new formula. You aren't going to just crunch numbers. >> No, you already know linear algebra. You know what a vector is and you know how to calculate a dot product, >> right? Today is about visualizing the hidden geometry of financial markets and human behavior. >> We are going to look at why the mathematical properties of vector spaces serve as the perfect map for financial risk. >> Things like linearity, superposition, orthogonality. >> Exactly. And to understand why we use these specific geometric tools to model human chaos, we uh we actually have to go back to the 19th century. >> Yeah. back to a moment of sheer mathematical desperation, >> right? Because if you really want to understand the architecture of modern multi-dimensional modeling, you have to look at the psychological walls early mathematicians were hitting. >> So let's introduce William Rowan Hamilton. This is the early 1800s. Right. >> Yes. And at that point the mathematical community had a very elegant completely functional system for describing the two-dimensional plane. >> They had complex numbers. Exactly. Right. So that's the format of like a plus bi. You have the real axis and you have the imaginary axis. >> And the beauty of complex numbers isn't just that they give you flat coordinates. >> It's that the algebra of complex numbers perfectly mirrors the geometry of that 2D plane. >> Right? If you multiply by I, you get a perfect 90° rotation. >> It's an incredibly tight closed mathematical system. It just works perfectly on paper. >> It does. The operations define the space beautifully. But um Hamilton looked at that and recognized the obvious limitation >> which is that we don't live on a flat piece of paper. >> Exactly. We live in a threedimensional world. >> Right. So according to the text for 10 years Hamilton became utterly obsessed with trying to invent the three-dimensional equivalent of complex numbers. >> He wanted a mathematical language like a pure algebra that could capture and manipulate physical 3D space with that exact same elegance. And from the source text, it's clear this wasn't just like a fun weekend puzzle for him. This was a decadel long psychological torment. >> Oh, absolutely. He was trying to build a system with one real part and two imaginary parts >> like say a plus b i plus cj. >> Yes. He called them triplets. He spent years trying to figure out how to multiply these 3D triplets together in a way that preserve their length, you know, their norm. Okay, let's unpack this a bit because in normal math, if you multiply two complex numbers, the length of the resulting number is just the product of the lengths of the original two. Right. >> Exactly. It is a beautifully consistent rule. But every time Hamilton tried to multiply his 3D triplets, the algebra just completely broke down. >> The terms cross multiplied into absolute chaos, >> right? He couldn't preserve the norms. The rotations wouldn't hold mathematically. >> It's just agonizing to read about. I mean, imagine having a word right on the tip of your tongue. >> But instead of a word, it's the fundamental structure of the physical universe. >> Yes. And it takes you a full decade to remember it. Imagine waking up every single day knowing exactly what geometric concept you want to prove. >> But the mathematical language simply refuses to bend your will. Every time you try to mathematically rotate an object in this hypothetical 3D math, it tears itself apart. >> It's tragic. But then uh we reach October 16th, 1843, >> the famous day, >> right? Hamilton is walking with his wife along the Royal Canal in Dublin. >> And in this sudden, almost violent flash of intuition, he finally realizes the flaw in his premise. >> He realizes that to describe three dimensions, he actually needs four. >> He needed four dimensions. It's totally counterintuitive. >> Wait. Okay, let's unpack this right here. Why would you need four dimensions to understand three? That sounds crazy, right? >> Yeah. I mean, if I am trying to map the rotation of a physical object in a 3D room, throwing a fourth dimension into the algebra seems like it would just over complicate everything. >> It seems that way until you look at how physical rotation actually works. Right. >> In three dimensions, to rotate an object, you don't just need to know the angle of the rotation. >> You need to know the axis of rotation, too. >> Exactly. And an axis requires three coordinates just to define its orientation in space. >> Oh, wow. >> Right. And then the angle of the rotation itself requires a fourth number. >> So the algebra inherently demands four degrees of freedom to operate smoothly without breaking down. That's fascinating. >> It is. He needed one dimension to act as the scalar, the real number, and three distinct imaginary dimensions I, J, and K to handle the complex rotations. He called these quatronians. >> But more importantly, to make this work, Hamilton had to abandon a really fundamental rule of arithmetic, didn't he? >> He did. He had to abandon commutativity, >> which is the idea that like A* B always equals B * A. >> Right. Exactly. In Hamilton's new 4D system, I * J does not equal J * I. >> It equals the negative of J * I. >> Yes. The order of operations radically changes the outcome. >> Which, if you think about it, perfectly maps to physical reality. Like if you take a book and you rotate it 90° forward and then 90° to the left, >> it ends up in a completely different physical orientation than if you reverse those steps, >> right? If you rotate it 90° left first and then forward, it's totally different. The order fundamentally changes the geometry. >> And the moment he allowed himself to step outside the restrictive rules of standard arithmetic, that 4D math just clicked perfectly into place. >> He solved the 3D rotation problem by stepping into a higher dimension. And the s source text vividly describes his reaction to this and it is just pure visceral panic. >> He was terrified of forgetting it. >> Yeah. He doesn't have any paper on him. He's terrified this decadel long obsession is going to slip out of his mind before he gets home. >> So he pulls out his pocketk knife right there on the walk. >> And he physically carves the foundational equations into the stone of bromeridge. >> He carved I^2= J^2= K^2= JK=1. Just gouged it right into the stone. It is the exact moment the mathematical framework of the physical world changed forever. And it was out of this specific work on quitterians that Hamilton actually extracted the concept of the vector. >> He coined the term from the Latin vir meaning to carry. >> Because the vector part of his math literally carried a point from one specific coordinate in space to another. It had magnitude and direction. >> Exactly. But here is the really fascinating constraint about Hamilton's work. >> What's that? His entire system was built specifically to map physical real world space. It was fundamentally tethered to literal physics. >> Ah right. And that is where the parallel story of Herman Grassman becomes absolutely vital for us and especially for actuarial science. >> Yes. Grassman is the other half of this puzzle. So while Hamilton is physically carving equations into a stone bridge in Ireland to conquer three physical dimensions, >> Grassman is a high school teacher in Germany publishing a book called the theory of extension. >> And Grassman looks at mathematics not as a tool constrained by physical reality like Hamilton did, but as a language of pure abstract logic, >> right? Grassman asked this incredibly radical question. If the logical rules of algebra hold for two dimensions and they hold for three, and Hamilton just proved they hold for four, >> why stop at four? Exactly. Why stop there? Why not infinite dimensions? >> If the underlying algebraic structure is sound, why not five dimensions? Why not 50? >> It is a massive conceptual leap. He is taking the geometry out of the physical room you're sitting in and putting it entirely into the abstract mind. >> Right? So, Grassman basically invented the concept of the generalized n-dimensional vector space. He established that as long as your mathematical entities obey a few highly specific axiomatic rules, you can build a consistent geometry in literally any dimension. >> You don't need to be able to visualize a 50-dimensional object in your head to mathematically prove its properties. >> No, you just follow the logic. But the tragedy of the source text here is how he was treated. The mathematical community at the time just completely ignored him. >> What's fascinating here is that they didn't just ignore him. They couldn't even comprehend him. >> Right? The mathematicians of the mid-9th century were deeply, deeply rooted in applying math strictly to physics and mechanics. >> So, Glassman's work was so purely abstract, so entirely divorced from visual reality, they simply saw no use for it. >> He died thinking his life's work was a profound failure, >> which is heartbreaking. Yet today, he is literally recognized as the father of linear algebra. >> Absolutely. Every single actuary listening to this owes their entire career to the fact that grassmen proved you can do math in dimensions you cannot physically see >> because an actuary does not model physical space. >> No, they model wrist >> and wrist requires hundreds sometimes thousands of dimensions to accurately map. >> So to really grasp this we have to look at the mathematical playground grassman created the vector space itself. >> Let's transition into that. Let's look at the actual architecture of this space. First, let's decode the arrow. >> Right? Let's translate vectors into plain language for a second. A standard number, what we call a scaler, like the number five, is just a quantity. >> It's just an amount. Like a car speedometer. It just tells you how fast you are going. >> But a vector is an arrow giving you two pieces of information at once. Distance and direction. >> Like your car's GPS. It tells you how fast you're going and exactly where you are heading. Like walk five miles directly north. >> Exactly. So if you are an actuarial student, you know that a formal vector space requires two fundamental operations to be valid. >> Vector addition and scalar multiplication, >> right? And the space must be closed under these operations. >> The principle of closure is everything here. It means that if you take any vector in that space, say our arrow pointing north, and you stretch it by a scaler, >> the resulting stretch arrow still lives in that exact same mathematical space, >> right? Like walking 10 miles north instead of five. That's rule one, stretching or shrinking. >> And rule two is headtotail addition. If you add any two vectors together, the resulting vector is still in that space. >> So walking north then turning east resulting in a single new arrow pointing northeast. Geometrically we are just stacking arrows. >> That is the basic linear algebra. But let's connect this to the bigger picture. Why does the actuarial profession care so deeply about the closure of scalar multiplication and vector addition? >> Right? Why does this matter to the person managing billions of dollars? >> Because those two geometric rules of stretching and combining arrows perfectly map to the exact mechanisms of building and scaling financial portfolios. >> The vector space is literally isomorphic to the management of risk. >> It's exactly let's take scaler multiplication first. Imagine you have a vector that represents the exact risk profile and return projection of a specific you know commercial real estate trunch. >> Okay. So the vector maps the direction of that asset's behavior under various economic pressures and the magnitude is its expected return. >> Right? Now what happens in the real world when a pension fund decides they want to invest twice as much capital into that specific real estate tunch. >> They just apply scaler multiplication. They take that assets vector and multiply it by a scaler of two. And because of the axioms of grassman's vector space, they know with absolute mathematical certainty that the fundamental direction of the risk hasn't changed. The underlying nature of the asset is identical. Only the magnitude of the exposure has stretched. >> The math guarantees that scaling your investment does not inadvertently twist the geometric orientation of your risk profile into some unpredictable new dimension. >> That is the elegance of linearity. And the exact same logic applies to vector addition. Right? >> It does. [snorts] Think about merging businesses. If you take a massive life insurance portfolio, which is just one huge vector representing mortality, risk, demographics, stuff like that, >> and you combine it with an auto insurance portfolio, which is a totally different vector pointing in a completely different economic direction. >> When you combine those businesses, you are doing head-to-tail vector addition. And the mathematical guarantee of the space is that the superposition of those two distinct risks will result in a single perfectly calculable new vector. >> You are generating a new combined risk profile whose properties can be entirely derived from the original two. >> You don't have to guess what the new combined company looks like. >> The geometry literally tells you. >> But um okay, let me push back a little bit here. >> We are treating this linearity like it is some unbreakable law of physics. We assume that risk in the real world always scales perfectly and always adds perfectly, >> right? >> But in actuarial science, especially, you know, catastrophic risk modeling, this assumption of strict linearity eventually breaks down, doesn't it? >> That is a phenomenal point and it is exactly where the pure math meets the messy reality of economics. Yes, the vector space assumes perfect linearity, but in the real world, what happens if you multiply your exposure to a highly illquid asset by a scaler of like 100? >> Oh, you become the entire market. >> Exactly. You become so large that your very presence distorts the asset's behavior. The vector doesn't just stretch, its direction fundamentally warps because liquidity dries up. >> The linearity breaks, >> it shatters. >> Or think about vector addition with catastrophic events. say combining a property insurance vector in Florida with a business interruption vector in the exact same state. >> Ah right. If a category 5 hurricane hits, those vectors don't just politely add together headto tail. They violently amplify each other. >> The superp position fails because systemic failure is highly nonlinear. >> Which is why actuaries can't just be pure mathematicians. They have to be risk philosophers. You use Grassman's vector space because it is the most robust logically consistent framework we have for modeling up to the point of catastrophic failure. It's the baseline. >> But you always have to be aware of the boundaries where the real world physics of the market just tear the mathematical canvas. >> Absolutely. >> Okay. So that perfectly sets up the next critical layer of the math from our source text, >> the compatibility test. >> Right. We know what the arrows are and we know how they move and combine. But how do we know if two arrows are actually working together or fighting each other? >> Because a basic vector space only gives you an fine structure. It tells you how things add in scale, but it does not give you a metric structure. >> Meaning a basic vector space alone doesn't actually allow you to measure angles or distances between the vectors. >> Exactly. If I have a massive portfolio of thousands of different asset vectors, simply knowing I can add them together isn't enough to survive. >> I need to know how those vectors interact. Are they moving together? Are they totally opposed? >> I need to measure the angles between them. And to do that, you have to introduce a new tool, the inner product space. >> This is the engine of risk measurement. If Grassman built the playground, the inner product is like the laser grid that measures everything happening inside of it. It takes any two vectors in your n-dimensional space and runs them through a specific operation, usually the dotproduct, and collapses their relationship into a single scalar value. >> Now, any actuarial student listening knows the basic formula for a dot product. You multiply the corresponding components of the two vectors and you sum them up. >> But geometrically, what is that calculation actually revealing about the financial reality? >> Right? It is an operational measure of alignment. It is mathematically extracting the angle between the two vectors. >> If you look at the fundamental formula, the inner product of vector A and vector B is equal to the magnitude of A time magnitude of B time the coine of the angle between them. >> That cosine is the absolute secret weapon because it dictates the entire spectrum of financial correlation. >> Let's break down that spectrum. Yeah. >> Imagine two vectors pointing in the exact same direction. >> Okay. Say the return profile of two highly correlated large cap tech stocks. >> Right. The angle between them is 0 degrees and the cosine of 0 is one. >> So the inner product is mathematically maximized. It is a huge positive number. >> And translated into the language of risk, a maximized inner product is a massive alarm bell. >> It means your assets are in lock step. If a macroeconomic shock hits that specific trajectory, both assets will crash simultaneously. >> You have absolutely no diversification. Your risk is concentrated. Okay, so let's swing to the opposite end of the spectrum. What if the vectors are pointing in perfectly opposite directions? >> The angle is 180° and the cosine of 180 is ne1. >> So the inner product becomes a large negative number. >> In finance, this represents perfect negative correlation. If one asset loses value, the other reliably gains value. >> This is the mathematical basis for hedging, right? You intentionally buy a vector that points in the exact opposite direction of your primary exposure. >> Precisely. But here's where it gets really interesting. The holy grail of portfolio management isn't just hedging. It is finding true independence. >> What happens when you find two vectors where the angle between them is exactly 90°. >> The vectors are perfectly perpendicular and the cosine of 90° is exactly zero. >> Therefore, the inner product is exactly zero. In linear algebra, we call this orthogonality. >> Orthogonality is without a doubt the most beautiful word in the actuarial dictionary. >> Wait, so if they don't share a direction, what happens to the math? >> When two vectors are orthogonal, they share absolutely no common directional components? Moving along the trajectory of vector A gets you absolutely no closer and no further away from the trajectory of vector B. >> They are completely independent. They operate in completely mathematically independent dimensions. Yes, finding a mathematical zero here isn't just some empty result in the context of risk. Finding a zero finding independence is the ultimate prize. >> So if I have a stock vector and a bond vector and their inner product is zero, it means an economic disaster that completely destroys the stock vector will theoretically have zero impact on the bond vector. >> That is the mathematical definition of perfect diversification. When your co-variance matrix is filled with zeros between assets, your systemic risk plummets. >> You have literally insulated the retirement funds through geometry. >> Exactly. But um I should note the math here can be almost too clean. >> What do you mean? >> Well, we are equating geometric orthogonality, a zero inner product with absolute statistical independence. But from a purely statistical standpoint, a correlation of zero does not always guarantee that two variables are truly independent. >> Ah, right. That is a highly sophisticated distinction. Geometric orthogonality proves there is no linear relationship between the two vectors. >> The linear correlation coefficient is zero, but it does not rule out complex nonlinear dependencies. >> Right? You could have two assets where the inner product of their returns over time is zero. So they look perfectly orthogonal on paper, >> but under the surface, asset B's volatility might be massively dependent on the squared returns of asset A. >> The geometry says they are safe, but the statistical reality says they are deeply entangled. >> Exactly. The only time a zero inner product strictly guarantees true statistical independence is if the two variables follow a joint normal distribution. >> And spoiler alert for the real world, financial markets are almost never perfectly normally distributed. >> Never. They have fat tails. They have skew. >> Which is exactly why modern actuaries use advanced tools like copulus to model tail dependencies rather than relying solely on the linear inner product. >> But the inner product space remains the foundational bedrock. You just cannot optimize a multi-billion dollar portfolio without first mapping the orthogonal relationships of its coariance matrix. >> Okay. So, let's take these abstract arrows and apply them to the literal billions of dollars our listeners will be managing in their careers. Let's look at actuarial reality number one from our text. Taming the portfolio. >> Yes. Applying vectors to portfolio management and investment strategy. Let's set the scene. Visualize a massive state pension fund. >> Okay. They are managing $50 billion >> and they don't buy one stock. They are invested in 5,000 distinctly different assets worldwide. equities, sovereign debt, emerging market infrastructure, commodities. To someone looking at a normal spreadsheet, this is just a chaotic database with 5,000 rows updating every second. >> But to an actuary utilizing Grassman's logic, that entire portfolio is visualized as a single massive vector. >> One single point existing in a 5,000dimensional vector space. The exact percentages of money allocated to each of those 5,000 assets represent the specific coordinates on those axis. >> That is a staggering visualization. One arrow suspended in a 5,000dimensional void. >> And the actuary's job is to continually adjust the coordinates of that single vector to keep its tip resting precisely on the efficient frontier. They are trying to maximize the expected return which is the length of the vector while minimizing the portfolio variance >> which is derived directly from the inner products of all 5,000 asset return vectors. >> So how do you possibly measure the total risk of thousands of moving parts? I mean the sheer computational weight of that geometry is absurd. >> It is if you have 5,000 assets to understand the total risk you have to calculate how every single asset interacts with every other single asset. You have to calculate the inner product for every possible pair which means building a covariance matrix. >> A 5,000x 5,000 grid. >> That is 25 million inner products that have to be calculated and updated constantly. >> 25 million measures of geometric alignment. And here is where the math hits a massive real world obstacle, >> right? Because if we are calculating 25 million correlations based on historical market data, how much of that coariance matrix is actually capturing true persistent economic relationships >> and how much of it is just capturing pure statistical noise? >> That is the curse of dimensionality. As the number of dimensions in your vector space explodes, the volume of that space becomes so unfathomably massive that your data points become incredibly sparse. >> Exactly. If you are calculating the inner product of two assets in a 5,000dimensional space using only say five years of daily historical returns, >> the math will almost certainly find phantom correlations. >> It will calculate an inner product that looks highly significant, but it's completely spirious. It's an artifact of the highdimensional geometry, not a reality of the market. >> So, how does an actuary handle a 5,000dimensional vector space without the noise completely destroying the optimization? Because if modern portfolio theory requires the inversion of that massive coariance matrix to find the optimal weights and the matrix is full of noise, >> the resulting vector is going to be wildly unstable. >> Exactly. This is where dimensional reduction techniques become mandatory. An actuary cannot use the raw 5,000dimensional matrix. >> They have to mathematically force the noise out of the system. >> They use techniques like principal component analysis, PCA. Okay, let's dive into PCA because it is pure vector geometry in action. >> Principal component analysis essentially looks at that massive cloudy swarm of 5,000 asset vectors >> and it mathematically hunts for the single direction through that 5,000dimensional space that captures the maximum amount of variance. >> It finds the primary axis of risk. >> It finds the main highway of the market movement. Right. >> Exactly. And it defines that main highway as principal component one. Then it mandates orthogonality. >> Right. It searches for a second direction that captures the maximum remaining variance but with a strict mathematical constraint. >> It must be perfectly orthogonal, a zero inner product to the first component. >> And it repeats this process building a whole new set of orthogonal axes, a new basis for the vector space. >> And the brilliance is that the actuary might find that the first 20 orthogonal components actually explain 95% of all the movement in the entire 5,000 asset portfolio. So they just discard the other 4,980 dimensions. >> They collapse the 5,000dimensional space down to a tightly controlled 20dimensional space. >> The noise is strip away and the coariance matrix becomes mathematically stable enough to actually invert and optimize. They use orthogonality to compress reality into something computationally survivable. >> It's beautiful and it's a perfect segue into actuarial reality number two from our text collapsing dimensions for asset valuation. >> Right? Because PCA is an empirical way to collapse dimensions based purely on the data. But actuaries also use theoretical frameworks to map the vectors of individual assets. >> We need to explore multiffactor models. How does an actuary actually price the risk of a complex financial instrument before it even hits the market? >> You have to accept that an assess return is not just one thing. It's not a single scalar reaction. >> No, it is driven by a vector of macroeconomic forces. M >> you have to build two distinct vectors to understand the asset. >> First you have the macroeconomic factor vector. >> This vector represents the fundamental forces operating in the global economy at any given moment. >> Right? So if we are using a five factor model, the coordinates of this vector might be the unexpected change in inflation, the shift in the yield curve, the growth rate of GDP, >> changes in corporate bond spreads, and maybe a marketwide equity risk premium. That five-dimensional vector represents the environment is the raw weather of the financial world. >> But simply knowing the weather doesn't tell you how a specific house is going to handle the storm. >> For that, you need the second vector, the assets sensitivity vector, how strongly that specific asset reacts to those forces. >> In finance, we usually call these the assets betas. The beta vector maps precisely how violently and in what direction this specific asset reacts to each of those five economic factors. >> I like to compare it to like an allergy test. >> Oh, that's a good analogy. >> Right. The macroeconomic vector is the environment, the pollen, the dust, the dander in the air. >> And the sensitivity vector is your specific immune system. >> Exactly. Or think of it like the specific EQ settings on an audio soundboard. The macroeconomic vector is the raw audio signal coming into the board. The bass, the midtones, the treble. >> And the beta vector is how the sound engineer has pushed the sliders for that specific channel. >> Right? If the inflation slider is pushed way up, this asset is hyper sensitive to inflation. If the yield curve slider is pulled down to zero, the asset is completely orthogonal to interest rate shifts. So you have these two vectors in a five-dimensional space. The economic reality and the assets specific sensitivity profile. How does the actuary determine the actual expected return? >> They run the mathematical magic. They take the inner product. >> They calculate the dotproduct of the macroeconomic vector and the sensitivity vector >> and the geometry takes all those messy swirling economic dimensions and aligns them. It multiplies the inflation factor by the assets inflation sensitivity. >> It multiplies the yield curve shift by the assets yield curve beta. >> Yeah. >> It calculates the alignment across all five dimensions simultaneously. >> And it collapses them into a single precise expected return. >> That single number that the actuary needs to price the policy or balance the ledger. >> It is incredibly elegant. It shows how seamlessly Hamilton's desperate carvings and grassman's abstract spaces have been retrofitted to run global finance. But you know, as our source text points out, five dimensions or even 50 dimensions in a multiffactor model is practically archaic compared to the absolute bleeding edge of the industry today. >> We have to move from the macroeconomy down to the hyperspecific reality of the individual policy holder. >> Which brings us to the final application, the frontier. You are a multi-dimensional arrow. >> Applying vector spaces to data science and machine learning in modern insurance. We are talking about mapping human behavior in highdimensional space. Let's look at the sheer scale of a modern insurance database today. >> We aren't looking at small sample sizes anymore. An autoinsurer might have 10 million active policies. >> 10 million individual points of risk. >> And for every single one of those policy holders, the database isn't just holding basic facts like their age and their zip code. >> The future engineering is staggering. A single policyholder's row might contain 50 or even 500 distinct known facts. >> It includes their exact driving calmetry, how hard they break, what time of night they drive, their average highway speed, >> their medical history, their credit utilization ratio, the square footage of their home, localized weather patterns. >> Every single one of those distinct features is treated as a completely independent mathematical dimension. >> So what does this all mean? When the machine learning algorithm looks at policyholder number 4,212, it does not see a human being. >> It plots a single vector in a 500dimensional vector space. >> Your life quantified, stretches the arrow along the breaking force axis, twists it along the credit score axis, and extends it along the commute time axis. >> You are locked into a highly specific geometric coordinate. And there are 10 million of these arrows hovering in this 500dimensional space. So the actuary builds neural networks and runs clustering algorithms like K means or DBS scan to find patterns in this massive geometric cloud. >> They need to find the vectors that are acting identically so they can accurately set hyperpersonalized premiums or detect organized insurance fraud. >> But this raises an important question. What is the computer actually doing when it runs these AI algorithms? >> Right? Because we use terms like neural networks and artificial intelligence and it makes the computer sound like it possesses cognitive intuition. But the AI is blind. It's literally just doing geometry. It is simply calculating distances. >> That is the core of machine learning. The algorithm rapidly calculates the mathematical distance or the inner product between millions of different vectors to see who clusters together. >> But calculating distance in a 500dimensional space introduces some bizarre mathematical paradoxes, doesn't it? >> Oh, absolutely. If the computer tries to use standard uklidian distance, you know, the simple straight line distance we use in the physical 3D world, the math starts to fail. >> Why does it fail? I mean, the uklidian distance formula is just the square root of the sum of the squared differences, right? That should work in any dimension. >> It's another manifestation of the curse of dimensionality. In extremely highdimensional spaces, the sheer volume of the space becomes so vast that the concept of empty space takes over. >> Okay. So, everything gets really spread out. Exactly. The uklidian distance between almost any two random vectors becomes mathematically indistinguishable. The concept of near and far literally breaks down >> because the 500dimensional void is so uniformly huge that every single person looks incredibly far apart from everyone else. >> Right? The distances are mathematically meaningless for clustering. >> Wait, so if Uklitian distance breaks, how does the AI actually figure out which policy holders are clustering together? How does it find the fraudsters? They abandon the measurement of absolute distance and they return to our most powerful actuarial tool, the inner product, but they normalize it. >> They use a metric called cosine similarity. >> Yes. Which is literally just the geometric angle between the two vectors stripped of their magnitude. >> So the AI completely stops caring about how long the vectors are. It stops caring if one person has been a customer for 20 years and another for 2 months. >> Exactly. It solely analyzes the direction those 500dimensional arrows are pointing. It calculates the inner product, divides it by the product of their lengths, and isolates that pure cosine value. >> It looks for angles that are approaching 0 degrees. >> If the AI finds a cluster of 50 vectors, all pointing in the exact same, highly specific, obscure trajectory within that 500dimensional space. >> The algorithm flags it immediately. >> It says, "Look, these 50 people live in different states. They have totally different names and their policy sizes are different. >> Their uklidian distances are far apart, but geometrically their 500dimensional behavior vectors are perfectly parallel. >> They share the exact same telemetry anomalies and the exact same credit quirks. And that is how you uncover a sophisticated distributed insurance fraud ring. >> The fraudsters try to hide by randomizing their surface details, but they cannot hide the fundamental geometry of their behavior in a 500dimensional space. the inner product exposes them. >> Conversely, it is exactly how the actuary justifies hyperpersonalized pricing. >> Right? The AI finds a cluster of vectors pointing in a trajectory historically associated with absolute safety. Their cosign similarity to historical claims is perfectly orthogonal. >> The math proves their independence from typical risk factors, allowing the insurer to confidently drop their premium to capture market share. >> It is stunning to step back and reflect on how far we've come. From Hamilton scratching equations into a bridge to computers finding frosters in 50-dimensional space, >> it fundamentally rewrites how you view the tools of the trade. I mean, we started with Hamilton driven to the brink of despair because physical 3D space refused to be captured by standard arithmetic. >> We watched him shatter the commutative law just to map a simple rotation. And we watch a grassman take that mathematical rebellion and abstract it into infinity, defining the rigid, perfectly logical axioms of the vector space that liberated math from the visual world, >> which gave actuaries the precise ability to mathematically stretch and combine risk portfolios, knowing the underlying geometry wouldn't spontaneously collapse. We explored the inner product space, moving from mere addition to the actual measurement of alignment. Unpacking how the cosine dictates financial correlation. >> From fatal concentration to the ultimate prize of true orthogonal diversification, >> we applied that geometry to the co-variance matrices of massive pension funds, fighting the noise of the curse of dimensionality using PCA. >> And we used the inner product to collapse the macroeconomic vector and the asset sensitivity vector into a single scalar price. And finally, throwing the individual human being into the 500dimensional void using cosign similarity to find parallel trajectories of fraud. >> The ultimate takeaway for the listener today is that when you look at a massive spreadsheet in your future actuarial work, you shouldn't just see flat numbers. >> The flat grid of rows and columns is just a user interface. >> You should see arrows. You should see geometry. You should see the shape of risk. >> Once you see the geometry of risk, you can never go back. And that realization leaves me with one final deeply provocative thought for you to mull over today. >> What's that? >> Well, we have built these incredibly sophisticated 500dimensional vector spaces to map every measurable telemetry point and financial metric of a human being. >> And the math works beautifully. >> It does. But if Herman Grassman proved that our math allows for infinite dimensions and modern actuaries use 500dimensional vectors to measure human behavior, >> what dimensions of human risk are we currently failing to measure simply because we haven't thought to add them to our vectors yet? >> The unmeasured axis. >> Exactly. What behavioral biases, what emerging geopolitical shifts, what cascading psychological network effects are currently warping the trajectory of our portfolios simply because we haven't defined them as a mathematical feature. >> We might be optimizing perfectly within our 500 dimensions, completely ignorant of the fact that the catastrophe is approaching from dimension number 501. >> That is the true frontier, finding the missing dimensions before they find you. Thank you for joining us on this deep dive.