Lessons

Lesson Info

Analyzing Systems

Any questions from what we covered anything about systems things that you've been percolating on things that you're not quite sure about before we get into really the nuts and bolts of looking at a really system and trying to figure out what to do with it can you elaborate on elevation and give some more examples yes so so elevation so going back to the gold that's theory of constraints elevation is the idea of adding more capacity to the system whatever adding more capacity means and so in the case of the windshields it was buying another machine so you can make twice as many windshields that's spending resource is to give you more capabilities to do the thing that's holding you back that looks very different in very different circumstances so if you are programming a website and your web server keeps going down because too many people are trying to use it elevation is buying a bigger beef your machine so it has more capacity so it doesn't do that uh whatever you're really purchasing ...

or investing or increasing your capabilities by buying something that's elevation any other questions um it's regarding the concept of change so like in my business and in my future business I think so I sometimes struggle with accepting that ok this is going to change in my office is it's working really smoothly right now and now this person is doing something different so my question is how long should a business owner wait if they see that somebody in their business is not adaptive to change um the rial are the way to answer that question is I understand why that person is in the system and what they are responsible for doing and then it's asking yourself in the moment is this person really doing the thing that they're here to do and if they are that's great and if you see that there's going to come a time when that person is not really capable of doing what they're there to do, you can start to prepare in advance for a transition dab in uh the same thing really happens with if somebody is in your business helping you out but you know this is you know, not the their dream job you know that this is this is something that they're doing for a certain amount of time to serve a purpose, but they're probably going to eventually go off and do something else you can start to prepare for that early. So I think the big thing with change is a lot of us have would, particularly when we're running a business we have this idea that if only I get everything right if I can build this particular system for I can build this process if I can have the right people when I get it right everything is going to be perfect and it's gonna stay that way forever it just doesn't work that way people come and go customers come and go people move on tio two other things and that's ok so if you release that expectation of I need to get a perfect and then never ever change it you khun really focus on what's actually happening which is this business is you know in a very real sense a living breathing moving changing thing in and of itself and it will it will adapt it will evolve it will maybe grow into something sometimes very completely different over time and that's ok that's part of the process it's just having that expectation as you're building it instead of trying to keep things static all right so we understand the basics of systems theory and what systems are how they work moving parts flow stocks constraints all of that wonderful stuff now it's time to start using this very basic knowledge of what systems are and how they work and the first thing before we can really change how a system operate wait we need to have a good functional understanding of how it's operating right now what's working and what's not and you wait the way you do that is by analyzing the system by looking at what's happening as the system is working and collecting some information collecting data collect doing certain measurements to figure out okay where are we right now and where would we like to go? And that information can help you figure out, how do we get from point a to point b as quickly quickly as you can so you can't go through the parts of the theory of constraints, right? You can't decide to spend a million dollars on a new windshield machine if you don't first measure exactly how everything is going right? So that's the purpose of this section, we're going to learn how to look at a system as it's operating and figure out whether or not is working. Um now here's, the thing as we were talking about change, ideally, it would be easiest if we could just snap our fingers and freeze everything right, put the system in stasis and then spend a lot of time and pick it apart and figure out what's working and what's not, and then snap her fingers again and start the system again without anything changing. That is not how you actually measure systems in practice, you can't stop the system it's always doing something, so when we're analyzing a system, we're analyzing something that changes and something that we cannot stop, so we have to learn how to measure as the system is going in a way that doesn't actually impede the functioning of the system, whatever it is, they're a bunch of different ways to do that. The first idea is something that may look familiar our good friend from day one deconstruction so the reason that we talked about deconstruction in the skill acquisition section is we're trying to do something complex, right? We're trying to figure out how we can make this complex system called our body and our mind learn how to do something new, and the skill that we're trying to learn is often very complex. And so what do you do to understand a very complex thing? Break it down, break it down into smaller parts and the smaller the parts are the easy they are easier, they are in general to understand, and so you can take something very complete list let's use the technology that we're using before he went to lunch, take a car if you've never opened the inside the hood of the car and actually looked inside, you'll just see a jumble of part and who knows what all of those parts do right? When you're looking at the entire system all at once, it appears a very complex. What we can do is break it down and the way you break it down as you look at what looks like an inflow into something and what looks like an outflow, and then you look at that part in isolation and you can start to brake the car apart. You can examine the engine in isolation, you can examine the spark plug in isolation, you can examine the breaking section in isolation and when you do that you do break it down into these smaller parts and understand what each of the parts do and how each part goes into the other partner, how each system goes into the next system all of a sudden if you do the deconstruction well, you can put that knowledge together into how the whole thing works. So when you have a failure like you're on the side of the road and the car is not working, understanding how to deconstruct the system into smaller parts gives you a much better idea of where to look for trouble based on what you're seeing that makes sense. So deconstruction really is the fundamental thing that we're doing when we're measuring a system where break, making complex things very, very simple look at them in isolation to see whether or not they're working go measurement measurement is the process of collecting data as the system operates. So as the system is doing what it is designed to do or what it is has been built to do, you can watch you can measure you khun figure out what's working what's not how quickly things air working, how slowly they're working, how much of something is producing whether that is going up and down there all sorts of different things that you can measure measurements the very basic types of measurements are usually just very basic counting addition and subtraction types of things looking at the system as it's operating you know let's let's take our automotive factory assembly line you can measure use a watch stand at the end of the production line and you say how much I'm going to stand here for an hour how many cars come off the end of that line? It's a measurement doesn't need to be any more complicated than that. If you're running a website, there are wonderful tools that can let you know how many people are here watching this course right now and that's just a very basic counting exercise you can do all sorts of things like when you're running a computer program, how long does it take for that computer to run a one run in particular process? There are millions upon millions upon millions of different measurements you could take and they're all very basic looking at the system counting, adding, subtracting that's it that's measuring it's not any more complicated than that. Now there are millions of things that we could measure, but not all of them are equally important, and so the idea that some measurements arm or important than others is an idea called keep performing it's indicators usually abbreviated two k p I so if business people want tio to impress you with their competence and fluency in business type language they'll throw acronyms like k p I so now you know what it means ahh key performance indicator basically just means very, very very important measurement it's one of the really critical things that defines if the system is working or not and so the question is of all of the millions of things that you could measure what are the measurements that really are critical? What are the key things that indicate whether or not your business system in particular is working any hypotheses or theory is about what would define a key performance indicators versus and not keep performance and to get what makes something important to measure something that influences let's say profit yeah ok that's not quite there but yes so profit is an outcome that you want write anything that goes that helps you measure profit it's a good thing no seals yeah yeah what's what's a principal what's a criteria that would make something really key to understand remember what the very first thing that we talked about the five parts of every business what makes a business of business the critical things that a business must do in order to work here's the criteria if the measurement is directly related to one of the five parts ofyour business it is a key performance indicators so measuring things like how quickly do you create something of value which we'll talk about here in two minutes called throughput that's key performance indicators how many prospects you attract and how interested you make them is a key performance indicator how many prospects that u turn into paying customers? It is a key performance indicator how happy those customers are to do business with you how quickly you're able to deliver value to him that's key performance indicators and then the measurement of how much money is coming in how much money is going out and is it enough those things are key performance indicators so by definition if it's one of those things you have to pay attention to it because those are the things that most directly improve your business pretty much anything else not so important to take a look at unless it's directly related to one of those things so this is one of the first things in coaching or advising particularly a new business owner that I really really stress most people aren't measuring things that are actually important and so this is something the nuances of exactly what you measure differ for every business the key is breaking down so doing that very basic deconstruction on your own business how do you create value how do you market how do you sell how you deliver value and what is important to you in terms of finances you need to break down your business structure into those component parts the deconstruction comes first then for each of those parts you just ask yourself this simple question how can I measure how well I'm doing sometimes in terms of counting the prospects coming in the door that's pretty easy you just count sometimes like for example, if your business runs on a website and that website is doing the selling and your website is down as people are looking at the web site and wanting to buy from you, that downtime for your website becomes a key performance indicators because the less your side is up the list people could buy from you right really, really important and so the absolute key performance indicators it's very based on the business, so you have to do the deconstruction for yourself. But what makes a key performance indicators critical is that it's related to the five parts of every business doesn't make sense. Yeah, uh, something like something that's like subjective, like coaching or consulting in terms of value creation will be k p I or key performance indicators retention of your customers. So if somebody so you're providing value in a subjective way so the person who determines if value is being in is being provided or not is actually you're paying customers in a way that you can back into something is subjective is that is just asking yourself a question are they likely to stay on coaching or consulting with me if I am not providing value and probably not right so if your customers are staying with you for a long time you can use that this is actually a form of measurement we're going to talk about more detail in just a few minutes called a proxy so it's measuring something else to kind of triangulate into measuring something that's really important to you there's all sorts of different ways to do this but it's like in that example what if your value creation is really good but the rest of it is really bad so you won't be able to measure retention if you're not promoting sales so but then if you're measuring attention will say that bradley creation is not good maybe that is good so you find another proxy and there's all sorts of different ways that you could do it and then that's actually very dependent on what it is that you're actually selling and how long that cycle is so if you have specific questions we can talk about that that here in a little bit but yeah that's if you can't measure directly you measure it indirectly we'll talk about that hey everybody understand what a key performance indicator is here's the rule of thumb your business probably has five or six measurements that are really, really, really, really important and so if you're going to measure anything decide with those key performance indicators are for your business track those like a hawk and you can pretty much safely ignore everything else the number of retweets on something you post on twitter is probably not one of your kp ice so there's there's also you know this enormous tendency of we pay attention to measurements that are really really easy to pay attention to we personally tends to pay attention to measurements that are very social signally right everybody loves me on the internet fantastic not related to the five parts of your of your business so it doesn't actually help you that much that's the the key you have five to six of these things pay attention to those above and beyond anything else general principle of collecting information in accurate measurements is something that we've probably all heard of at least at one point in our life for another garbage in garbage out if the information that you're collecting is flawed from the beginning, you're not going to be able to use it to make better decisions and the sounds like absolute complete common sense until you actually look at how most businesses function and you see that the information that they're working from is just not very high quality at all. So one of the things that I did went in my corporate career at procter and gamble was using measurements from the web to figure out whether marketing things were working or not and very early we found out that the system that we were using to measure how many people were coming to this particular website I wasn't counting just people it was counting programs like google uses to crawl the entire internet for search engine indexing so we're getting these enormous numbers of visitors coming to our website millions and millions of visitors but when you took the google bought out of it, it was like maybe a couple thousand visitors so that's garbage data right there's a problem they're so when you're measuring things you really need to pay very close attention to what is the source of the data? How reliable is it? Can you you can you rely on that to make a decision in a way that's going to get you accurate information and so it pays as you're setting up these types of processes just doing a little check step of how high quality is this information can save you a ton of time and and a lot of heartache because I guarantee you if we were optimizing for the google bought in the cases of of the campaigns that I was working for, we would have made some really bone headed decisions no tolerance tolerance is just like normal accidents that we talked about earlier tolerance is an acceptable level of normal error in a system things happen, things break and the key is it's okay, if accidents happen or things break every once in a while, but if things are breaking all the time, if accidents are happening all the time, you really do have a legitimate problem. And so a tolerance is a way of defining how many errors can you really take? What is the point at which you start? Tio, you need to do something different. You should start to be concerned and put a plan in place to avoid these things breaking so one of the most common failures and actually websites are really great example of this most business people when they create a website and they they haven't had very much experience and putting something on the internet and having it up for an extended period of time web site will be up will be really excited, and then the website will go down. They freak out right freak out because the web site has to be up all the time. One hundred percent reliability is what we're going for when you look at what it takes to make sure a website is up one hundred point zero zero zero zero zero percent of the time you are looking at an investment that is approaching the hundreds of millions of dollars, if it can be done at all, so companies that really do rely on being online all the time. Like for example, facebook would be in that category spend enormous amounts of money and enormous amounts of staff that have the sole responsibility of making sure the site never goes out. If you're running a site for your business that is not that particularly important or doesn't have that your entire business is not online guess what your side is going to go down it's gonna be okay and you can do things to minimize the probability of that happened but you know what it's going to be down for a couple minutes once a week or twice a week in the world is not going end that's what tolerances right so it goes down it's down for two seconds comes right back up ok, so what no big deal so that normal amount of air is okay where you need to start worrying is establishing that tolerance and if it becomes more than that then that's a signal to you that something's wrong I need to do something different does that make sense? Analytical honesty? It is kind of the upshot of all the things we've been talking about in terms of measurement quality it's very, very comfortable sometimes to use data to fool yourself into proving something you already believe or one picture it really is and sometimes it's difficulty if if what we're doing is really meaningful to us having data that it's not working can be intensely uncomfortable even maura uncomfortable if you're looking at this data and you have to report it to your boss or the ceo or the president of your company or an investor or somebody who is not going to like what the data says and so analytical honesty, this is a cheque step for yourself is the purpose of measurement the reason we're doing all of this is to collect information about the system and to use that information to do something different, change our behavior in some way shape or form a change the system in some way shape or form to have a better result that's the on ly purpose of it the purpose is not to make ourselves feel good. And so if the data says something that you don't like, you want to know that as quickly as possible. Here's the back half of my uh google bought story from from my marketing campaign days, so I found this issue we're getting all sorts of visitors that aren't really visitors their computer and computers don't thought it don't really care about blind laundry detergent, they just don't write, so so we see this problem we say ok, we need to fix it here's, how to fix it, we're going to use another measurement system that will give us a more accurate count and in order to to implement this, we need to go to the manager of this particular brand saying, hey, we need to install this new system on your web site. It'll give us more accurate measurements. So we did that, and the person who is responsible for it said here were saying not gonna do it like, really that's weird, because google but doesn't care about buying laundry detergent, why? And they said, if we do that, I am going to have to report to our president that we had ten percent of the traffic to our web site this month that we had last month. It's gonna make me look bad, not gonna do it don really done right? But the loss of social status was such that they were they were comfortable, they were more comfortable having inaccurate information and a wasting a bunch of money doing things that didn't work. Then they were reporting that there was an air in a measurement process because it was a loss of face. This is extremely easy to do extremely easy, so the object is not to fool ourselves but were the easiest person to full because we want to be. So this is the check step of the measurement process. We really need to kind of steal ourselves before we look at this data, because whatever it says, assuming that the quality of the information is good we need to act upon it we need to be comfortable with doing what the data tells us to do so knowing what you said yesterday about, like confirmation bias and how we look for things that make us feel good and election stuff, how how do you how do you confirm that? Like when you have something that that's that's making you feel good, how do you confirm that it really is? How do you confirm that the data is accurate? How do you make sure that you're not fooling yourself? Now? The first thing is to make sure that the data you're looking at is actually your k p I stuff. So you're collecting the right measurements in the first place, and the trick is to look at all of the data and not necessarily the data that tells you happy things, and so you'll find very quickly if you're tracking your k p I as well when something goes off kilter that's a signal that there's something there toe pay attention to and sometimes this is the value of an outside of the data is actually telling you something you don't want to hear, you need to pay attention to it anyway and so here's the resolution over the png story this is a problem, right? The individual site owners didn't want this change to happen because it would make them look bad but the president of this particular division wants good data to make good decisions and so would he have to do is convince the president that this is a systematic problem everybody is affected by this it's a problem on all of it we're going to implement the change to everybody and everybody is going to not look bad it's just a change in how reporting and we're going to make it across the entire system and the president is not going to be disappointed when the number of people on that particular measurement goes down this month because they know it exists, right? So it was a social solution to a measurement problem is because of all the social things happening there sure does that make sense need to be honest with ourselves context now as we're collecting information it's important to be able to collect not just one thing so when we're setting up measurement processes there's there's this enormous tendency toe want to look for the one magic number? Let me know the number that I need to focus on and I will I will make that as high as we need to as they go as low as it needs to in order to succeed we want the one number here's the thing one number in isolation tells you precisely nothing absolutely nothing so let's say our hypothetical company made a billion dollars last month and revenue billion dollars ceiling you know is that good or bad? Not enough information? Why not? Because they won't be losing two billion exactly right? So if if revenue last month was two billion dollars and you have one billion dollars this month that is not a good thing. If you had ten dollars last month and you have a billion dollars the next month that's an awesome thing so likewise we're looking at a revenue number all right? We're not looking at profitability so let's say it costs nine hundred ninety nine million nine hundred ninety nine thousand dollars to get one billion well that's actually not all that great in the grand scheme of things you may descend to congratulation you know it's in order for a measurement to mean something you need to be able to compare it against something else and so everything that you collect the principle here is different measurements when you look at them in combination provide context versus each other you can compare one measurement versus another to figure out whether or not it's good or bad the nfl can ask them how do you assess the role each copy I plays an overall performance? Do they have the same value? Not necessarily and so the biggest thing is and the thing that makes answering that a little bit tricky is business owners can have different primary goals at different points in time and so one of the things that's interesting about businesses you can optimize and we'll talk about optimization here in a minute you can optimize for all sorts of different things, so you may say, ok, the emphasis now is on profitability. I want to make that number go as high as we possibly can. What can we do to rearrange things to make that happen? Another business owner may say, I want customer growth, I want to get as many customers into the system as humanly possible, and at some point we may optimize for profitability, but we're not really focusing on that so much right now we want to grow, um, you may have business owners who say, you know what? I want to optimize this business for the amount of dollars we get for the amount of time and energy that is invested in hk creating those dollars so very, very different goals, and what that does to your kp eyes is it changes what you really hyper focus on what you don't and so it's really important to understand what are you trying to do and how does that impact the five parts of your particular business? And then you choose your keep guys from there so uh, common new gs so kind of from an analytical perspective, what happens when k p is conflict after a change for example, change makes one keep you to go way up but another one go way down yeah let's see think of a good example of that um and again it depends on what the goal of the business actually is. So yeah, let's let's say you make a change to your business structure and number of customers goes up really huge and for that particular month, your profitability goes way down the goodness or badness of that particular situation depends on whether you're trying to maximize your active customers or whether you're trying to maximize your profitability. So be really clear I'm trying todo also terek request if you have any books that you recommend to get more, more info on the important kp eyes for each of the five pieces of business yeah, there's there's actually a ton of really great books in in this area and I'll make I'll mention a couple turning numbers into knowledge by jonathan comey is a book about all the things that we're talking about, how to construct good measurements, how to get good information out of it there's another book slightly more academic called how to measure anything it's a textbook I don't remember the author's last name with a president but search for ana names on a winner found the values of intangibles in business exactly good and that's that's a really great book for trying to measure things that you really can't quantify and we'll talk about proxies here in a second but um how to kind of back in or triangulate into things that really are important but are hard to count there's also a really, really great book on presenting information called show me the numbers so if you put together financial reports or like a one page report on here are key performance indicators this month you want to make sure that report is as good as it possibly can be to convey the information to other people show me the numbers is a great book on how to actually do that. Okay, so in general the rule of context is no number is presented in isolation it's always compared to something else that's how it means something now when you're measuring things sometimes you can measure everything so in the case of our automotive assembly line standing at the end of the production line with the stopwatch counting the number of cars that run off the line you can count every single car that comes off the line that's great until you know let's say another hypothetical manufacturing process where you're making millions of items every second really hard to count all of them really hard to inspect all of them really hard too really give each individual unit the level of attention that it would need in order to make sure everything is absolutely perfect it's a difficult challenge, and so what do you do in those situations where you can't really feasibly measure everything all the same time? It's sampling instead of trying to measure everything all at the same time, you take a little bit, you take a random sample, you take just a little bit very periodically, and then you measure that, and by measuring that, if there's an issue in the broader population of things that you're measuring, you're likely to find it. So for example, medical joke? Uh, you can if you want, tio see how healthy a person is, you can take a blood sample, but you probably don't want to take it all because that would be bad, right? You don't take it all. You just take a small sample and you measure the sample, and if something's wrong with the sample, something is probably wrong with the larger system. We use the same thing in business all the time, and so measuring a small percentage of the total output allows ito to still identify those errors without really trying, going through the effort and expense of measuring everything. All the symptoms in general, the more samples you take and the larger the sample is versus the entire population, the more accurate measurement you're going to get, and this is an idea called margin of error. So it's just an estimate of how much you can trust the quality of your sample and the quality of your conclusions so in general if you're using a random sampling strategy you want to make sure the sample is as random as possible because that's how you find errors if it's not very random you may introduce some bias into your sample which influences the measurements you take from it if you don't sample very much or you don't sample very often there's a higher probability of the sample really isn't reflective of the overall thing that you're trying to measure so in general the more samples you take the larger those samples the more confident you khun b in the conclusions that you draw from that so we're not going to get into the very geeky statistics of what constitutes a higher a low margin of error we're not going to talk about confidence intervals and bounds and all that stuff if you're interested there's great information about this on a week wikipedia right? Just look up confidence intervals and you can see all the math you want but this is the general principle if you're if you're using a sample for pretty much anything the more samples you take the larger those samples are the more confident you can be in the results you could hear the conclusions you drop from now we've been talking a lot about collecting data collecting measurements now we're going to actually start kind of doing stuff with it and the first thing that we can do with it aside from just a normal one hundred cars came off a far assembly line. Yea for us, right? The addition and subtraction stuff to do something a little bit more sophisticated and the first thing only slightly more sophisticated it was called a ratio and a ratio is a very simple method of comparing to measurements against each other. You take measurement a you take measurement b and you divide measurement a by measurement b and it gives you a number and you can express that as a decimal or you can express that as a percentage. Now what that tells you remember noah number and isolation really tells you very much of anything. What a ratio allows you to do is compared to numbers side by side at one moment in time. But if you track ratios over a period of time, you can see the relationships changing between things so let's say way take the number of dollars our company is taking in in terms of revenue and we divided by the number of customers that we have served in that period of time what do you get dollars per customer? Ok uh you can do this with anything. Some of some of the very basic things that you use in finance is how much money did we sell this for? Versus how much money did it cost? Gives us our profit margin it's all a ratio it's just a percentage and so ratios allow you to track how things are changing in relation to each other over time it also allows you to compare one type of thing with a certain ratio versus another type of thing with a certain ratio for example, you have two different products, one has a five percent profit margin and the other one has a ninety five percent profit margin which one is better? Which one should you probably invest more time and attention into the ninety five percent one right? You're making more money for everyone that you saw also that's a really good thing. You should probably do that. Okay that's what ratios? Live video comparing two numbers against each other and that allows you to compare versus other things it's very simple it's just division but it's really, really, really hopeful in establishing relationships between different parts of your system. Go topicality when we're measuring a system, it really helps to have a working understanding of what normal is and so typical ity is just a method of defining what's normal what's typical what's the expected result most people understand or have been exposed to the idea of typical ity using the word average and averages just just a way of finding out what the typical member of the population is what's normal. So you take all of the money that you've made from your customers divided by the number of customers you have in the typical customer is worth x right that's just this very, very simple average. There are lots of different types of finding what's typical, and so I'm going to take you all the way back to probably eight grade seventh grade eighth grade mathematics class, right? Different word for average is mean, right? I mean median, which is what is the one in the middle mode? What is the one that occurs most often? And if you want a really, really fast way of getting a typical thing, take the highest number and the lowest number, add them together and divide by two, and that gives you it's called a midpoint, not really good for anything aside from quick estimation but it's easier than at everything so mean media mode midpoint those air, all of the things that we use to determine what's typical and usually in business, you want to use a median. So the midpoint for the mode and here's why let's say we want to know the typical income of a person who lives in silicon valley, for example, now there are some people who in silicon valley make an enormous amount of money there are some people who make not a whole lot of money and if you have those folks in your sample and you try to do an average like what's that we could take all of all of us here in the room and add bill gates and try to find our average income and we would each be worth several billion dollars right on average but the average really doesn't tell us much because we have those out layers that are part of the process right so mode really gives you a more accurate estimation of what is the fiftieth percentile what's the person in the middle what does that look like now similarly um for that's the media the mode is what's typical so you can look at all of your customers and instead of trying to do an average you can look at what are the values that appear most often write your average customer or your typical customer is worth a certain amount of money to your business every year what is typical and finding that typical what what happens the most often is really a better gauge of your decisions if you have a couple of customers that are worth an enormous amount make sense okay topicality now the point of doing all of this is we're trying to find we're doing some pattern matching here going back to what we talked about yesterday right, we're trying to collect a bunch of information about how the system is working we're trying to compare certain things versus each other we're trying to find patterns of what works and so we need to understand ideally we're looking for things that if we do x then why happens if I spend one hundred one hundred dollars in advertising? I am going to get two hundred dollars in profit and do it again we're looking for those very causal types of things I want to act in a way and get a particular result that's causation cause and effect when we're measuring things cause and effect is actually really difficult to establish a lot because these systems are complex there's a whole bunch of things going on at the same time and it's really hard to figure out sometimes whether this specific thing that we do we pull the lever over here and something happened is something happening because we pulled the lever because something else changed in the environment hey, this is the idea of correlation certain things happen at certain times that maybe associated but may not be a cause and effect type of thing so let's say pizza restaurant it is late august early september they spend a thousand dollars on an advertising campaign and their profits go up thirty percent over that much awesome great did spending the money on the advertising cause profit to go up thirty percent probably not. Here's. Why late august, early september is back to school time, there's something happening in the environment that is causing people to order more pizza and it's, not the advertising. So a lot of the measurements we need to be really careful when, when we say thing a caused thing be because we need to think about all the things that happen in the environment, to see if maybe something else, influence to be to lots of different ways to do it. This is just more of a pay attention, because the environment that you're working and really, really impacts this stuff.