Probabailistic techniques in practice

Reducing the number of inputs (II)

As discussed in a previous post, a high number of inputs can cause a number of problems. Therefore, methods to reduce the number of inputs are of value. In the aforementioned post I mentioned that we can take advantage of the central limit theorem, and replace a large number added distributions with a single Normal distribution. However, we can sometimes change our perspective on the nature of the system we are considering, and reduce the number of input distributions that are required.

To demonstrate, consider the situation where we wish to analyze the risks associated with a new venture. An important part of this analysis would be the sales. This could be handled a number of ways, but let’s consider two basic ways. The first would be to specify a growth curve and use this to find the average sales for each period being considered (week, month, quarter or year, etc.), and then we would specify a standard deviation for each period and a distribution.  In this way each period is random and independent of the other, and a distribution is required for each. The second way would be to use a model (like the Bass model or a Gaussian CDF) to define the sales for each period, but this time we only allocate a distribution to each model parameter. This second way requires only around 2 or 3 distributions.

One might ask which approach (distributions for each period or distributions for the model parameters) is the most accurate. In fact, both are probably needed. However, using the approach that only allocates distributions to the small number of model parameters still provides a good representation of reality and requires less effort.  Thus, the general approach of developing a representative model of a situation and focusing upon allocating random variability to key parameters (as opposed to a larger number of intermediate variables) provides an avenue to generate a reliable and efficient model. Such a model makes an ideal first (and potentially final) attempt at analysis and optimization in the face of uncertainty.

Reducing the number of inputs

Some models that are set up to predict the effects of uncertainty and assist with risk management have a phenomenal number of inputs. This is often a result of dealing with large projects or business models where the basic model structure is the same as for smaller cases, but the number of inputs is much larger.

If a distribution is allocated to each input, then there will be a significant load placed on the simulation software. This in turn slows down the speed of simulation. It also increases the chance of an error during the building of the model that will cause errors in the results. For these reasons it would be ideal if it were possible to create a smaller model (with fewer inputs), but that still gave correct results.

One way of doing this is to take advantage of analytical techniques to partially solve the model first. As mentioned in previous posts, one should always consider the central limit theorem and how it can be used. By finding parts of a model that are collections of additions (or subtractions) we identify a section of the model that can be replaced with a simple analytical surrogate. For example a large number of loss lines in a P&L can be converted into a total loss represented by a single Normal distribution. If this proves to be a significant contributor to variability then the analytical surrogate can be easily analyzed to find the largest contributor(s) within.

Therefore, when you are creating your model be vigilant for opportunities to simplify the model by utilizing the central limit theorem. It will increase speed and reduce the chance of error. There are other cases where it is possible to reduce the number of inputs by reconsidering the nature of the model that is needed. But I will talk about that later.

Is it the model or the distributions?

In a previous post I made a brief (but probably controversial to some) statement. I stated that often the type of input distributions used is not that important because the nature of the model has a much more significant affect upon the shape of the output distribution. I therefore stated that when dealing with the input distribution you probably should not worry too much about the moments other than the mean or the standard deviation for input variables.

 This is a fairly bold statement, and it probably requires more support. So why do I believe that you often don’t need to worry too much about the distribution type? This conviction is a result of what I experienced during my PhD. During my PhD I had to model a number of manufacturing operations. This was to ascertain heuristics that could be used to allocate a distribution to variable once the associated method of manufacture was known. I frequently found that it was the phenomena underlying the manufacturing method (and thus the model of that operation) that had the greatest affect upon the calculated distribution. This was because the central limit theorem would often be present because multiplicative and additive operations are so common. In fact, it is almost impossible to develop a model of any real system that does not include a number of additive or multiplicative operations.

This is certainly not always the case. I found that in cases where there a small number of random variables there was a reduced opportunity for the central limit theorem to express itself, and the type of distribution became very important. However, the majority of cases considered in the real world have a large number of random variables. Therefore, the central limit theorem is more common, and the distribution type is less important in the majority of real life cases. Still, be aware of cases where this might not be the case, especially when there are only a small number of random variables.

If you wish to verify this contention of mine, consider the models that you often deal with. You will most likely find that there are a number of additive and multiplicative functions within any of these models. You can also try changing the distribution types to check that they do not matter that much. Of course, keep the mean and the standard deviation the same.

Dimensional Analysis

Dimensional analysis is a well-established technique that some are familiar with. For those who are familiar with it, dimensional analysis clearly presents an avenue for partially (sometimes almost completely) solving for the relationship between inputs and outputs of technical systems. The obvious application of dimensional analysis to probabilistic methods would be to the development of a model that can then be used for probabilistic analysis. However, it can also be used to augment DOE for robustification and to determine how much information is required to properly define random variability in input variables.

 

Because dimensional analysis finds the collections of products and quotients that would be present in an analytical model if it were possible to find, dimensional analysis effectively solves for part of the model. All that is needed is the relationship between the groups. This can be found through experimental methods or thought experiments. Such a model allows us to now apply probabilistic methods to optimize against negative random variability effects. You can read more about dimensional analysis in this book and the combination with DOE in this paper.

 

DOE for robustification requires two things. First, you need to set an array of variables (or parameters) that will be adjusted in turn so that the contribution of each variable to the output can be determined. Second, an ‘outer array’ must be developed so that the affects of random variability can be emulated, and the effects that parameter values can have on output random variability. This typically requires a considerable experimental effort and the effort increases geometrically with the number of parameters. Therefore, if we can effectively reduce the number of parameters we can save considerably on time and effort. How can we use dimensional analysis to do this? Dimensional analysis identifies groupings of variables that will be present in the actual relationship between the inputs and outputs. Because these groupings are products and quotients, changing one variable is exactly the same as making an ascertainable change to any other variable within that grouping. Therefore, we no longer need to adjust each variable while performing the DOE, because we can use the relationships within the grouping to determine the effect of changing the other variables. What’s more, it does not matter which variable within the group is changed. Therefore, we can choose to change the variables that are easiest to adjust. This typically reduces the required experimental effort considerably.

 

To properly predict the random variability that the output will exhibit we need to have an adequate description of the random variability of the inputs. This can lead to even more uncertainty and effort. What can help here is a determination of what information is actually required so that we don’t waste our effort finding unnecessary information. This determination can be made by taking advantage of the central limit theorem. Because the groups found when applying dimensional analysis are products and quotients, the outcome is typically well approximated by a Lognormal distribution. All that is needed to define this distribution is the mean and standard deviation. Because multiplication is a linear process, all we need is the mean and standard deviation of each variable. However, if any of those variables are raised to a power greater than 1 then higher order moments will be needed. The order required is related to the value of the exponent. Therefore, we only need those identified moments to sufficiently define the random variability. This is a bit different for random variables that are in more than one group. These typically require a full distribution. Thus, we can use dimensional analysis to determine what information is actually required to properly define the random variability. I should point out that in my experience the model has had a greater effect upon the output distribution than the moments of a higher order than the variance. Therefore, if you can’t get more than the mean and standard deviation you don’t need to worry about it in most cases.

Six Sigma and Probabilistic design

Most people are familiar with Six Sigma. This includes the goals of Six Sigma, the typical methods used and the culture that goes with it. If not, I am not going to cover these here, but there are plenty of websites that do. One of the key characteristics of Six Sigma is the frequent use of experimental methods. You will note this in most technical texts on Six Sigma. I believe this is a result of the experimental approaches that dominated the quality revolution some decades ago. These approaches originated with quality control methods, which typically were best suited to a statistical approach. Statistics usually requires data, which means an experiment is also required. Taguchi then further reinforced the experimental approach in the minds of quality practitioners when he introduced DOE for robustification. This focus on the experimental approach has resulted in a neglect of the analytical approach. Certainly, there are some out there who appreciate the value the analytical approach and take full advantage of that value. However, there is a stronger focus on the experimental approach in Six Sigma, and this raises the question: can Six Sigma (and Design for Six Sigma) be improved with probabilistic/analytical methods?

 

Well I certainly think so and I think the greatest benefit is evident when probabilistic methods are applied to Design For Six Sigma (DFSS). DFSS is basically robustification so it’s obvious that probabilistic methods are of value: but why DFSS especially? DFSS is used when a new system is being developed. Under these situations it is much harder, at times impractical, to gain access to the kind of data that is required for an experimental or statistical approach. Because the system being developed is new it has not yet been properly implemented, and there is no quality data (on either system performance or component characteristics) that can be analyzed. Therefore, the experimental approach is of limited usefulness when developing a new system. This could be a new product or a new business venture or a one off engineering system like a dedicated machine tool or a large piece of construction like a bridge. In such cases, the analytical approach is really the only option because it is the only way we can gain any insight into the effects of random variability upon system performance. So how should the probabilistic approach be combined with Six Sigma?

 

A full probabilistic analysis of a system can be very intellectually demanding, and thus seem very daunting. Therefore, I think an approach that only relies on the simplest and easiest of methods at the early stages is required. The more demanding methods can be saved for when the final design is selected, and the extra effort becomes worthwhile. If we assume that the Define step of DMAIC has already been taken and that there is little need to Measure (because there is currently nothing to measure), then we can look at the Analyze stage:

• The earliest, and possibly most valuable, step would be a simple sensitivity analysis. I spoke about this in an earlier entry. This does of course require an analytical model of the system, and this was covered earlier too. Such a model at this stage might be a simple approximation, which provides a general ‘feel’, so that less effort is required.
• If a preliminary sensitivity analysis does not rule a design out, then it is time to develop a more comprehensive model. This model can then be combined with Monte Carlo, error propagation or FORM (depending upon the model and your preferences) to make a more accurate prediction of quality.
• Now that it is possible to predict the quality, the next step is to optimize the design. This type of optimization is robustification. This is essentially the Improve stage.
• Once the optimized design has been developed, it might be necessary to test the accuracy of the model with a physical model. Depending upon how the model was developed, this step might not be required.
• The final probabilistic step is to use the model and the optimized design to ascertain the system parameters that most affect the quality. With this information one can determine where the      quality control efforts can be most efficiently directed. Thus we have the Control stage.

 

The above approach allows you to most effectively utilize probabilistic methods with DFSS. Hopefully, you are able to use it as a guide so that you can develop high quality, high reliability systems with a short development period.

Applying probabilistic techniques to business strategy to minimise business risk

Probabilistic techniques typically found their practical application in the fields of science and engineering. Certainly, they have demonstrated a great usefulness in engineering and design for quality as well as quality in general. However, given the uncertainty that is inherent in business, it is likely that even greater benefits can be had by applying probabilistic techniques to a business’ strategy. However, I have noticed that this is not often done. Why is this so; does it mean that there actually isn’t much benefit because if there were someone would have done it by now; are people in the business fields really that ignorant of the probabilistic techniques used in technical fields? These questions remind me of the time I first applied advanced probabilistic techniques to a business case.

My first exposure to the application of probability theory to business was during a Master of Entrepreneurship and Innovation (MEI) at the Australian Graduate School of Entrepreneurship (AGSE). I had almost finished my PhD in probabilistic design, and I was interest in applying them to business in some way. For a final semester research assignment I applied Monte Carlo and error propagation to the Clarion case to see what insight such an approach could offer. This case was part of the assessment for the course, and it was familiar to others in the course. Therefore, when the findings of the investigation were presented to them they would be able see which insights came from the probabilistic approach. I would then be able to gauge if they could see the value in this approach or not.

 Few doing the course (or the academic staff of the school) were familiar with the various probabilistic techniques that were applied. Definitely none had heard of robustification. What’s more, they typically didn’t know that it was possible to predict and quantify the risks associated with a business strategy. And the notion of optimising a business strategy against those risks almost seemed like magic to them. However, it became clear to pretty much everyone on present how beneficial probability theory could be to the management of business risk once the case had been presented.

 As alluded to before, one of the key benefits was the ability to predict the probability of outcomes that the traditional approach could only identify as possible problems. Everyone who had done the case realised that there was potential for a problem in the case example, but it was not possible to say whether it was around 1, 10 or 100 percent likely to happen. For the Clarion case a key concern was the exhaustion of the cash resources during an R&D project. With the probabilistic approach, not only was it possible to predict the probability of this event, but it was also possible to find the optimum investment strategy to minimise the probability. This was not possible with regular techniques or intuition, and it was the ability to make this quantification that clearly demonstrated the advantage of the probabilistic approach to everyone else. While others were impressed with the results there was still some concern about the application.

 Our understanding of the risks associated with a business venture can indeed be increased by an order of magnitude by applying probabilistic methods to a model of the business plan; however, the complexity of the model increases by a comparable amount. Therefore, the application might seem daunting. It is worth starting with a typical model (a multi sheet Excel spreadsheet file with P&L, balance sheet, cash flow, asset acquisition and disposal, and other pertinent sheets as dictated by the nature of the business). However, many of the key aspects (sales, operating costs, project timelines etc.) need to be based on outside phenomena. For example, the sales might be predicted with the Bass model, which might be affected by the advertising budget, which might be limited by manufacturing costs, which might be dependent upon an R&D project, which might be dependent upon the availability of key employees that have not yet been employed. In short, you need to determine the important phenomena that affect the business strategy’s success and your model needs to be set up to predict the effects of random fluctuations in the parameters of these phenomena; they parameters will typically be your random inputs. This requires significantly more effort.

 There is one last thing that you must keep in mind when applying probabilistic techniques to a business plan’s strategy. Some argue that there is a difference between risk and uncertainty. Frank Knight (a fairly famous economist in the context of entrepreneurship) argued that risk is that which can be predicted (even if it’s only a probability) and uncertainty is that which remains unknown. The point is that there will always be the possibility of a completely unforseen event that can have a serious affect upon a business (good or bad, but usually bad). Often these events aren’t even thought of until they happen, and this will always put a limit on the certainty that you can expect from a probabilistic analysis. Still, by using a probabilistic approach you can limit much of the risk that will be faced.

 In summary, you need to work harder to make your model of the business plan suitable for a probability analysis. However, the benefits that come from the extra insight and optimisation are considerable. Nevertheless, as considerable as the benefits are, they will always be limited by the fact that we can’t think of everything.

 
Finally, take a look here under ‘New venture financing’ to see a case example showing how you can do this.

Is a model needed for probabilistic analysis and optimization?

Most who have been exposed to the concept of robustness in product development have been introduced to the experimental approaches. However, few are aware of the analytical approach, and many hold the belief that the experimental approach to robustification is the sole approach. Why is this the case and is there any advantage to the analytical approach, which is rarely considered?

I believe that the focus on the experimental approach comes from the dominance of Taguchi and his approach in this field. While Taguchi’s approach has often provided remarkable improvements in quality, it should be noted that it as been heavily criticized by a number of statisticians. The key criticism revolves around the extra experimental effort that the outer array demands. By refocusing this effort upon the parameters in the inner array, it is argued, one can develop an empirical model that can then be combined with various probabilistic methods.

I personally believe that each approach (using the outer array or focusing solely on the inner array) has its advantages, and the engineer needs to consider each case as it is presented. Therefore, I can’t say that one of these approaches is better than the other; however, the notion of developing an empirical model raises something worth considering further: the model itself.

This brings us to the advantages of the analytical approach. If we have a model, we are able to apply probabilistic methods (Monte Carlo, FORM/SORM, error propagation and such). This allows us to predict the effects of randomness and to optimize against the negative effects. Therefore, it is worth developing a model that can be used for optimization.

One of the major problems that I see many have with the development of a model is the implicit assumption that it must be developed empirically. I believe that this is a carry over from the dominance of the Taguchi approach and the assumption that quality is a statistical phenomenon; statisticians seem to have a natural tendency toward the empirical approach. The result is that few people consider developing an analytical model for robustification.

An analytical model has a number of advantages. It improves your understanding of the system being considered, it typically requires less effort/resources to develop, and it can be kept simple for initial considerations, but increased in complexity later on when more accuracy is required. With these advantages, it is clearly worth developing an analytical model for robustification. There are however two common problems: many are reluctant to make a model and the full power of the analytical model is not appreciated.

• Many of us were taught the basics of scientific theory, but few of us were ever taught to use these theories to create a model. This in turn seems to result in a lack of confidence. I can’t tell you too much about developing a model here, but I can tell you that it is more of an art than a science, and it requires practice. Also, you don’t need to worry about getting it wrong; you can always try again, especially given that the development requires relatively little effort. Don’t be surprised if up to 5 attempts are required.

• When you have a model you are no longer restricted to empirical methods, and the faster analytical or numerical optimization methods can be considered. However, I have seen cases where an analytical model was developed and then combined with Monte Carlo and DOE for optimization. Not only would this take longer, but DOE would only offer a limited improvement. This reiterates the current focus on the empirical approach, and the extra consideration you might want to give to the alternative methods optimization methods.

If you are prepared to have a go at creating a model and then combine it with probabilistic methods and various optimization techniques, you can robustify a design at the early stages of development. With the extra insight this offers, you can anticipate greater quality once the product is developed. However, don’t forget that you might get it wrong first time around, and another attempt will be needed. Also, even though an analytical model is ideal, there will still be times when the experimental approach is the only option.

Statistics Vs Probability

To many, probability and statistics are synonymous. While they have much in common, they do have some significant differences. When these differences are fully appreciated, you will develop a perspective that allows you to fully appreciate how you can make the most of a probabilistic approach.

While there are a number of differences, the key difference focuses upon how we acquire systemic information:

• In a statistical approach, which is probably the most common approach, we would use experimentation and the collected data to improve our understanding of the quality we would expect in the system that we are considering. The key problem with this approach is that the actual system is required for experimentation. This means we need to have built a prototype or perhaps have even started production before we can perform these experiments. In the first case, much of the design works needs to have been completed, and in the second case, we would have already finalised the design and stared production. In either case, we would have made a considerable investment before we would know if we were going to be able to meet our quality target.

• In a probabilistic approach, we start with an analytical model and use this for probabilistic analysis. By taking this approach the quality can be predicted, and even optimized, before we commit to a final design or manufacturing. This can save considerable cost and time. In particular, we can determine if a design will even be capable of providing the quality we want, and rejecting it if it can’t.

The advantage of the probabilistic approach over the statistical is clear. However, it does require an analytical model and this can be overwhelming for some. Still, you should always try developing an analytical model and analyzing the model for quality before committing to take a design further. Paradoxically, there will be times when you will need to perform an experiment(s) to develop the model so it should not be thought that the probabilistic approach is a complete replacement of the statistical. This is especially the case when dealing with quality control as opposed to product development.

 In summary, a probabilistic approach allows us to be more predictive than does the statistical. Therefore, the probabilistic approach should be used whenever possible, but it should be noted that the statistical approach is still essential. Also, we need to bring our focus onto the development of an analytical model so that we can apply probabilistic methods.

Choosing the best distribution

One of the key tasks in probabilistic analysis is the selection of the appropriate distribution. While this is of key importance in many cases it is not something that many are sufficiently familiar with. If you have a large data sample, it is fairly easy to use a number of tests, such as the Chi-Square or Anderson Darling, to find the best-fitted distribution to the data. However, there are times when you won’t have data. Still, there are a few principles that can be used to guide you in selecting the most appropriate distribution if you don’t have any data to guide you.

The most significant of these is the central limit theorem. The central limit theorem tells us that if a number of distributions are added to each other, the resultant distribution will be very close to a Normal distribution. In fact, if the relationship between the inputs and the outputs is close to linear over the region of random variability, then the output distribution will also be close to a Normal distribution. This has lead to the Normal distribution being a default assumption for any input distribution. James Siddall, a researcher who put a lot of focus on the selection of input distributions, argued that because of the frequent use of the Normal distribution many think that it is indeed normal to use the Normal distribution. This is not to say that the Normal distribution should not be used or that it should only be used on special occasion, but it is probably used too often.

There is another element of the central limit theorem that is less commonly known. If a number of distributions are multiplied (or divided by each other) then the outcome is close to a Lognormal distribution. The Lognormal distribution can look a lot like a Normal distribution, but it can also be very different. Therefore, given how common multiplicative phenomena are, there are likely to be many cases where a Lognormal distribution is the best choice. This was most evident to me when I met an analyst who frequently dealt with biological processes. In his experience the Lognormal distribution was very much the safest choice.

If you can work out what kind of a process produces the variable you are interested in, you can potentially determine a suitable distribution for that variable. For example, casting operations are essentially multiplicative: the dimension of a cast feature is equal to the respective dimension of the tool, multiplied by the thermal expansion to the tool temperature, multiplied by the thermal contraction as the casting cools. Therefore, we would be safest to assume that the distribution for the dimension of a cast feature is likely to be well representative of a Lognormal distribution. Another example is composite materials. Some composites are made by gluing sheets together in different orientations so that the composite effectively has no grain, and the material’s strength is more uniform. The thickness of such a material is equal to the summation of the thickness of the material that makes it up. Therefore, the distribution of the thickness of the composite is likely to be well represented by a Normal distribution. 

This only covers two processes: multiplicative and additive. However, this covers many real life processes. Therefore, when you are choosing a distribution for an input variable try to think about what kind of an operation produces the respective feature: is it additive or multiplicative? It is not always easy to determine this, but multiplicative is more common. Still, the operation might be something different or there might be more to it. Consider the length of a hypotenuse of a 3D triangle. Three edges are added together after being squared, this will give us a Normal distribution. However, it is then square rooted; therefore, the final distribution will be the square root of a Normal distribution.

A simple sensitivity analysis can prevent a lot of quality issues

A basic sensitivity analysis can save considerable time. The other day I was working with another engineer who was working on a water filtration system that relied on passing household water through a sand filter. The aim was to have the water take as along as possible to move through the filter, to increase the filtration effectiveness, while still ensuring that the time taken to filter the maximum anticipated amount of water would not exceed statutory requirements. The simplest way to achieve this was to tune the flow rate by having it pass through an orifice at the bottom of the filter tank. The design would require no moving parts and the orifice could be moulded into the main body of the filter housing, further reducing the number of parts. This all seemed good. The orifice size was found by using a simple fluids equation that models the time taken to empty a tank through an orifice at the bottom and solver in Excel. As usual the answer in solver had around 14 significant figures so the other engineer took only the first two to specify the design. Just to be on the safe side I put the rounded diameter into the Excel spreadsheet to see what effect the rounding would have. To the surprise of both of us the effects of rounding made the design unsuitable. In fact, we found that simply changing the diameter by an amount representative of moulding tolerances produced an excessive amount of variability in the time taken for the water to empty. The design option was unviable. If we hadn’t performed this check, it might not have been until production that this problem would have been found. This simple example shows two things that I think are worth keeping in mind. First, it is always worth performing some kind of analysis at the start of a design to evaluate the effects of random variability. Second, the analysis does not need to be a full scale Monte Carlo, FORM/SORM or DOE type of analysis. Simply being mindful of the fact the random variability can be a problem and performing some kind of simple analysis during the early stages can save you a lot of quality grief later on. What’s more, it is within the capabilities of most of us so there is really no reason not to.