About Design of Experiments

Learn more about general information on Design of experiments (DOE) techniques and a description of the DOE tools in Adams View:

Design of experiments (also called experimental design) is a collection of procedures and statistical tools for planning experiments and analyzing the results. In general, the experiments may measure the performance of a physical prototype, the yield of a manufacturing process, or the quality of a finished product.

Although DOE techniques were developed around physical experiments, they work just as well with virtual experiments in Adams View. In the case of Adams View, the experiments help you better understand and refine the performance of your mechanical system. DOE techniques can improve your understanding of your design, increase the reliability of your conclusions, and often get you an answer faster than trial-and-error experimentation.

For simple design problems, it is often possible to explore and optimize the behavior of your system using a combination of intuition, trial-and-error, and brute force. As the number of design options increases, however, it becomes more and more difficult to do this quickly and systematically. Varying just one parameter at a time does not tell you a lot about the interactions between parameters. Trying many different parameter combinations can require many simulations, therefore leaving you with a great deal of output data to sift through and understand.

DOE methods provide planning and analysis tools for running a series of experiments. The basic process is to first determine the purpose of the experiments. You might want to identify which variations have the biggest effect on your system, for example. You then choose a set of parameters (called factors) for the system you are investigating and develop a way to measure the appropriate system response. You then select a set of values for each parameter (called levels) and plan a set of experiments (called runs, trials, or treatments) in which you vary the parameter values from one experiment to another. The combination of actual runs to perform is called the design.

An experiment set up in this way is called a designed experiment, or matrix experiment. The runs are described by the design matrix, that has a column for each factor and a row for each run. The matrix entries are the level for each factor for each run. For an example of a design matrix, see Specifying a Design Matrix.

You then execute the runs, recording the performance of the system at each run and analyze the changes in performance across the runs. The type of analysis depends on the purpose of the experiment. Common analyses are analysis of variance (ANOVA) that determines the relative importance of the factors, and linear regression, which fits an assumed mathematical model to the results.

Experiments with two or three factors may only require five or ten runs. As the number of factors and levels grows, however, the number of runs can quickly escalate to dozens, even hundreds. As a result, a good design is critical to the success of the experiment. It should contain as few runs as possible, yet give enough information to accurately depict the behavior of your system. The best design depends on the number of factors and levels, the nature of the factors, assumptions about the behavior of the product or process, and the overall purpose of the experiment.

DOE methods allow you to combine all of these requirements into a efficient, effective design for your problem, and couple it with the appropriate analysis of the results.

Three common uses of Design of experiments (DOE) are:

Screening identifies which factors and combinations of factors most affect the behavior of the system. You consider every factor that may potentially affect the response, and use a screening analysis to determine how much each contributes to the response. This helps you narrow down further experimentation to just the important factors, and also ensures that you do not leave out significant factors. Screening is usually followed by a more in-depth experiment on the most important factors.

Robust design, developed by Dr. Genichi Taguchi, is a methodology for improving quality by controlling the effects of variations in a system. All real-world systems encounter variations due to manufacturing tolerances, material properties, age, wear, or operating conditions. These variations often decrease the quality of the system. Robust design identifies which parameters contribute most to quality variations, and helps you discover how to best minimize their impact on quality. This might mean choosing the least-sensitive configuration from the best-performing combinations, or modifying your system to react less to the variations.

Response surface methods (RSM) fit polynomials to the results of the runs, which gives you an easy-to-use approximation of your system's behavior. The fitted relationships estimate the performance of your system. You may use this model for plotting and evaluating, quick what-if studies, as input for an optimization algorithm, or as a subsystem model in a larger system.

Although screening, robust design, and RSM all use the same basic DOE process as described above, they use different means to generate the designs and analyze the results. Screening, robust design, and RSM are all applicable to Adams models, and you can use Adams as the experimental evaluation for screening and robust design methods.

It may seem that automated Optimization techniques should make Design of experiments (DOE) unnecessary with Adams View. After all, you should be able to automatically perfect your design using Adams View since it includes optimization features. So, why use DOE?

Actually, DOE complements optimization techniques, and is often used in conjunction with optimization. A screening analysis can determine which parameters are good candidates for optimization that improve the reliability and speed of an optimization algorithm. Response surface methods can also create a simplified mathematical model for optimization, that can be evaluated much more quickly and easily than a full simulation or experiment. Even if the simplified model gives only an approximate optimum, it can be used as a good starting point for a full optimization.

More than just helping you find the right answer, however, DOE also helps you explore the relationships between the parameters and your system's performance. A design may combine the optimum parameters, but what are the effects of real-life variations due to manufacturing, wear, or changes in operating conditions? Or perhaps you only need to ensure that the performance stays within a certain tolerance, and you want to know the range of values that will meet that tolerance.

Knowing the optimum point for your system is often important, but may not be the whole story. In many cases, it is just as important to understand what happens in the area surrounding the optimum, and why.

Adams View contains very flexible tools for applying Design of experiments (DOE) techniques to your models. You can build parametric relationships into your model and take advantage of them to run a designed experiment and collect the results. For information on parameterizing your model and running parametric analyses, including DOE, see Preparing for Parametric Analyses.

Adams View also offers simple experiment design and analysis capabilities:

When you run a DOE in Adams View, you may select from several built-in designs. If you select any one of these, Adams View generates the design matrix for you. Adams View generates full-factorial designs.

The full-factorial design uses all of the combinations of levels. The total number of runs will be mn, where m is the number of levels and n is the number of factors. Because this grows very quickly, full factorial is only practical for a small number of factors and levels.

For more sophisticated cases, or other types of DOE methods, MSC's Adams Insight provides a better means for performing DOE. For more information, see your MSC sales representative and the Adams Insight online help, if installed.

You should also consult a good reference guide on the particular method you are using. There are many textbooks on DOE and related methods, such as robust design. Any math library should have references on the statistical aspects of DOE, and engineering libraries have references on applying DOE to quality and design problems.

For information about specifying your own trial matrix or transferring a design matrix from an outside program into Adams View, see Specifying a Design Matrix.

You can specify a design of your own, or a design you generated in an outside program, by directly entering the design matrix or reading it from a file. You can specify the design matrix or file containing the design matrix in the Design Study, DOE, Optimize dialog box or you can run the simulation multi_run doe command.

The design matrix does not directly specify factor values. Instead, it specifies indexes to the levels for each factor. The indexes center on zero. This means that for a two-level factor, the only possible values are -1 and +1; for three-levels, -1, 0 and +1; for four-levels, -2, -1, +1, +2; and so on.

This convention implies that the levels (allowed values or range of values) are ordered from smallest to largest, and cover a range above and below a baseline value. For example, if a factor has three levels, you can think of the -1 index as the low value, the 0 index as the middle or baseline value, and the +1 index as the high value. Note that Adams View does not make any assumptions about the order of the allowed values, therefore, you can use whatever order you find most convenient.

For example, consider an experiment with two factors, each with three levels, and four runs. A design matrix might look like this:

Each row of the matrix represents a run, and each column represents a factor. A -1 indicates the first level for the factor, a 0 the second, and a +1 the third.

If the levels for the first factor are 9, 10, and 11, and the levels for the second factor are 85, 90, and 95, then the matrix would give the following runs:

To specify this matrix using the simulation multi_run doe command, enter the matrix by row: