design of experiment levels

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Experimental Design: Definition and Types

By Jim Frost 3 Comments

What is Experimental Design?

An experimental design is a detailed plan for collecting and using data to identify causal relationships. Through careful planning, the design of experiments allows your data collection efforts to have a reasonable chance of detecting effects and testing hypotheses that answer your research questions.

An experiment is a data collection procedure that occurs in controlled conditions to identify and understand causal relationships between variables. Researchers can use many potential designs. The ultimate choice depends on their research question, resources, goals, and constraints. In some fields of study, researchers refer to experimental design as the design of experiments (DOE). Both terms are synonymous.

Ultimately, the design of experiments helps ensure that your procedures and data will evaluate your research question effectively. Without an experimental design, you might waste your efforts in a process that, for many potential reasons, can’t answer your research question. In short, it helps you trust your results.

Learn more about Independent and Dependent Variables .

Design of Experiments: Goals & Settings

Experiments occur in many settings, ranging from psychology, social sciences, medicine, physics, engineering, and industrial and service sectors. Typically, experimental goals are to discover a previously unknown effect , confirm a known effect, or test a hypothesis.

Effects represent causal relationships between variables. For example, in a medical experiment, does the new medicine cause an improvement in health outcomes? If so, the medicine has a causal effect on the outcome.

An experimental design’s focus depends on the subject area and can include the following goals:

• Understanding the relationships between variables.
• Identifying the variables that have the largest impact on the outcomes.
• Finding the input variable settings that produce an optimal result.

For example, psychologists have conducted experiments to understand how conformity affects decision-making. Sociologists have performed experiments to determine whether ethnicity affects the public reaction to staged bike thefts. These experiments map out the causal relationships between variables, and their primary goal is to understand the role of various factors.

Conversely, in a manufacturing environment, the researchers might use an experimental design to find the factors that most effectively improve their product’s strength, identify the optimal manufacturing settings, and do all that while accounting for various constraints. In short, a manufacturer’s goal is often to use experiments to improve their products cost-effectively.

In a medical experiment, the goal might be to quantify the medicine’s effect and find the optimum dosage.

Developing an Experimental Design

Developing an experimental design involves planning that maximizes the potential to collect data that is both trustworthy and able to detect causal relationships. Specifically, these studies aim to see effects when they exist in the population the researchers are studying, preferentially favor causal effects, isolate each factor’s true effect from potential confounders, and produce conclusions that you can generalize to the real world.

To accomplish these goals, experimental designs carefully manage data validity and reliability , and internal and external experimental validity. When your experiment is valid and reliable, you can expect your procedures and data to produce trustworthy results.

An excellent experimental design involves the following:

• Lots of preplanning.
• Developing experimental treatments.
• Determining how to assign subjects to treatment groups.

The remainder of this article focuses on how experimental designs incorporate these essential items to accomplish their research goals.

Learn more about Data Reliability vs. Validity and Internal and External Experimental Validity .

Preplanning, Defining, and Operationalizing for Design of Experiments

This phase of the design of experiments helps you identify critical variables, know how to measure them while ensuring reliability and validity, and understand the relationships between them. The review can also help you find ways to reduce sources of variability, which increases your ability to detect treatment effects. Notably, the literature review allows you to learn how similar studies designed their experiments and the challenges they faced.

Operationalizing a study involves taking your research question, using the background information you gathered, and formulating an actionable plan.

This process should produce a specific and testable hypothesis using data that you can reasonably collect given the resources available to the experiment.

• Null hypothesis : The jumping exercise intervention does not affect bone density.
• Alternative hypothesis : The jumping exercise intervention affects bone density.

To learn more about this early phase, read Five Steps for Conducting Scientific Studies with Statistical Analyses .

Formulating Treatments in Experimental Designs

In an experimental design, treatments are variables that the researchers control. They are the primary independent variables of interest. Researchers administer the treatment to the subjects or items in the experiment and want to know whether it causes changes in the outcome.

As the name implies, a treatment can be medical in nature, such as a new medicine or vaccine. But it’s a general term that applies to other things such as training programs, manufacturing settings, teaching methods, and types of fertilizers. I helped run an experiment where the treatment was a jumping exercise intervention that we hoped would increase bone density. All these treatment examples are things that potentially influence a measurable outcome.

Even when you know your treatment generally, you must carefully consider the amount. How large of a dose? If you’re comparing three different temperatures in a manufacturing process, how far apart are they? For my bone mineral density study, we had to determine how frequently the exercise sessions would occur and how long each lasted.

How you define the treatments in the design of experiments can affect your findings and the generalizability of your results.

Assigning Subjects to Experimental Groups

A crucial decision for all experimental designs is determining how researchers assign subjects to the experimental conditions—the treatment and control groups. The control group is often, but not always, the lack of a treatment. It serves as a basis for comparison by showing outcomes for subjects who don’t receive a treatment. Learn more about Control Groups .

How your experimental design assigns subjects to the groups affects how confident you can be that the findings represent true causal effects rather than mere correlation caused by confounders. Indeed, the assignment method influences how you control for confounding variables. This is the difference between correlation and causation .

Imagine a study finds that vitamin consumption correlates with better health outcomes. As a researcher, you want to be able to say that vitamin consumption causes the improvements. However, with the wrong experimental design, you might only be able to say there is an association. A confounder, and not the vitamins, might actually cause the health benefits.

Let’s explore some of the ways to assign subjects in design of experiments.

Completely Randomized Designs

A completely randomized experimental design randomly assigns all subjects to the treatment and control groups. You simply take each participant and use a random process to determine their group assignment. You can flip coins, roll a die, or use a computer. Randomized experiments must be prospective studies because they need to be able to control group assignment.

Random assignment in the design of experiments helps ensure that the groups are roughly equivalent at the beginning of the study. This equivalence at the start increases your confidence that any differences you see at the end were caused by the treatments. The randomization tends to equalize confounders between the experimental groups and, thereby, cancels out their effects, leaving only the treatment effects.

For example, in a vitamin study, the researchers can randomly assign participants to either the control or vitamin group. Because the groups are approximately equal when the experiment starts, if the health outcomes are different at the end of the study, the researchers can be confident that the vitamins caused those improvements.

Statisticians consider randomized experimental designs to be the best for identifying causal relationships.

If you can’t randomly assign subjects but want to draw causal conclusions about an intervention, consider using a quasi-experimental design .

Learn more about Randomized Controlled Trials and Random Assignment in Experiments .

Randomized Block Designs

Nuisance factors are variables that can affect the outcome, but they are not the researcher’s primary interest. Unfortunately, they can hide or distort the treatment results. When experimenters know about specific nuisance factors, they can use a randomized block design to minimize their impact.

This experimental design takes subjects with a shared “nuisance” characteristic and groups them into blocks. The participants in each block are then randomly assigned to the experimental groups. This process allows the experiment to control for known nuisance factors.

Blocking in the design of experiments reduces the impact of nuisance factors on experimental error. The analysis assesses the effects of the treatment within each block, which removes the variability between blocks. The result is that blocked experimental designs can reduce the impact of nuisance variables, increasing the ability to detect treatment effects accurately.

Suppose you’re testing various teaching methods. Because grade level likely affects educational outcomes, you might use grade level as a blocking factor. To use a randomized block design for this scenario, divide the participants by grade level and then randomly assign the members of each grade level to the experimental groups.

A standard guideline for an experimental design is to “Block what you can, randomize what you cannot.” Use blocking for a few primary nuisance factors. Then use random assignment to distribute the unblocked nuisance factors equally between the experimental conditions.

You can also use covariates to control nuisance factors. Learn about Covariates: Definition and Uses .

Observational Studies

In some experimental designs, randomly assigning subjects to the experimental conditions is impossible or unethical. The researchers simply can’t assign participants to the experimental groups. However, they can observe them in their natural groupings, measure the essential variables, and look for correlations. These observational studies are also known as quasi-experimental designs. Retrospective studies must be observational in nature because they look back at past events.

Imagine you’re studying the effects of depression on an activity. Clearly, you can’t randomly assign participants to the depression and control groups. But you can observe participants with and without depression and see how their task performance differs.

Observational studies let you perform research when you can’t control the treatment. However, quasi-experimental designs increase the problem of confounding variables. For this design of experiments, correlation does not necessarily imply causation. While special procedures can help control confounders in an observational study, you’re ultimately less confident that the results represent causal findings.

Learn more about Observational Studies .

For a good comparison, learn about the differences and tradeoffs between Observational Studies and Randomized Experiments .

Between-Subjects vs. Within-Subjects Experimental Designs

When you think of the design of experiments, you probably picture a treatment and control group. Researchers assign participants to only one of these groups, so each group contains entirely different subjects than the other groups. Analysts compare the groups at the end of the experiment. Statisticians refer to this method as a between-subjects, or independent measures, experimental design.

In a between-subjects design , you can have more than one treatment group, but each subject is exposed to only one condition, the control group or one of the treatment groups.

A potential downside to this approach is that differences between groups at the beginning can affect the results at the end. As you’ve read earlier, random assignment can reduce those differences, but it is imperfect. There will always be some variability between the groups.

In a  within-subjects experimental design , also known as repeated measures, subjects experience all treatment conditions and are measured for each. Each subject acts as their own control, which reduces variability and increases the statistical power to detect effects.

In this experimental design, you minimize pre-existing differences between the experimental conditions because they all contain the same subjects. However, the order of treatments can affect the results. Beware of practice and fatigue effects. Learn more about Repeated Measures Designs .

Design of Experiments Examples

For example, a bone density study has three experimental groups—a control group, a stretching exercise group, and a jumping exercise group.

In a between-subjects experimental design, scientists randomly assign each participant to one of the three groups.

In a within-subjects design, all subjects experience the three conditions sequentially while the researchers measure bone density repeatedly. The procedure can switch the order of treatments for the participants to help reduce order effects.

Matched Pairs Experimental Design

A matched pairs experimental design is a between-subjects study that uses pairs of similar subjects. Researchers use this approach to reduce pre-existing differences between experimental groups. It’s yet another design of experiments method for reducing sources of variability.

Researchers identify variables likely to affect the outcome, such as demographics. When they pick a subject with a set of characteristics, they try to locate another participant with similar attributes to create a matched pair. Scientists randomly assign one member of a pair to the treatment group and the other to the control group.

On the plus side, this process creates two similar groups, and it doesn’t create treatment order effects. While matched pairs do not produce the perfectly matched groups of a within-subjects design (which uses the same subjects in all conditions), it aims to reduce variability between groups relative to a between-subjects study.

On the downside, finding matched pairs is very time-consuming. Additionally, if one member of a matched pair drops out, the other subject must leave the study too.

Learn more about Matched Pairs Design: Uses & Examples .

Another consideration is whether you’ll use a cross-sectional design (one point in time) or use a longitudinal study to track changes over time .

A case study is a research method that often serves as a precursor to a more rigorous experimental design by identifying research questions, variables, and hypotheses to test. Learn more about What is a Case Study? Definition & Examples .

In conclusion, the design of experiments is extremely sensitive to subject area concerns and the time and resources available to the researchers. Developing a suitable experimental design requires balancing a multitude of considerations. A successful design is necessary to obtain trustworthy answers to your research question and to have a reasonable chance of detecting treatment effects when they exist.

Share this:

Reader Interactions

March 23, 2024 at 2:35 pm

Dear Jim You wrote a superb document, I will use it in my Buistatistics course, along with your three books. Thank you very much! Miguel

March 23, 2024 at 5:43 pm

Thanks so much, Miguel! Glad this post was helpful and I trust the books will be as well.

April 10, 2023 at 4:36 am

What are the purpose and uses of experimental research design?

Comments and Questions Cancel reply

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Guide to Experimental Design | Overview, Steps, & Examples

Guide to Experimental Design | Overview, 5 steps & Examples

Published on December 3, 2019 by Rebecca Bevans . Revised on June 21, 2023.

Experiments are used to study causal relationships . You manipulate one or more independent variables and measure their effect on one or more dependent variables.

Experimental design create a set of procedures to systematically test a hypothesis . A good experimental design requires a strong understanding of the system you are studying.

There are five key steps in designing an experiment:

• Consider your variables and how they are related
• Write a specific, testable hypothesis
• Design experimental treatments to manipulate your independent variable
• Assign subjects to groups, either between-subjects or within-subjects
• Plan how you will measure your dependent variable

For valid conclusions, you also need to select a representative sample and control any  extraneous variables that might influence your results. If random assignment of participants to control and treatment groups is impossible, unethical, or highly difficult, consider an observational study instead. This minimizes several types of research bias, particularly sampling bias , survivorship bias , and attrition bias as time passes.

Table of contents

Step 1: define your variables, step 2: write your hypothesis, step 3: design your experimental treatments, step 4: assign your subjects to treatment groups, step 5: measure your dependent variable, other interesting articles, frequently asked questions about experiments.

You should begin with a specific research question . We will work with two research question examples, one from health sciences and one from ecology:

To translate your research question into an experimental hypothesis, you need to define the main variables and make predictions about how they are related.

Start by simply listing the independent and dependent variables .

Then you need to think about possible extraneous and confounding variables and consider how you might control  them in your experiment.

Finally, you can put these variables together into a diagram. Use arrows to show the possible relationships between variables and include signs to show the expected direction of the relationships.

Here we predict that increasing temperature will increase soil respiration and decrease soil moisture, while decreasing soil moisture will lead to decreased soil respiration.

Prevent plagiarism. Run a free check.

Now that you have a strong conceptual understanding of the system you are studying, you should be able to write a specific, testable hypothesis that addresses your research question.

The next steps will describe how to design a controlled experiment . In a controlled experiment, you must be able to:

• Systematically and precisely manipulate the independent variable(s).
• Precisely measure the dependent variable(s).
• Control any potential confounding variables.

If your study system doesn’t match these criteria, there are other types of research you can use to answer your research question.

How you manipulate the independent variable can affect the experiment’s external validity – that is, the extent to which the results can be generalized and applied to the broader world.

First, you may need to decide how widely to vary your independent variable.

• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.

Second, you may need to choose how finely to vary your independent variable. Sometimes this choice is made for you by your experimental system, but often you will need to decide, and this will affect how much you can infer from your results.

• a categorical variable : either as binary (yes/no) or as levels of a factor (no phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).

How you apply your experimental treatments to your test subjects is crucial for obtaining valid and reliable results.

First, you need to consider the study size : how many individuals will be included in the experiment? In general, the more subjects you include, the greater your experiment’s statistical power , which determines how much confidence you can have in your results.

Then you need to randomly assign your subjects to treatment groups . Each group receives a different level of the treatment (e.g. no phone use, low phone use, high phone use).

You should also include a control group , which receives no treatment. The control group tells us what would have happened to your test subjects without any experimental intervention.

When assigning your subjects to groups, there are two main choices you need to make:

• A completely randomized design vs a randomized block design .
• A between-subjects design vs a within-subjects design .

Randomization

An experiment can be completely randomized or randomized within blocks (aka strata):

• In a completely randomized design , every subject is assigned to a treatment group at random.
• In a randomized block design (aka stratified random design), subjects are first grouped according to a characteristic they share, and then randomly assigned to treatments within those groups.

Sometimes randomization isn’t practical or ethical , so researchers create partially-random or even non-random designs. An experimental design where treatments aren’t randomly assigned is called a quasi-experimental design .

Between-subjects vs. within-subjects

In a between-subjects design (also known as an independent measures design or classic ANOVA design), individuals receive only one of the possible levels of an experimental treatment.

In medical or social research, you might also use matched pairs within your between-subjects design to make sure that each treatment group contains the same variety of test subjects in the same proportions.

In a within-subjects design (also known as a repeated measures design), every individual receives each of the experimental treatments consecutively, and their responses to each treatment are measured.

Within-subjects or repeated measures can also refer to an experimental design where an effect emerges over time, and individual responses are measured over time in order to measure this effect as it emerges.

Counterbalancing (randomizing or reversing the order of treatments among subjects) is often used in within-subjects designs to ensure that the order of treatment application doesn’t influence the results of the experiment.

Finally, you need to decide how you’ll collect data on your dependent variable outcomes. You should aim for reliable and valid measurements that minimize research bias or error.

Some variables, like temperature, can be objectively measured with scientific instruments. Others may need to be operationalized to turn them into measurable observations.

• Ask participants to record what time they go to sleep and get up each day.
• Ask participants to wear a sleep tracker.

How precisely you measure your dependent variable also affects the kinds of statistical analysis you can use on your data.

Experiments are always context-dependent, and a good experimental design will take into account all of the unique considerations of your study system to produce information that is both valid and relevant to your research question.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word “between” means that you’re comparing different conditions between groups, while the word “within” means you’re comparing different conditions within the same group.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 21). Guide to Experimental Design | Overview, 5 steps & Examples. Scribbr. Retrieved August 12, 2024, from https://www.scribbr.com/methodology/experimental-design/

Is this article helpful?

Rebecca Bevans

Other students also liked, random assignment in experiments | introduction & examples, quasi-experimental design | definition, types & examples, how to write a lab report, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Design of Experiments – A Primer

Published: November 5, 2010 by K. Sundararajan

Design of experiments (DOE) is a systematic method to determine the relationship between factors affecting a process and the output of that process. In other words, it is used to find cause-and-effect relationships. This information is needed to manage process inputs in order to optimize the output.

An understanding of DOE first requires knowledge of some statistical tools and experimentation concepts. Although a DOE can be analyzed in many software programs, it is important for practitioners to understand basic DOE concepts for proper application.

Common DOE Terms and Concepts

The most commonly used terms in the DOE methodology include: controllable and uncontrollable input factors, responses, hypothesis testing, blocking, replication and interaction.

• Controllable input factors , or x factors, are those input parameters that can be modified in an experiment or process. For example, in cooking rice, these factors include the quantity and quality of the rice and the quantity of water used for boiling.
• Uncontrollable input factors are those parameters that cannot be changed. In the rice-cooking example, this may be the temperature in the kitchen. These factors need to be recognized to understand how they may affect the response.
• Responses , or output measures, are the elements of the process outcome that gage the desired effect. In the cooking example, the taste and texture of the rice are the responses.

The controllable input factors can be modified to optimize the output. The relationship between the factors and responses is shown in Figure 1.

• Hypothesis testing helps determine the significant factors using statistical methods. There are two possibilities in a hypothesis statement: the null and the alternative. The null hypothesis is valid if the status quo is true. The alternative hypothesis is true if the status quo is not valid. Testing is done at a level of significance, which is based on a probability.
• Blocking and replication : Blocking is an experimental technique to avoid any unwanted variations in the input or experimental process. For example, an experiment may be conducted with the same equipment to avoid any equipment variations. Practitioners also replicate experiments, performing the same combination run more than once, in order to get an estimate for the amount of random error that could be part of the process.
• Interaction: When an experiment has three or more variables, an interaction is a situation in which the simultaneous influence of two variables on a third is not additive.

A Simple One-factor Experiment

The comparison of two or more levels in a factor can be done using an F-test. This compares the variance of the means of different factor levels with the individual variances, using this equation:

F = ns 2 Y-bar / s 2 pooled

where: n = the sample size s 2 Y-bar = the variance of the means, which is calculated by dividing the sum of variances of the individual means by the degrees of freedom s 2 pooled = pooled variance, or the average of the individual variances

This is similar to the signal-to-noise ratio used in electronics. If the value of F (the test statistic) is greater than the F-critical value, it means there is a significant difference between the levels, or one level is giving a response that is different from the others. Caution is also needed to ensure that s 2 pooled is kept to a minimum, as it is the noise or error term. If the F value is high, the probability ( p -value) will fall below 0.05, indicating that there is a significant difference between levels. The value of 0.05 is a typical accepted risk value.

If F = 1, it means the factor has no effect.

As an example of a one-factor experiment, data from an incoming shipment of a product is given in Table 1.

Table 1: Incoming Shipment Data

When a practitioner completes an analysis of variance (ANOVA), the following results are obtained:

Table 2: ANOVA Summary

Statistical software can provide hypothesis testing and give the actual value of F. If the value is below the critical F value, a value based on the accepted risk, then the null hypothesis is not rejected. Otherwise, the null hypothesis is rejected to confirm that there is a relationship between the factor and the response. Table 2 shows that the F is high, so there is a significant variation in the data. The practitioner can conclude that there is a difference in the lot means.

Two-level Factorial Design

This is the most important design for experimentation. It is used in most experiments because it is simple, versatile and can be used for many factors. In this design, the factors are varied at two levels – low and high.

Two-level designs have many advantages. Two are:

• The size of the experiment is much smaller than other designs.
• The interactions of the factors can be detected.

For an example of a two-level factorial design, consider the cake-baking process. Three factors are studied: the brand of flour, the temperature of baking and the baking time. The associated lows and highs of these factors are listed in Table 3.

Table 3: Cake-baking Factors and Their Associated Levels

The output responses considered are “taste” and “crust formation.” Taste was determined by a panel of experts, who rated the cake on a scale of 1 (worst) to 10 (best). The ratings were averaged and multiplied by 10. Crust formation is measured by the weight of the crust, the lower the better.

The experiment design, with the responses, is shown in Table 4.

Table 4: Settings of Input Factors and the Resulting Responses

Analysis of the results is shown in Table 5. Figures 2 through 4 show the average taste scores for each factor as it changes from low to high levels. Figures 5 through 7 are interaction plots; they show the effect of the combined manipulation of the factors.

Table 5: ANOVA Table for the Taste Response

From reading an F table, the critical F value at 1 percent is 16.47. As the actual value of F for time and temperature exceed this value (time is at 34.306 and temperature is 23.592), it’s possible to conclude that both of them have a significant effect on the taste of the product. This is also evident from Figures 3 and 4, where the line is steep for the variation of these two factors. Figure 7 also shows that when the temperature is high, the taste sharply decreases with time (as charring takes place).

For the crust formation, the data analysis is shown in Table 6.

Table 6: ANOVA Table for the Crust Response

In this case the actual F value for the three factors (brand, time and temperature) are below the critical F value for 1 percent (16.47). This shows that these are not significant factors for the crust formation in the cake. If further optimization of the crust formation is needed, then other factors, such as the quantity of ingredients in the cake (eggs, sugar and so on), should be checked.

Versatile Tool for Practitioners

Design of experiments is a powerful tool in Six Sigma to manage the significant input factors in order to optimize the desired output. Factorial experiments are versatile because many factors can be modified and studied at once. The following resources can be helpful in learning more about DOEs:

• DOE Simplified Practical Tools for Effective Experimentation (Productivity Inc., 2000)
• Design and Analysis of Experiments (John Wiley and Sons, 1997)

About the Author

K. Sundararajan

Pursue the Next You in 2024 with 20% Tuition Reduction on September Courses!

Design of Experiments

Last Updated October 19, 2023

Optimal performance of your process is locked inside current performance, just waiting to be discovered. The optimal process can emerge once all the variables are adjusted appropriately. The Design of Experiments (DOE) tool helps align process variables and arrange them to ensure optimal performance.

What is Design of Experiments?

Design of Experiments (DOE) is a Six Sigma tool that helps project teams determine the effects that the inputs of a process have on the final product. DOE helps uncover the critical relationships between variables in a process that are often hidden under all of the data and identifies the most critical inputs that must be modified to ensure optimal process performance. Once Design of Experiments identifies the critical inputs of the process, it helps project teams understand the impact that modifying the variables will have on process performance.

Design of Experiments Terminology

Six Sigma Design of Experiments is a systematic process that breaks down the variables of production and analyzes each one. This process has its own set of terms that we must understand to become conversant with how the technique works.

• Factor – This is an independent variable, or a variable you have control over. In DOE, factors are deliberately modified to determine the point of optimal performance.
• Level – This is a measurement of how much a factor has been modified. Levels can be discrete or numeric.
• Run – An experiment typically done at two or three levels for every factor; each separate level constitutes an experimental run.
• Response – The outcome of the run.
• Replication – Refers to multiple sets of experimental runs. Replication provides even more data and greater confidence in evaluating the results.

How to Apply Design of Experiments

Design of Experiments terminology is more clearly understood when applied to a practical example. Suppose a project team used DOE to optimize the process for baking a cake.

• Factors  – The factors in the process, the variables that the team controls, consist of the ingredients of sugar, flower, eggs, water and oil. The oven is also a factor. These are inputs into the process.
• Levels  – The levels in the cake baking process are the temperature of the oven, the cooking time and the amount of each ingredient used. These are the potential settings of each factor.
• Response  – The cake is the output or the response of the run. The characteristics of the output, cake, are then evaluated  to determine if the levels in this particular run lead to optimal performance, or a cake that ranks well in taste, color, and consistency.

Design of Experiments provides many ways to bake a better cake:

• Comparing alternatives – The team can test the results of using two different types of the same ingredient by keeping other factors the same.
• Reducing variability – Determining how the levels can be changed so that the cake is always the same quality.
• Targeting an output – Deciding what changes to make in the ingredients and amounts of ingredients used that make the cake just right.
• Evaluating tradeoffs – Discovering how to produce the best response (cake) possible by using the simplest factors (the smallest amount of ingredients).

Selecting the Factors

Many inputs determine the output of a process. The factors that are most relevant to the end result are the ones most important to DOE. These factors can be identified by the project team in a brainstorming session. In ordinary circumstances, where time and budget are finite, the team should limit the experiment to six or seven key factors. These factors are controlled by setting them at different levels for each run.

Setting the Levels

Once the factors have been identified, the team must determine the settings at which these factors will be run for the experiment. The example of baking a cake demonstrates that some factors are measured in numbers, such as oven temperature and cooking time. Some factors are also qualitative, such as how much icing to use. These are measured in categories and are converted into coded units for linear regression analysis.

The more levels that are identified for each factor, the more trials will be required to test these levels. To ensure that an optimal number of levels are selected, focus on a range of interest. This range includes settings used in the normal course of operations and may also include settings of more extreme scenarios. The greater the difference in factor levels, the easier it becomes to measure variance.

Evaluating the Response

The response is the outcome of the experiment. Outcomes are helpful in improving the process when they can be measured in quantitative terms, rather than in qualitative attributes. A response that is quantifiable makes the experiment well suited to the additional scrutiny of statistical regression techniques.

Design of experiments allows inputs to be changed to determine how they affect responses. Instead of testing one factor at a time while holding others constant, DOE reveals how interconnected factors respond over a wide range of values, without requiring the testing of all possible values directly. This helps reveal secrets hidden behind the different factors and levels in a process and allows the project team to understand the process much more rapidly.

Once completed, Design of Experiments helps the Six Sigma project team better identify the combination of inputs that lead to the highest-quality product or service.

Related Articles

Take the next step in your career with a program guide!

By completing this form and clicking the button below, I consent to receiving calls, text messages and/or emails from BISK, its client institutions, and their representatives regarding educational services and programs. I understand calls and texts may be directed to the number I provide using automatic dialing technology. I understand that this consent is not required to purchase goods or services. If you would like more information relating to how we may use your data, please review our privacy policy .

Design of Experiments (DOE)

• First Online: 07 November 2017

Cite this chapter

• Miryam Barad 3  

Part of the book series: SpringerBriefs in Applied Sciences and Technology ((BRIEFSAPPLSCIENCES))

1095 Accesses

1 Citations

This chapter comprises five sections. The first section is an introduction to DOE containing guidelines and some basic concepts and principles. The next three sections present examples published some years ago in diverse journals. Each example represents a different application category: a simulation experiment , a physical experiment , and a complex analytical expression . The scene of the simulation experiment was set in a flexible manufacturing environment and its aim was to investigate the impact of several factors in this environment. The physical experiment has been carried out to improve the quality of a special type of batteries. The third example examined the numerical value of a deterministic complex analytical expression representing a customer oriented logistics performance measure, as calculated for different values of its parameters (the given numerical values of the investigated factors). A review of these three examples has been recently published in Applied Mathematics, see Barad 2014 . The fifth section presents a discussion of Taguchi’s contribution to experimental design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Barad M (1992) The impact of some flexibility factors in fmss—a performance evaluation approach. Int J Prod Res 30:2587–2602

Article   Google Scholar  

Barad M (2014) Design of experiments (DOE)—a valuable multi-purpose methodology. Appl Math 5:2120–2129

Google Scholar  

Barad M, Even-Sapir D (2003) Flexibility in logistics systems—modeling and performance evaluation. Int J Prod Econ 85:155–170

Barad M, Bezalel C, Goldstein JR (1989) Prior research is the key to fractional factorial design. Qual Prog 22:71–75

Box GE, Hunter WG, Hunter JS (2005) Statistics for experimenters, 2nd edn. Wiley, New York

Dehnad K (ed) (1989) Quality control. Robust design and the Taguchi method. Wadsworth & Brooks/Cole, California

MATH   Google Scholar  

Law AM, Kelton WD (2000) Simulation modeling and analysis, 3rd edn. McGraw Hill, New York

Montgomery DC (2012) Design and analysis of experiments, 5th edn. Wiley, New York

Ross PJ (1996) Taguchi techniques for quality engineering, 2nd edn. McGraw Hill, New York

Taguchi G (1986) Introduction to quality engineering. Asian Productivity Organization, Tokyo

Download references

Author information

Authors and affiliations.

Department of Industrial Engineering, The Iby and Aladar Fleishman Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel

Miryam Barad

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2018 The Author(s)

About this chapter

Barad, M. (2018). Design of Experiments (DOE). In: Strategies and Techniques for Quality and Flexibility. SpringerBriefs in Applied Sciences and Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-68400-0_4

Download citation

DOI : https://doi.org/10.1007/978-3-319-68400-0_4

Published : 07 November 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-68399-7

Online ISBN : 978-3-319-68400-0

eBook Packages : Engineering Engineering (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Lean Six Sigma Training Certification

• Facebook Instagram Twitter LinkedIn YouTube
• (877) 497-4462

Step-by-Step Guide to DoE (Design of Experiments)

February 16th, 2017

DOE or Design of experiments helps identify the various factors that affect the productivity and the outcomes of a particular process or a design. The individual influence of the factors as well as the interactive power of these factors to influence the outcome comes to light through an efficient design of experiment. The trial and error approach of the past to consequently achieve the desired productivity and efficiency is obsolete. The sophisticated statistical approach taken by DOE makes it convenient for the businesses to design, conduct and analyze the experiments that can help multiply the output. Here is a systematic, step-by-step guide to design a fruitful experiment.

Set Objectives Clearly defined goals and objectives of the experiment are important to get the intended answer. A comprehensive brain storming session or an interactive meeting can help the team prioritize the goals. The type of design of the experiment depends heavily on your objectives. • Comparative Design: It lets you compare between two or more factors or effects to find out the one with the greatest impact. • Screening Design: It is vital when you are dealing with many factors and want to filter out a few important ones. • Response Surface Modeling: Typically employed when you want to maximize or minimize a response. • Regression Modeling: It is used to help figure out the degrees of dependence of a response on the factors.

Choose Your Variables The next step is to shortlist your variables. Choose your input i.e. factors and your output i.e. responses carefully, as this will define the efficacy and usability of your experiment. Setting the constraints or the range of the factors is vital. Two-level designs that involve a high and a low level for the factors seem to be the most efficient one, with +1 and -1 notations respectively.

Consider the Interactions The greatest advantage of Design of Experiments over traditional experiments is its allowance of analyzing the synergized impacts of the various factors on the responses. When many factors are in play together, finding out the combinations of factors that manage to inflict the most affect is crucial. The team needs to carefully prioritize the interactions they want to test. If you are using DOE software, it is best to run the experiment for all the possible interactions of factors.

Run the Experiment Once you have decided upon the type of experiment and the most important input and output, it is time to simply run the experiment. Ensuring all the relevant data is accurate and in process, is vital to your results. Before running the experiment, go over the design one more time. The team should come up with the minimum number of times to run the experiment to get any significant result. Run all the experiments with the same set of assumptions as well as factors and responses.

Analyze the Results After the necessary runs of your experiment have been carried out, the next obvious step is the analysis of the data obtained because of the experiment. Graphs and diagrams can help you greatly assess the data. Histograms, flowcharts as well as scatter diagrams can give an insight on the effects of various factors on different responses. Try to find correlations between input and output, the interactive impacts of the many factors as well as the magnitude of affects on the responses. Simple and step-by-step approach to design of experiments efficiently lets you test out the different ways in to improve a particular process. The results and findings of an experiment allow you to make the necessary tweaks and adjustments in a system to improve the yield.

Stop by 6Sigma.us and learn how we can make a difference in your business and bring clarity to your current challenges.

SixSigma.us offers both Live Virtual classes as well as Online Self-Paced training. Most option includes access to the same great Master Black Belt instructors that teach our World Class in-person sessions. Sign-up today!

Virtual Classroom Training Programs Self-Paced Online Training Programs

SixSigma.us Accreditation & Affiliations

Monthly Management Tips

• Be the first one to receive the latest updates and information from 6Sigma
• Get curated resources from industry-experts
• Gain an edge with complete guides and other exclusive materials
• Become a part of one of the largest Six Sigma community
• Unlock your path to become a Six Sigma professional

" * " indicates required fields

experimental design     .online  

Factors and Levels

Factorial design settings, response surface design settings, factor analysis, regresson model, optimization, quick guide.

Enter your experimental design variable names (factors) and values (levels). Examples of factors are ‘color’, ‘size’, ‘shape’, etc. Examples of levels are ‘red’ and ‘blue’ for ‘color’; 10, 20, 30 for ‘size’; ‘square’ and ‘round’ for 'shape'.

Select the number of measured responses and DoE types. Enter your responses into the DoE table. Use the ‘Settings’ button to set up the particular design.

Study the effect size for each factor and response breakdown.

Select the regression model and create it by pressing the ‘Create the Model’ button. Study the statistics.

Use the buttons to find factor levels for minimum and maximum possible responses. Use sliders to fine-tune the response values.

General Partner

Partnership

• Quality Improvement
• Talk To Minitab

8 Expert Tips for Excellent Designed Experiments (DOE)

Topics: Design of Experiments - DOE , Lean Six Sigma , Six Sigma , Statistics , Quality Improvement

If your work involves quality improvement, you've at least heard of Design of Experiments (DOE). You probably know it's the most efficient way to optimize and improve your process. But many of us find DOE intimidating, especially if it's not a tool we use often. How do you select an appropriate design, and ensure you've got the right number of factors and levels? And after you've gathered your data, how do you pick the right model for your analysis?

If you're comfortable enough to skip the Assistant, but still have some questions about whether you're approaching your DOE the right way, consider the following tips from Minitab's technical trainers. These veterans have done a host of designed experiments, both while working with Minitab customers and in their careers in before they became Minitab trainers. 

1. Identify the right variable space to study with exploratory runs.

Performing exploratory runs before doing the main experiment can help you identify the settings of your process as performance moves from good to bad. This can help you determine the variable space to conduct your experiment that will yield the most beneficial results.                                             

2. Spread control runs throughout the experiment to measure process stability.

3. identify the biggest problems with pareto analysis..

4. Improve power by expanding the range of input settings.

5. fractionate to save runs, focusing on resolution v designs..

In many cases, it's beneficial to choose a design with ½ or ¼ of the runs of a full factorial . Even though effects could be confounded or confused with each other, Resolution V designs minimize the impact of this confounding which allows you to estimate all main effects and two-way interactions. Conducting fewer runs can save money and keep experiment costs low.

6. Improve the power of your experiment with replicates.

Power is the probability of detecting an effect on the response, if that effect exists. The number of replicates affects your experiment's power. To increase the chance that you will be successful identifying the inputs that affect your response, add replicates to your experiment to increase its power.

7. Improve power by using quantitative measures for your response.

Reducing defects is the primary goal of most experiments, so it makes sense that defect counts are often used as a response. But defect counts are a very expensive and unresponsive output to measure. Instead, try measuring a quantitative indicator related to your defect level. Doing this can decrease your sample size dramatically and improve the power of your experiment.  

8. Study all variables of interest and all key responses.

Factorial designs let you take a comprehensive approach to studying all potential input variables. Removing a factor from the experiment slashes your chance of determining its importance to zero. With the tools available in statistical software such as Minitab to help, you shouldn't let fear of complexity cause you to omit potentially important input variables.

You Might Also Like

• Trust Center

© 2023 Minitab, LLC. All Rights Reserved.

• Terms of Use
• Privacy Policy
• Cookies Settings

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

Lesson 1: introduction to design of experiments, overview section  .

In this course we will pretty much cover the textbook - all of the concepts and designs included. I think we will have plenty of examples to look at and experience to draw from.

Please note: the main topics listed in the syllabus follow the chapters in the book.

A word of advice regarding the analyses. The prerequisite for this course is STAT 501 - Regression Methods and STAT 502 - Analysis of Variance . However, the focus of the course is on the design and not on the analysis. Thus, one can successfully complete this course without these prerequisites, with just STAT 500 - Applied Statistics for instance, but it will require much more work, and for the analysis less appreciation of the subtleties involved. You might say it is more conceptual than it is math oriented.

  Text Reference: Montgomery, D. C. (2019). Design and Analysis of Experiments , 10th Edition, John Wiley & Sons. ISBN 978-1-119-59340-9

What is the Scientific Method? Section  

Do you remember learning about this back in high school or junior high even? What were those steps again?

Decide what phenomenon you wish to investigate. Specify how you can manipulate the factor and hold all other conditions fixed, to insure that these extraneous conditions aren't influencing the response you plan to measure.

Then measure your chosen response variable at several (at least two) settings of the factor under study. If changing the factor causes the phenomenon to change, then you conclude that there is indeed a cause-and-effect relationship at work.

How many factors are involved when you do an experiment? Some say two - perhaps this is a comparative experiment? Perhaps there is a treatment group and a control group? If you have a treatment group and a control group then, in this case, you probably only have one factor with two levels.

How many of you have baked a cake? What are the factors involved to ensure a successful cake? Factors might include preheating the oven, baking time, ingredients, amount of moisture, baking temperature, etc.-- what else? You probably follow a recipe so there are many additional factors that control the ingredients - i.e., a mixture. In other words, someone did the experiment in advance! What parts of the recipe did they vary to make the recipe a success? Probably many factors, temperature and moisture, various ratios of ingredients, and presence or absence of many additives.  Now, should one keep all the factors involved in the experiment at a constant level and just vary one to see what would happen?  This is a strategy that works but is not very efficient.  This is one of the concepts that we will address in this course.

• understand the issues and principles of Design of Experiments (DOE),
• understand experimentation is a process,
• list the guidelines for designing experiments, and
• recognize the key historical figures in DOE.

Design of Experiments (DOE)

Design of Experiments (DOE) is a study of the factors that the team has determined are the key process input variables (KPIV's) that are the source of the variation or have an influence on the mean of the output.

DOE are used by marketers, continuous improvement leaders, human resources, sales managers, engineers, and many others. When applied to a product or process the result can be increased quality performance, higher yields, lower variation of outputs, quicker development time, lower costs, and increased customer satisfaction .

It is a sequence of tests where the input factors are systematically changed according to a design matrix.  The DOE study is first started by setting up an experiment with a specific number if runs with one of more factors (inputs) with each given two or more levels or settings. 

The DOE process has a significant advantage above trial and error methods. Yes, it may end up taking more time and resources but the result will most likely be more robust.

This initial time and effort up front can be costly (this is up to the team to decide how many experiments to conduct) and time consuming but the end result will be the maximizing the outputs shown above in bold. 

A DOE (or set of DOE's) will help develop a prediction equation for the process in terms of Y = f(X 1 ,X 2 ,X 3 ,X 4 ,....X n ). GOAL:

1) Understand the influential variables and understand any interactions

2) Quantify the effect of the variables on the outputs

3) Determine the setting that optimize your response (which could be to minimize or maximize an value for an output "y")

Designing a DOE

The number of runs (treatments) depends on amount of resources that can be afforded...such as time and money and keep in mind, replications are ideal to help validate the results and help detect any fluke results. 

The power of efficiency in a DOE is within hidden replication. However, the may be instances with a block design is incomplete when it isn't possible to apply all treatments in every block. 

More treatments takes more time and money but offers the most information. It's a trade-off between the amount of Type I and II error you can afford to risk along with time and money. 

The more factors and levels you have, the more combinations are possible and thus adding more time and money to the project, unless you choose to take more risk and reduce the runs by using a fractional design. Of course, being able to adjust setting of more variables and playing more with factors is great, but it comes with a price. 

The guide below shows that the amount of 'levels' to the power of the number of 'factors' is the number of combinations of treatments for a full factorial design. 

For example, with two factors (inputs) each taking two levels, a factorial DOE will have four combinations. With two levels and two factors the DOE is termed a 2×2 factorial design. A memory tactic.... Levels lie low and Factors fly high A DOE with 3 levels and 4 factors is a 3×4 factorial design with 81 (3 4 = 81) treatment combinati ons. It may not be practical or feasible to run a full factorial (all 81 combinations) so a fractional factorial design is done, where usually half of the combinations are omitted.

STEPS to conduct a DOE:

• Define the objective for the DOE
• Select the process variables (independent and dependent)
• Determine DOE design - which d epends on resources (time & money) and amount of Type I and II error you're willing to accept
• Execute the design (randomization where possible)
• Verify results (r eplicate the tests if practical to help verify results)
• Interpret the results

The next step is to implement IMPROVEMENTS (that’s the goal….implementing improvements that matter)

Here are some characteristics of factorial experiments in general:

• A Response is the output and is the dependent variable
• Response = sum of process mean + var iation about the mean
• Factors are independent variables
• Variation about the mean is sum of factors + interactions + unexplained residuals (or experimental error)

ANOVA is used to decompose the variation of the response to show the effect from each factor, interactions, and experimental error (or unexplained residual). Statistical software will help manage the entire DOE.

• Enter the factors
• Set the levels (at least two for each factor)
• Determine how many runs (full factorial, fractional factorial)
• Run the experiment at each treatment level
• Enter the response for each treatment level
• Use statistical software to use ANOVA on the data
• Continue to refine until prediction equation is obtained
• IMPROVE the KPIV's
• Last phase is CONTROL the KPIV's

Other methods of experimentation such as "trial and error" or "one factor at a time (OFAT)" are prone to waste, will provide less information and will not provide a prediction equation. These may seem easier to run and get results but the risk is a less robust solution and decisions made on a poor experiment. These input factors behave to create an output, the team needs to make improvements in the IMPROVE phase that control the inputs. Controlling the input factors will provide the desired response. The DOE will quantify the factor interactions and offer a prediction equation. ANOVA will help indicate which factors and combinations are statistically significant and which are not thus giving direction to the priority of improvements. DOE Assumptions since ANOVA is used to analyze the data:

• The residuals are independent
• The residuals have equal variance
• The residuals are normally distributed
• All inputs (factors) are independent of one another

Most prediction equations will be linear and reliable when using only two levels. This saves time and money while obtaining a prediction equation. Prediction equations are useful to analyze what-if scenarios. Many times data can not be collected at all levels and factors so a prediction equation can be used to estimate the output. The input factors are x's and the response is Y-hat.

Full Factorial DOE

The following are characteristics of a Full Factorial DOE:

• Usually results is large number of tests. If the number of parameters is large then the number of test becomes significantly large (there are more and more interaction combinations and possibilities).
• Testing every combination of factor levels.
• Captures all interactions which of course is nice to have but this comes at a cost and time.

For instance, if there 9 factors and 3 levels for each factor that the team wants to test, then that is 3 9 = 19,683 runs to determine all the interactions! 

3 * 3 Full Factorial DOE

Using the same vehicle throughout and maintaining all external variables as constant as possible a study is being created to find a prediction equation for the miles per gallon (MPG). There are 27 runs needed to bring out all the interactions (3 3 ). The team has determined that coefficient of friction of surface, ambient temperature, and tire pressure are three critical input factors (KPIV's) to study. The goal isn't always to maximize MPG but to understand the impact on vehicle MPG based on these factors. The problem statement may be to improve the accuracy of MPG claims on this specific vehicle.

The table below summarizes the three levels chosen for each of the three factors.

How many trials are required if you want to run a Full Fractional DOE with 5 factors at 4 levels each?

ANSWER: 4 5 = 1,024 trials (this could be impractical...thus look into the option below).

Fractional Factorial DOE

A Fractional Factorial experiment uses subset of combinations from a Full Factorial experiment. They are applicable when there are too many inputs to screen practically or cost or time would be excessive. 

This type of DOE involves less time than One-Factor at a Time (OFAT) and a Full Fractional Factorial but this choice will result in less data and some interactions will be confounded (or aliased). This means that the effect of the factor cannot be mathematically distinguished from the effect of another factor.  

Most processes are driven by main effects and lower order interactions so choose the higher order interactions for confounding. Lower confounding is found with higher resolution.

If a half fractional factorial experiment is determined to be most practical and economical where there are two levels and five factors then there will be a combination of 16 runs analyzed. Usually higher order interactions are omitted to focus on the main effects.

One-Way Experiment:  involves only one factor. 

Response (Y, KPOV): the process output linked to the customer CTQ. This is a dependent variable.

Factor (X, KPIV): uncontrolled or controlled variable whose influence is being studied. Also called independent variables. Inference Space: operating range of factors under study Factor Level: setting of a factor such as 1, -1, +, -, hi, low. Treatment Combination (run): setting of all factors to obtain a response Replicate: number of times a treatment combination is run (usually randomized). Replication is done to estimate the Pure Trial Error to the Experimental Error. Replication is very important to under confounding and interactions. ANOVA : Analysis of Variance Blocking Variable: Variable that the experimenter chooses to control but is not the treatment variable of interest. Interaction: occurrence when the effects of one treatment vary according to the levels of treatment of the other effect.

Main effect: estimate of the effect of a factor that is independent from any of the other factors.

Collinear:  variables that are linear combinations of one another. Two perfectly collinear variables with an exact linear relationship will have correlation of -1 or 1. Confounding: variables that are not being controlled by the experimenter but can have an effect on the output of the treatment combination being studied. It describes the mixing of estimates of the effects from the factors and interactions. Two (or more) variables are confounded if effects of two or more factor aren't separable.

Sensitivi ty:  refers to the ability to identify significant treatment differences in the response variable.

Covariate:  Factors that change in an experiment that were not planned to change .

Explaining DOE

This is a lengthy video but it slowly but clearly teaches the concepts and jargon and then jumps into an example at the end. The prelude to the example helps put all the pieces together before diving into an example.

Other Types of DOE's

Taguchi's design.

Taguchi's Design uses orthogonal arrays to estimate the main effects of many levels or even mixed levels. A selected and often limited group of combinations are investigated to estimate the main effects.

The goal is to find and develop a parameter that can improve a performance characteristic. It can be used to look for alternative materials or design methods that deliver equivalent or better performance.

The intent is to reduce the quality loss to society. Taguchi has the concept of loss function and assumes losses when a process doesn't meet a target value. The losses are from the variation of a process output. He states that losses rise quadratically as they move from the target value to the LSL/USL (may be one or the other or both).

Signal to Noise (S/N) ratios are used to improve the design. Ideally, the output from the design should not react to variation from the noise factors. 

Plackett-Burman

Plackett-Burman - two level fractional factorial design that analyzes only a few selected combinations to evaluate on main effects and no interactions.

Response Surface Methodology

Response Surface Methodology (RSM)  is used to study multiple factors although two are normally done.

RSM creates a map of the response from running a series of full factorial DOE's and comes up equations that describe how the factors affect the response.

RSM designs are used to refine processes after an experiment such as Plackett-Burman has identified the vital main effects. Then one can determine the settings of the factors to achieve of the desired response. 

Circumscribed Central Composite (CCC), Face Centered Composite (CCF), and Inscribed Central Composite (CCI) are designs that require 5 factor levels. As you can see in their names, these are all varieties of central composite designs. They are (and also Box-Behnken) RSM designs. 

RSM's may have a 3D response. Consider the following equation that came from an experiment:

y = 3.2 + 4.5x 1  + 5.2x 2  + 9x 2 2  + 8.2x 1 x 2

The formula contains two slope components (4.5x 1  and 5.2x 2 ), and curve component (9x 2 2 ) and a twist component (8.2x 1 x 2 )

Box-Behnken

Similar to the Face Centered Composite (CCF) in which it requires 3 factor levels

Considered a Response Surface Methodology Design

Circumscribed Central Composite  (CCC)

Similar to Inscribed Central Composite (CCI) design which is also a higher order design and both require 5 factor levels.

Alpha, α, = [2 k ] 1/4  where k = number of factors

Alpha is the distance of each axial point (star point) from the center in a central composite design. A value <1 puts the axial points in the cube; a value =1 puts them on the faces of the cube; and a value >1 puts them outside the cube.

Return to the IMPROVE phase Get access to all the material within this site Search active Six Sigma job openings Return to Six-Sigma-Material Home Page

Recent Articles

Process capability indices.

Oct 18, 21 09:32 AM

Six Sigma Calculator, Statistics Tables, and Six Sigma Templates

Sep 14, 21 09:19 AM

Six Sigma Templates, Statistics Tables, and Six Sigma Calculators

Aug 16, 21 01:25 PM

• Templates + Calcs
• Training Slides
• Six Sigma Jobs

Practice Exam

• Six Sigma Courses

FULL Site Access

• Subscription

A few Member Pages

• Project Mgmt

Project Pitfalls

• Project Acceleration
• Power/Sample Size
• Hyp. Test Flowcharts
• Cost of Inventory
• Financial Savings
• Icebreakers
• Multi-Vari Study
• Fishbone Diagram
• Normalized Yield
• Spearman's Rho
• DEFINE Phase
• MEASURE Phase
• ANALYZE Phase
• IMPROVE Phase
• CONTROL Phase
• LEAN Manufacturing

Basic Statistics

• Predictive Maint.

Six Sigma Careers

• MBB Training
• BLACK BELT Training
• GREEN BELT Training
• YELLOW BELT Training
• Certifications
• Stats Software
• Quality Gurus
• Six Sigma Books

Six Sigma, Six Sigma Training, Courses, Calculators, Certification

Aug 15, 21 10:27 PM

Site Membership LEARN MORE

Templates, Tables & Calculators

Six Sigma Slides

Green Belt Program (1,000+ Slides)

Cost of Quality

Process Mapping

Capability Studies

Cause & Effect Matrix

Multivariate Analysis

Central Limit Theorem

Confidence Intervals

Hypothesis Testing

1-Way ANOVA

Correlation

Control Plan

MTBF and MTTR

Error Proofing

Line Balancing

Yield Metrics

Sampling Methods

Data Classification

... and more

Need a Gantt Chart?

Copyright © 2024 Six-Sigma-Material.com. All Rights Reserved. | Privacy Policy

Experimental Design: Types, Examples & Methods

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Experimental design refers to how participants are allocated to different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs.

Probably the most common way to design an experiment in psychology is to divide the participants into two groups, the experimental group and the control group, and then introduce a change to the experimental group, not the control group.

The researcher must decide how he/she will allocate their sample to the different experimental groups.  For example, if there are 10 participants, will all 10 participants participate in both groups (e.g., repeated measures), or will the participants be split in half and take part in only one group each?

Three types of experimental designs are commonly used:

1. Independent Measures

Independent measures design, also known as between-groups , is an experimental design where different participants are used in each condition of the independent variable.  This means that each condition of the experiment includes a different group of participants.

This should be done by random allocation, ensuring that each participant has an equal chance of being assigned to one group.

Independent measures involve using two separate groups of participants, one in each condition. For example:

• Con : More people are needed than with the repeated measures design (i.e., more time-consuming).
• Pro : Avoids order effects (such as practice or fatigue) as people participate in one condition only.  If a person is involved in several conditions, they may become bored, tired, and fed up by the time they come to the second condition or become wise to the requirements of the experiment!
• Con : Differences between participants in the groups may affect results, for example, variations in age, gender, or social background.  These differences are known as participant variables (i.e., a type of extraneous variable ).
• Control : After the participants have been recruited, they should be randomly assigned to their groups. This should ensure the groups are similar, on average (reducing participant variables).

2. Repeated Measures Design

Repeated Measures design is an experimental design where the same participants participate in each independent variable condition.  This means that each experiment condition includes the same group of participants.

Repeated Measures design is also known as within-groups or within-subjects design .

• Pro : As the same participants are used in each condition, participant variables (i.e., individual differences) are reduced.
• Con : There may be order effects. Order effects refer to the order of the conditions affecting the participants’ behavior.  Performance in the second condition may be better because the participants know what to do (i.e., practice effect).  Or their performance might be worse in the second condition because they are tired (i.e., fatigue effect). This limitation can be controlled using counterbalancing.
• Pro : Fewer people are needed as they participate in all conditions (i.e., saves time).
• Control : To combat order effects, the researcher counter-balances the order of the conditions for the participants.  Alternating the order in which participants perform in different conditions of an experiment.

Counterbalancing

Suppose we used a repeated measures design in which all of the participants first learned words in “loud noise” and then learned them in “no noise.”

We expect the participants to learn better in “no noise” because of order effects, such as practice. However, a researcher can control for order effects using counterbalancing.

The sample would be split into two groups: experimental (A) and control (B).  For example, group 1 does ‘A’ then ‘B,’ and group 2 does ‘B’ then ‘A.’ This is to eliminate order effects.

Although order effects occur for each participant, they balance each other out in the results because they occur equally in both groups.

3. Matched Pairs Design

A matched pairs design is an experimental design where pairs of participants are matched in terms of key variables, such as age or socioeconomic status. One member of each pair is then placed into the experimental group and the other member into the control group .

One member of each matched pair must be randomly assigned to the experimental group and the other to the control group.

• Con : If one participant drops out, you lose 2 PPs’ data.
• Pro : Reduces participant variables because the researcher has tried to pair up the participants so that each condition has people with similar abilities and characteristics.
• Con : Very time-consuming trying to find closely matched pairs.
• Pro : It avoids order effects, so counterbalancing is not necessary.
• Con : Impossible to match people exactly unless they are identical twins!
• Control : Members of each pair should be randomly assigned to conditions. However, this does not solve all these problems.

Experimental design refers to how participants are allocated to an experiment’s different conditions (or IV levels). There are three types:

1. Independent measures / between-groups : Different participants are used in each condition of the independent variable.

2. Repeated measures /within groups : The same participants take part in each condition of the independent variable.

3. Matched pairs : Each condition uses different participants, but they are matched in terms of important characteristics, e.g., gender, age, intelligence, etc.

Learning Check

Read about each of the experiments below. For each experiment, identify (1) which experimental design was used; and (2) why the researcher might have used that design.

1 . To compare the effectiveness of two different types of therapy for depression, depressed patients were assigned to receive either cognitive therapy or behavior therapy for a 12-week period.

The researchers attempted to ensure that the patients in the two groups had similar severity of depressed symptoms by administering a standardized test of depression to each participant, then pairing them according to the severity of their symptoms.

2 . To assess the difference in reading comprehension between 7 and 9-year-olds, a researcher recruited each group from a local primary school. They were given the same passage of text to read and then asked a series of questions to assess their understanding.

3 . To assess the effectiveness of two different ways of teaching reading, a group of 5-year-olds was recruited from a primary school. Their level of reading ability was assessed, and then they were taught using scheme one for 20 weeks.

At the end of this period, their reading was reassessed, and a reading improvement score was calculated. They were then taught using scheme two for a further 20 weeks, and another reading improvement score for this period was calculated. The reading improvement scores for each child were then compared.

4 . To assess the effect of the organization on recall, a researcher randomly assigned student volunteers to two conditions.

Condition one attempted to recall a list of words that were organized into meaningful categories; condition two attempted to recall the same words, randomly grouped on the page.

Experiment Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. Extraneous variables should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of taking part in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

Design of Experiments

DOE, or Design of Experiments, is a branch of applied statistics that uses planning, conducting, analyzing, and interpreting controlled tests to explain the variation of information under conditions hypothesized to reflect the variation. It’s a powerful data collection and analysis tool that investigates how different factors or variables affect an outcome or response of interest.

Design of Experiments (DOE) is also referred to as Designed Experiments or Experimental Design - all of the terms have the same meaning.

The term experiment is defined as the systematic procedure carried out under controlled conditions in order to discover an unknown effect, to test or establish a hypothesis, or to illustrate a known effect. When analyzing a process, experiments are often used to evaluate which process inputs have a significant impact on the process output and what the target level of those inputs should be to achieve a desired result (output). Experiments can be designed in many different ways to collect this information, and it's helpful to have software that guides you in the most accurate direction to answer your research questions. EngineRoom's DOE tool has built-in features that specifically cater to helping you design statistically sound experiments. It guides you through selecting a design streamlined in favor of the resources needed, but it's also powerful enough to detect an effect if it exists.

Experimental design can be used at the point of greatest leverage to reduce design costs by speeding up the design process, reducing late engineering design changes, and reducing product material and labor complexity. Designed Experiments are also powerful tools to achieve manufacturing cost savings by minimizing process variation and reducing rework, scrap, and the need for inspection.

This Toolbox module includes a general overview of Experimental Design and links and other resources to assist you in conducting designed experiments. A glossary of terms is also available at any time through the Help function, and we recommend that you read through it to familiarize yourself with any unfamiliar terms. For an additional resource, check out the web recording of our two-part webinar, Getting Started with DOE.

Preparation

If you do not have a general knowledge of statistics, review the Histogram , Statistical Process Control , and Regression and Correlation Analysis modules of the Toolbox prior to working with this module.

You can use the MoreSteam's data analysis software EngineRoom® to create and analyze many commonly used but powerful experimental designs.

Components of Experimental Design

Consider the following diagram of a cake-baking process (Figure 1). There are three aspects of the process that are analyzed by a designed experiment:

• Factors , or inputs to the process. Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are the ingredients for the cake and the oven that the cake is baked in. The controllable variables will be referred to throughout the material as factors. Note that the ingredients list was shortened for this example - there could be many other ingredients that have a significant bearing on the end result (oil, water, flavoring, etc). Likewise, there could be other types of factors, such as the mixing method or tools, the sequence of mixing, or even the people involved. People are generally considered a Noise Factor (see the glossary) - an uncontrollable factor that causes variability under normal operating conditions, but we can control it during the experiment using blocking and randomization. Potential factors can be categorized using the Fishbone Chart (Cause & Effect Diagram) available from the Toolbox.
• Levels , or settings of each factor in the study. Examples include the oven temperature setting and the particular amounts of sugar, flour, and eggs chosen for evaluation.
• Response , or output of the experiment. In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels. Experimenters often desire to avoid optimizing the process for one response at the expense of another. For this reason, important outcomes are measured and analyzed to determine the factors and their settings that will provide the best overall outcome for the critical-to-quality characteristics - both measurable variables and assessable attributes.

Purpose of Experimentation

Designed experiments have many potential uses in improving processes and products, including:

• Comparing Alternatives. In the case of our cake-baking example, we might want to compare the results from two different types of flour. If it turned out that the flour from different vendors was not significant, we could select the lowest-cost vendor. If flour were significant, then we would select the best flour. The experiment(s) should allow us to make an informed decision that evaluates both quality and cost.
• Identifying the Significant Inputs (Factors) Affecting an Output (Response) - separating the vital few from the trivial many . We might ask a question: "What are the significant factors beyond flour, eggs, sugar and baking?"
• Achieving an Optimal Process Output (Response). "What are the necessary factors, and what are the levels of those factors, to achieve the exact taste and consistency of Mom's chocolate cake?
• Reducing Variability . "Can the recipe be changed so it is more likely to always come out the same?"
• Minimizing, Maximizing, or Targeting an Output (Response). "How can the cake be made as moist as possible without disintegrating?"
• Improving process or product " Robustness " - fitness for use under varying conditions. "Can the factors and their levels (recipe) be modified so the cake will come out nearly the same no matter what type of oven is used?"
• Balancing Tradeoffs when there are multiple Critical to Quality Characteristics (CTQC's) that require optimization. "How do you produce the best tasting cake with the simplest recipe (least number of ingredients) and shortest baking time?"

Experiment Design Guidelines

The Design of an experiment addresses the questions outlined above by stipulating the following:

• The factors to be tested.
• The levels of those factors.
• The structure and layout of experimental runs, or conditions.

A well-designed experiment is as simple as possible - obtaining the required information in a cost effective and reproducible manner.

MoreSteam.com Reminder: Like Statistical Process Control, reliable experiment results are predicated upon two conditions: a capable measurement system, and a stable process. If the measurement system contributes excessive error, the experiment results will be muddied. You can use the Measurement Systems Analysis module from the Toolbox to evaluate the measurement system before you conduct your experiment.

Likewise, you can use the Statistical Process Control module to help you evaluate the statistical stability of the process being evaluated. Variation impacting the response must be limited to common cause random error - not special cause variation from specific events.

When designing an experiment, pay particular heed to four potential traps that can create experimental difficulties:

1. In addition to measurement error (explained above), other sources of error, or unexplained variation , can obscure the results. Note that the term "error" is not a synonym with "mistakes". Error refers to all unexplained variation that is either within an experiment run or between experiment runs and associated with level settings changing. Properly designed experiments can identify and quantify the sources of error.

2. Uncontrollable factors that induce variation under normal operating conditions are referred to as " Noise Factors ". These factors, such as multiple machines, multiple shifts, raw materials, humidity, etc., can be built into the experiment so that their variation doesn't get lumped into the unexplained, or experiment error. A key strength of Designed Experiments is the ability to determine factors and settings that minimize the effects of the uncontrollable factors.

3. Correlation can often be confused with causation. Two factors that vary together may be highly correlated without one causing the other - they may both be caused by a third factor. Consider the example of a porcelain enameling operation that makes bathtubs. The manager notices that there are intermittent problems with "orange peel" - an unacceptable roughness in the enamel surface. The manager also notices that the orange peel is worse on days with a low production rate. A plot of orange peel vs. production volume below (Figure 2) illustrates the correlation:

If the data are analyzed without knowledge of the operation, a false conclusion could be reached that low production rates cause orange peel. In fact, both low production rates and orange peel are caused by excessive absenteeism - when regular spray booth operators are replaced by employees with less skill. This example highlights the importance of factoring in operational knowledge when designing an experiment. Brainstorming exercises and Fishbone Cause & Effect Diagrams are both excellent techniques available through the Toolbox to capture this operational knowledge during the design phase of the experiment. The key is to involve the people who live with the process on a daily basis.

4. The combined effects or interactions between factors demand careful thought prior to conducting the experiment. For example, consider an experiment to grow plants with two inputs: water and fertilizer. Increased amounts of water are found to increase growth, but there is a point where additional water leads to root-rot and has a detrimental impact. Likewise, additional fertilizer has a beneficial impact up to the point that too much fertilizer burns the roots. Compounding this complexity of the main effects, there are also interactive effects - too much water can negate the benefits of fertilizer by washing it away. Factors may generate non-linear effects that are not additive, but these can only be studied with more complex experiments that involve more than 2 level settings. Two levels is defined as linear (two points define a line), three levels are defined as quadratic (three points define a curve), four levels are defined as cubic, and so on.

Experiment Design Process

The flow chart below (Figure 3) illustrates the experiment design process:

Test of Means - One Factor Experiment

One of the most common types of experiments is the comparison of two process methods, or two methods of treatment. There are several ways to analyze such an experiment depending upon the information available from the population as well as the sample. One of the most straight-forward methods to evaluate a new process method is to plot the results on an SPC chart that also includes historical data from the baseline process, with established control limits.

Then apply the standard rules to evaluate out-of-control conditions to see if the process has been shifted. You may need to collect several sub-groups worth of data in order to make a determination, although a single sub-group could fall outside of the existing control limits. You can link to the Statistical Process Control charts module of the Toolbox for help.

An alternative to the control chart approach is to use the F-test (F-ratio) to compare the means of alternate treatments. This is done automatically by the ANOVA (Analysis of Variance) function of statistical software, but we will illustrate the calculation using the following example: A commuter wanted to find a quicker route home from work. There were two alternatives to bypass traffic bottlenecks. The commuter timed the trip home over a month and a half, recording ten data points for each alternative.

MoreSteam Reminder: Take care to make sure your experimental runs are randomized - i.e., run in random order. Randomization is necessary to avoid the impact of lurking variables. Consider the example of measuring the time to drive home: if a major highway project is started at the end of the sample period increases commute time, then the highway project could bias the results if a given treatment (route) is sampled during that time period.

Scheduling the experimental runs is necessary to ensure independence of observations. You can randomize your runs using pennies - write the reference number for each run on a penny with a pencil, then draw the pennies from a container and record the order.

The data are shown below along with the mean for each route (treatment), and the variance for each route:

As shown on the table above, both new routes home (B&C) appear to be quicker than the existing route A. To determine whether the difference in treatment means is due to random chance or a statistically significant different process, an ANOVA F-test is performed.

The F-test analysis is the basis for model evaluation of both single factor and multi-factor experiments. This analysis is commonly output as an ANOVA table by statistical analysis software, as illustrated by the table below:

The most important output of the table is the F-ratio (3.61). The F-ratio is equivalent to the Mean Square (variation) between the groups (treatments, or routes home in our example) of 19.9 divided by the Mean Square error within the groups (variation within the given route samples) of 5.51.

The Model F-ratio of 3.61 implies the model is significant.The p-value ('Probability of exceeding the observed F-ratio assuming no significant differences among the means') of 0.0408 indicates that there is only a 4.08% probability that a Model F-ratio this large could occur due to noise (random chance). In other words, the three routes differ significantly in terms of the time taken to reach home from work.

The following graph (Figure 4) shows 'Simultaneous Pairwise Difference' Confidence Intervals for each pair of differences among the treatment means. If an interval includes the value of zero (meaning 'zero difference'), the corresponding pair of means do NOT differ significantly. You can use these intervals to identify which of the three routes is different and by how much. The intervals contain the likely values of differences of treatment means (1-2), (1-3) and (2-3) respectively, each of which is likely to contain the true (population) mean difference in 95 out of 100 samples. Notice the second interval (1-3) does not include the value of zero; the means of routes 1 (A) and 3 (C) differ significantly. In fact, all values included in the (1, 3) interval are positive, so we can say that route 1 (A) has a longer commute time associated with it compared to route 3 (C).

Other statistical approaches to the comparison of two or more treatments are available through the online statistics handbook - Chapter 7: Statistics Handbook .

Multi-Factor Designed Experiments

Multi-factor experiments are designed to evaluate multiple factors set at multiple levels. One approach is called a Full Factorial experiment, in which each factor is tested at each level in every possible combination with the other factors and their levels. Full factorial experiments that study all paired interactions can be economic and practical if there are few factors and only 2 or 3 levels per factor. The advantage is that all paired interactions can be studied. However, the number of runs goes up exponentially as additional factors are added. Experiments with many factors can quickly become unwieldy and costly to execute, as shown by the chart below:

See a Full Factorial Experiment in EngineRoom:

To study higher numbers of factors and interactions, Fractional Factorial designs can be used to reduce the number of runs by evaluating only a subset of all possible combinations of the factors. These designs are very cost effective, but the study of interactions between factors is limited, so the experimental layout must be decided before the experiment can be run (during the experiment design phase).

MoreSteam Reminder: When selecting the factor levels for an experiment, it is critical to capture the natural variation of the process. Levels that are close to the process mean may hide the significance of factor over its likely range of values. For factors that are measured on a variable scale, try to select levels at plus/minus three standard deviations from the mean value.

You can also use EngineRoom , MoreSteam's online statistical tool, to design and analyze several popular designed experiments. The application includes tutorials on planning and executing full, fractional and general factorial designs.

See a Fractional Factorial Experiment in EngineRoom:

Advanced Topic - Taguchi Methods

Dr. Genichi Taguchi is a Japanese statistician and Deming prize winner who pioneered techniques to improve quality through Robust Design of products and production processes. Dr. Taguchi developed fractional factorial experimental designs that use a very limited number of experimental runs. The specifics of Taguchi experimental design are beyond the scope of this tutorial, however, it is useful to understand Taguchi's Loss Function, which is the foundation of his quality improvement philosophy.

Traditional thinking is that any part or product within specification is equally fit for use. In that case, loss (cost) from poor quality occurs only outside the specification (Figure 5). However, Taguchi makes the point that a part marginally within the specification is really little better than a part marginally outside the specification.

As such, Taguchi describes a continuous Loss Function that increases as a part deviates from the target, or nominal value (Figure 6). The Loss Function stipulates that society's loss due to poorly performing products is proportional to the square of the deviation of the performance characteristic from its target value.

Taguchi adds this cost to society (consumers) of poor quality to the production cost of the product to arrive at the total loss (cost). Taguchi uses designed experiments to produce product and process designs that are more robust - less sensitive to part/process variation.

Choosing the Right DOE Software

When planning a DOE, it is essential to use statistical software that helps you design and analyze the most appropriate experiment to answer your research questions. EngineRoom's DOE tool has built-in features specifically designed to guide you through selecting a design streamlined in favor of the resources needed but also powerful enough to detect an effect if it exists.

It provides a comprehensive list of full, fractional, and general factorial designs to cover a wide variety of DOE scenarios. It also allows you to run automated algorithms to select the best model for the data, making it easier to draw conclusions and take informed actions. Using EngineRoom for your designed experiments can save time, reduce costly errors, and help make data-driven decisions.

See a General Factorial Experiment in EngineRoom:

Designed experiments are an advanced and powerful analysis tool during projects. An effective experimenter can filter out noise and discover significant process factors. The factors can then be used to control response properties in a process and teams can then engineer a process to the exact specification their product or service requires.

A well built experiment can save not only project time but also solve critical problems which have remained unseen in processes. Specifically, interactions of factors can be observed and evaluated. Ultimately, teams will learn what factors matter and what factors do not.

Learn more about EngineRoom

• Webster's Ninth New Collegiate Dictionary

Additional Online Resources

• An excellent online Statistics Handbook is available that covers Design of Experiments and many other topics. See Section 5 - "Improve" for a complete tutorial on Design of Experiments.
• Check the White Paper Section for related online articles.
• Mark J. Anderson and Patrick J. Whitcomb, DOE Simplifie d (Productivity, Inc. 2000). ISBN 1-56327-225-3. Recommended - This book is easy to understand and comes with copy of excellent D.O.E. software good for 180 days.
• George E. P. Box, William G. Hunter and J. Stuart Hunter, Statistics for Experimenters - An Introduction to Design, Data Analysis, and Model Building (John Wiley and Sons, Inc. 1978). ISBN 0-471-09315-7
• Douglas C. Montgomery, Design and Analysis of Experiments (John Wiley & Sons, Inc., 1984) ISBN 0-471-86812-4.
• Genichi Taguchi, Introduction to Quality Engineering - Designing Quality Into Products and Processes (Asian Productivity Organization, 1986). ISBN 92-833-1084-5

See more tools

Check out additional tools in the Lean Six Sigma tool belt

Want to learn more?

MoreSteam.com offers a wide range of Lean Six Sigma online courses, including Black Belt, Green Belt, and DFSS training. Flexible training at an affordable price.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Optimization of culture condition for Spodoptera frugiperda by design of experiment approach and evaluation of its effect on the expression of hemagglutinin protein of influenza virus

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliations Biotechnology Research Center, Department of Medical Biotechnology, Pasteur Institute of Iran, Tehran, Iran, Department of Research & Development, AryoGen Pharmed Inc., Karaj, Iran

Roles Conceptualization, Methodology, Project administration

* E-mail: [email protected] , [email protected] (YT); [email protected] (HA)

Affiliation Department of Research & Development, AryoGen Pharmed Inc., Karaj, Iran

Roles Conceptualization, Funding acquisition, Project administration

Roles Data curation, Formal analysis, Methodology, Project administration, Validation, Visualization, Writing – review & editing

Affiliation Biotechnology Research Center, Department of Medical Biotechnology, Pasteur Institute of Iran, Tehran, Iran

Roles Methodology, Project administration, Writing – review & editing

Affiliation Molecular Virology Lab, Department of Microbiology, School of Biology, College of Science, University of Tehran, Tehran, Iran

Roles Methodology

Roles Investigation

• Fatemeh Alizadeh, 
• Hamideh Aghajani, 
• Fereidoun Mahboudi, 
• Yeganeh Talebkhan, 
• Ehsan Arefian, 
• Sepideh Samavat, 
• Rouhollah Raufi

• Published: August 16, 2024
• https://doi.org/10.1371/journal.pone.0308547
• Reader Comments

The baculovirus expression vector system (BEVS) is a powerful tool in pharmaceutical biotechnology to infect insect cells and produce the recombinant proteins of interest. It has been well documented that optimizing the culture condition and its supplementation through designed experiments is critical for maximum protein production. In this study, besides physicochemical parameters including incubation temperature, cell count of infection, multiplicity of infection, and feeding percentage, potential supplementary factors such as cholesterol, polyamine, galactose, pluronic-F68, glucose, L-glutamine, and ZnSO 4 were screened for Spodoptera frugiperda (Sf9) cell culture and expression of hemagglutinin (HA) protein of Influenza virus via Placket-Burman design and then optimized through Box-Behnken approach. The optimized conditions were then applied for scale-up culture and the expressed r-HA protein was characterized. Optimization of selected parameters via the Box-Behnken approach indicated that feed percentage, cell count, and multiplicity of infection are the main parameters affecting r-HA expression level and potency compared to the previously established culture condition. This study demonstrated the effectiveness of designing experiments to select and optimize important parameters that potentially affect Sf9 cell culture, r-HA expression, and its potency in the BEVS system.

Citation: Alizadeh F, Aghajani H, Mahboudi F, Talebkhan Y, Arefian E, Samavat S, et al. (2024) Optimization of culture condition for Spodoptera frugiperda by design of experiment approach and evaluation of its effect on the expression of hemagglutinin protein of influenza virus. PLoS ONE 19(8): e0308547. https://doi.org/10.1371/journal.pone.0308547

Editor: Haitham Mohamed Amer, Cairo University Faculty of Veterinary Medicine, EGYPT

Received: May 24, 2024; Accepted: July 26, 2024; Published: August 16, 2024

Copyright: © 2024 Alizadeh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: This work was financially supported by Aryogen Pharmed Company, Iran. No additional external funding was received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Recombinant proteins produced in prokaryotic (mostly bacteria) and eukaryotic (fungi, insect, and mammalian) expression systems are used as vaccines and therapeutics [ 1 ]. Since 1983, baculovirus expression vector systems (BEVS) within insect cells have been widely used to produce recombinant proteins, designation of viral vectors for gene therapy and gene delivery, and antigen carriage [ 2 ]. These viruses are the most prominent double-stranded, circular DNA ones that infect insect cells [ 3 , 4 ] mainly Lepidopteran cell lines including S . frugiperda (Sf21 and Sf9), and Trichoplusia ni (HighFiveTM) which are the commonly used cells in suspension cultures [ 5 ].

The advantages of BEVS such as high expression level, ease of scale-up, adaptability to suspension culture, and acceptable post-translational modifications (PTMs) make it a powerful system for the production of recombinant proteins [ 6 , 7 ]. Well-characterized products originating from these systems include CERVARIX® (against cervical cancer), PROVENGE® (against prostate cancer), FluBlok® (influenza vaccine), and Nuvaxovid® (covid-19 vaccine) [ 8 ].

According to the World Health Organization (WHO) reports, annual influenza occurrence is one of the most important epidemics that involves approximately 5–15% of the Northern Hemisphere population and causes about 290,000 to 650,000 global respiratory deaths indicating vaccination is the most efficient preventive approach [ 9 ]. Amongst four types of influenza virus (A, B, C, and D), types A and B are usually included in annual influenza vaccines. In comparison, the other two types have not represented significant impacts on human population health [ 10 ]. Influenza type A has more than 100 subtypes based on random mutations that usually occur within its two surface proteins, hemagglutinin (HA) and neuraminidase (NA) which the former is a spike-shaped protein that sticks to the viral surface [ 10 ]. Therefore, influenza (flu) vaccines need to be updated annually [ 10 ]. Unlike traditional egg-based flu vaccines which encounter disadvantages such as pathogen-free egg shortage, complex purification procedures, egg-based protein impurities, antibiotics, and preservatives within the final product [ 9 ], the BEVS-based recombinant flu vaccines compromise HA gene fragment replaced with polyhedrin gene and expressed under the control of its promoter within virally infected insect cells [ 11 – 13 ] that has no infective live virus particle and is free from adjuvants, antibiotics, preservatives, possible pathogens, and unrelated proteins [ 9 ].

Common basic media for insect cell culture are usually chemically defined and consist of amino acids, sugars, vitamins, organic acids, and inorganic salts [ 14 ] which their selection and screening are critical in any process development [ 15 ]. Therefore, plenty of time and budget is usually paid for media selection and optimization of culture conditions [ 16 , 17 ] through the design of experiments (DOE) [ 18 , 19 ]. DOE is a statistical approach for identifying effective parameters and their optimal levels through defined experiments to lower costs and increase efficiency [ 20 ]. Several DOE approaches (Full and fractional factorial) are designated based on the number of selected parameters, their defined levels, and the time and budget dedicated to each study [ 20 ].

Increasing the HA potency through optimization procedures inevitably reduces the industrial production scale and costs which are great achievements for the healthcare systems. Therefore, in the present study, potentially effective Sf9 culture parameters/supplements were selected by a literature review which include feed percentage, cell count of infection (CCI), multiplicity of infection (MOI), temperature, cholesterol, polyamine, galactose, pluronic-F68, initial cell density, glucose, L-glutamine and ZnSO 4 [ 10 , 20 – 34 ]. In the next step, their impacts on r-HA expression were screened using DOE and response surface methodology (RSM) to reduce the cost and time of experiments.

Materials and methods

Cell culture and viral infection.

Sf9 cells (OET, UK) were cultured in 125 ml shake flasks containing 30 ml PSFM-J1 medium (Fujifilm, Japan) and incubated at 28°C, 85 rpm (pH 6.0–6.4; osmolality of 345–380 mOsm/kg) with an initial cell density of 1.5–2×10 6 cells/ml. The cells were sub-cultured when viability and cell density reached ≥90% and 4–5×10 6 cells/ml, respectively. After 5 passages, viral infection was done based on the desired CCI (cells/ml): MOI (PFU/ml) ratio. After 48 to 96 hours when the viability dropped to 40–60%, the cells were harvested by centrifugation at 4,000 rpm for 15 min.

Design of experiment by Plackett-Burman approach

Selected parameters for optimization of insect cell culture were initial cell density (0.8–2×10 6 cells/ml), feed percentage (3–9% of the total culture volume), CCI (2–6×10 6 cells/ml), MOI (0.3–3 PFU/ml), temperature (22–28°C), cholesterol (4–20 mg/L), polyamine (0.5–1.5X), galactose (10–30 mM), pluronic-F68 (0.1–0.5 w/v), glucose (10–20 g/L), L-glutamine (10–20 mM), and ZnSO 4 (10–40 μM). All selected parameters were screened by Plackett-Burman design in Design Expert® (v.11) in 23 experiments at 2 (low and high) levels ( S1 Table ) and their effects on HA expression level (the main objective), cell density, and cell viability were daily monitored within 500 ml baffled shake flasks with a working volume of 100 ml. The cells were harvested by centrifugation at 4,000 rpm for 15 min when the viability dropped to 40–60%. The optimization procedure was conducted in comparison to the previously developed un-supplemented culture condition in which a defined constant CCI: MOI ratio (6.0 ×10 6 cells/ml: 0.5 PFU/ml) was applied for viral infection after 72h incubation and the cells were harvested at 40–60% viability.

Evaluation of protein expression

Response surface methodology

In the next step, the optimization of significantly effective parameters was performed using Box-Behnken response surface methodology in three levels (low, middle, and high) by Design Expert. Fifty-four experiments were run ( S2 Table ). Cell culture procedure, protein extraction, and expression evaluation were done according to the mentioned protocols. Cell viability and HA expression level were evaluated as the main responses.

Bioreactor culture

Based on DOE experiments, the optimal conditions were applied to benchtop 2 L bioreactors (Eppendorf, Germany) where the main responses were monitored compared to the previously described established control culture condition. In brief, the insect cells were expanded for up to 4 passages and transferred into three 2 L benchtop bioreactors (working volume of 1700 ml) under control and optimized conditions. After 72 h, the cells were infected with optimized infection condition and incubated for protein expression till the viability dropped to 40–60%.

r-HA protein purification

The soluble protein fraction was extracted from harvested cells and filtered as previously described. The filtered protein solution was loaded on DEAE-S resin (Arg Biotech, Iran) where r-HA was separated in a flow-through mode. The diluted r-HA containing flow-through sample was loaded on Hi-Trap Capto Lentil Lectin affinity chromatography resin (Cytiva, USA) within the equilibration buffer (30 mM Tris, 500 mM NaCl, 0.05% Triton X-100, 0.01% 2-ME, pH 8.3, conductivity of 4.7 ms/cm) and eluted by the elution buffer (30 mM Tris, 200 mM D-glucose, 0.003% Tween-20, 0.03% Triton X-100, 150 mM NaCl, pH 7.4, conductivity of 14 ms/cm), buffer exchanged in phosphate buffer (7.3 mM Na 2 HPO 4 , 2.8 mM NaH 2 PO 4 , 150 mM NaCl) by diafiltration and its concentration was measured by BCA method.

Potency measurement

Single Radial Immunodiffusion assay (SRID) has been internationally recognized as the gold standard method in the potency assessment of HA protein. The assay is based on the reaction of HA and its specific antibody in which HA concentration can be proportionally calculated by the size of the ring (diameter) relative to the known standard HA protein. In this assay, standard HA protein (Strain 2-Influenza Antigen-B-Phuket-3073-2013-NIBSC, UK; 66 to 0.515 μg/ml concentrations) and purified r-HA (1:1 to 1:128 dilutions) were loaded into wells of agarose gel supplemented with annually generated anti-HA antibodies. After 18–20 h incubation at 25°C, the wells were stained with coomassie blue and the produced immune-precipitin rings were quantified with Digimizer® software.

Protein characterization

The purified r-HA protein was characterized compared to the standard HA protein. The characterization tests included purity, native folding, glycan profiling, and size heterogeneity. Recombinant HA characteristics were compared to the original protein through standard and homemade assays.

Statistical analysis

The obtained data has been presented as the mean of three experiments with standard deviation (SD) after evaluation of data normalization with the Shapiro-Wilk normality test. The mean values were compared using the student’s t-test. ANOVA analysis was applied in optimization studies. The significant differences were considered to be 0.05. Figures were created using GraphPad Prism (v.8.0) and Design Expert (v.11.0).

Screening of culture conditions by Plackett-Burman design

Plackett-Burman design was applied to screen selected parameters and study their effects on cell density and HA expression level. Other responses such as cell viability were also analyzed. Amongst the studied parameters, six parameters had significant effects on r-HA expression level, while only one affected cell density ( Fig 1 and S3 Table ).

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

The charts demonstrate the order and effect of each parameter on (A) r-HA expression level and (B) Viable cell count.

https://doi.org/10.1371/journal.pone.0308547.g001

The ANOVA results summarized in Table 1 suggested the initial cell density as the only effective positive parameter for cell count response. Feed percentage, temperature, and cholesterol had positive effects on HA expression level, while CCI, MOI, and pluronic represented significantly negative effects ( Fig 1 ).

https://doi.org/10.1371/journal.pone.0308547.t001

Viable cell count and HA expression level were expressed as empirical first-order polynomial equations in Eqs 1 and 2 where A, B, C, D, E, H, and J represent feed percentage, CCI, MOI, temperature, cholesterol, pluronic, and initial cell density, respectively.

Fig 2 shows the comparative analysis of 23 Placket-Burman-designed experiments on cell count and HA expression level as the main responses which confirms the ANOVA results.

(A) Cell count and (B) r-HA expression. Error bars represent the SD of three independent experiments.

https://doi.org/10.1371/journal.pone.0308547.g002

Optimization of culture condition by RSM

The significantly effective parameters selected from Plackett-Burman experiments (feed percentage, CCI, MOI, temperature, cholesterol, and pluronic) were further investigated in three levels within 54 experiments by response surface methodology (Box-Behnken) ( S2 Table ) amongst the feed percentage, CCI, and MOI were defined as statistically significant parameters ( Table 2 ). Analysis of r-HA expression level represented a significant difference in experiment 14 ( Fig 3 ). Other responses including viable cell count and viability were also monitored ( S4 Table ). As previously mentioned, the cells were intended to be harvested with 40–60% viability on day 7. However, in some experiments, the cells were harvested at earlier times due to their low cell count and viability.

The highest expression level has been achieved under conditions described in experiment#14 (Black bar). Error bars represent the SD of three independent experiments.

https://doi.org/10.1371/journal.pone.0308547.g003

https://doi.org/10.1371/journal.pone.0308547.t002

Response surface 3D plots represent the interactions between r-HA expression level and (A) Feed percentage and TOI; (B) Feed percentage and MOI; (C) CCI and MOI.

https://doi.org/10.1371/journal.pone.0308547.g004

(A) r-HA expression level in different culture conditions (Control condition in bioreactor, optimized condition in shake flask, and optimized condition in bioreactor: 71, 239, and 205 μg/ml, respectively); (B) SDS-PAGE (non-reduced samples): M: Mw marker; #1: Standard HA protein; #2: Cell lysate from control culture condition; #3: Cell lysate from optimized culture condition; #4: Eluate of lectin resin from control culture condition; #5: Eluate of lectin resin from the optimized condition. **** represents P <0.0001. Error bars show the SD of triplicate runs.

https://doi.org/10.1371/journal.pone.0308547.g005

Scale-up of the optimized culture condition

To validate the obtained optimal condition from RSM experiments, fed-batch mode 2 L benchtop bioreactors were run under optimized condition (5.37% feed, CCI of 5.32×10 6 cells/ml, MOI of 1.65 PFU/ml, 12 mg/L cholesterol, 0.35% pluronic at 23.31°C) when the control batches were run without any supplementation under CCI/MOI ratio of 6/0.5 at 28°C. The cells were harvested and the r-HA protein was purified as previously described ( Fig 5B ). The obtained r-HA expression level confirmed the reproducibility of the RSM results which was significantly elevated (3.6 folds) compared to the control culture condition ( Fig 5A ).

Potency assay

The potency of r-HA expressed under control and optimized conditions was evaluated through the SRID method ( Fig 6 ) compared to the standard HA protein. Through similar dilution rates (1:1 to 1:128), the potency of r-HA proteins expressed under control and RSM-optimized conditions were 128 and 332 μg/ml, respectively indicating a 2.6-fold increased potency.

https://doi.org/10.1371/journal.pone.0308547.g006

Following final filtration and formulation, protein characterization of the r-HA drug substance was done for glycosylation pattern, size heterogeneity, native fold, and host cell protein content in comparison to the original product (Flublok®) characteristics by standard and homemade assays ( Table 3 , S1 Fig ).

https://doi.org/10.1371/journal.pone.0308547.t003

According to the literature, different physicochemical and supplemental parameters have been repeatedly reported as potentially important factors in cell culture and protein expression. To our knowledge, optimization of these critical parameters using DOE has not been reported in the case of HA protein expression in Sf9 insect cell culture. Therefore, in the present study, a set of experiments was designed to screen and then optimize the important selected parameters involved in the suspension culture of insect cells through the design of the experiment approach to achieve a higher level of recombinant HA protein expression. Due to ethical, safety, cost, and regulatory concerns [ 35 – 37 ], the selected supplements had non-animal origins.

As previously mentioned, the design of the experiments includes multiple methods such as one factor at a time, full factorial, and fractional factorial design. In a full factorial design, all selected parameters are examined at all levels. This approach is the most comprehensive, but it is also costly and time-consuming [ 20 ]. Fractional factorial design (screening) is a type of full factorial design that uses fewer experiments to save time and budget [ 20 ]. Plackett-Burman, a two-level classical screening method, identifies critical parameters and their main effects without considering the interactions between studied factors [ 38 ]. Response surface methodology (RSM) fits mathematical models to identify the optimal levels of the studied parameters, their combination, and predicts the responses using the obtained equations [ 20 ]. The most commonly used approaches in RSM include Box-Behnken design (BBD) and the central composite design (CCD) [ 39 ]. BBD, the most efficient RSM approach, provides reliable information through a minimum number of experiments compared to CCD. It requires three levels (-1, 0, 1) for each parameter and can be applied to 3 to 21 numerical and categorical parameters. It aids in detecting nonlinearity and interactions among parameter factors [ 39 ]. In CCD, five levels (-α, -1, 0, 1, α as 1.414) will be defined for 2 to 50 parameters. Similar to BBD, CCD can be applied for both numerical and categorical parameters and helps in the detection of nonlinearity and interactions among parameters [ 39 ]. Taken together, BBD was selected which defines 3 levels for each parameter.

The initial screening of 12 selected parameters was done by the Placket-Burman approach and 6 selected parameters underwent further optimization to reach the optimized conditions. Polynomial regression and RSM were used in the present study due to the mild nonlinearity among studied variables. Support vector regression (SVR), multiple linear regression, and artificial neural network (ANN) are also powerful theoretical methods based on statistical learning theory which deal with data nonlinearities [ 40 , 41 ]. The use of machine learning (ML) and artificial intelligence has attracted attention in applied sciences like Biotechnology and Biology. It evaluates suitable and experimental models to identify potential patterns, especially in the case of complex and huge datasets. In the bioprocess development and scale-up procedure of biotechnological products, it is crucial to identify optimal parameters/conditions. Due to the huge number of potentially unknown parameters and processes, the application of ML can help in designing the experiments, predicting the models, and identifying the significant parameters (based on the selected outputs such as cell viability, and protein expression level) in a shorter time at a lower cost [reviewed in 42 ].

A 5% feeding strategy was observed to have a significant positive effect on HA expression level. Previous studies have shown that during viral infection, nutritional depletion can reduce the expression of recombinant proteins in insect cells [ 43 ]. On the other hand, the accumulation of by-products in the culture medium may negatively impact cellular physiology for the expression of recombinant proteins [ 44 ]. This phenomenon can be overcome through fed-batch cultivation which significantly enhances the production of desired proteins [ 10 , 45 – 47 ].

On the other hand, the significant negative effect of CCI on HA expression level could be due to the nutritional depletion following uncontrolled cell density increase. Therefore, it is important to carefully select the optimum cell density value at the time of the infection and its ratio to the viral load to prevent the lack of nutrients and accumulation of toxic compounds to maximize cellular productivity [ 22 , 44 ]. Hence, a CCI of 5 ×10 6 cells/ml was defined as the optimal value to achieve the highest protein expression level. Low MOI values (<1PFU/ml) not only offer economic advantages but also prevent nutritional depletion during the post-infection phase. In contrast, high MOI values (>2 PFU/ml) lead to a synchronous infection which stops cell growth following the infection [ 22 , 23 ]. Our results demonstrated that MOI values near 1.65 PFU/ml could result in optimal protein expression levels.

The role of temperature, as a physicochemical parameter, was also investigated and a statistically significant effect was found. Previous studies have shown that insect cell growth and its ability to express recombinant proteins can significantly increase by lowering the temperature from 27°C [ 24 , 48 – 50 ]. Although the temperature shift had a significant effect in the Placket-Burman screening step, it was not significantly effective on protein expression in the RSM optimization step.

Previous studies have indicated that adding polyamine improves membrane rigidity and prevents lipid oxidation. It results in nucleic acid stabilization and transcription regulation which have positive impact on the production of enveloped viruses [ 26 , 51 – 53 ].

Cholesterol is also assumed an external essential additive for insect cell culture which is required for the flexibility of the cell membrane [ 24 , 25 ]. It also has been reported that adding cholesterol and polyamines together can boost the specific yields of BVs by 7-fold [ 26 ]. Therefore, optimization of their concentrations in the culture medium will directly affect the yield of produced viral particles. Although cholesterol (4 mg/L) represented a positive effect in PB-designed experiments, it did not show significant interactions within RSM-designed analyses.

Copolymers like Pluronic are usually recommended for insect cell bioreactor cultures for protection against shear stress of agitation and sparging [ 22 , 54 ] and their optimum concentration should be reached. Analysis of the Placket-Burman designed experiments revealed that this additive has a significant effect on HA expression, which was not confirmed in RSM. It can be assumed that other parameters, such as the unpublished components of the feed, may neutralize its effect.

Glucose and L-glutamine are the most important sources of carbon and nitrogen that can be metabolized by Sf9 cells. Supplementation of culture medium with different concentrations of these two components has resulted in varied cell growth density, and specific yields of the recombinantly expressed proteins [ 23 , 33 , 55 – 57 ]. It also has been shown that supplementation of insect cell culture medium with metal ionic salts such as ZnSO 4 can enhance viral replication and consequently the yield of the recombinant protein up to 100% through reducing the negative charge of the surface of the virus and the insect cell [ 31 ]. However, these findings were not supported by the present study, possibly due to their unknown concentrations in the basal medium used.

The cell density was monitored as another main response besides HA expression level. The initial cell density was the only factor affecting this response, reconfirming the importance of the seeding strategy through the Placket-Burman designed experiments for more rapid arrival to the S-phase of the cell cycle [ 30 , 58 – 61 ]. Taken together, supplementation of culture medium through selection and optimization steps resulted in approximately 3.36-fold increased r-HA expression level. Although our results reconfirmed the success of traditional DOE approaches using ANOVA regression analysis in the optimization of parameters, it is worth noting that newly developed methods such as artificial intelligence, machine learning (ML), and global sensitivity analysis (using SOBOL’ sampling method) should get more attention for a better and deeper understanding of the effective uncertain parameters [ 62 ].

Due to the importance of the scale-up procedure in eukaryotic cell cultures and because of cell sensitivity to the shear stress [ 63 , 64 ], we conducted a scale-up process, shifting from laboratory 500 ml shake flasks (working volume of 100 ml) to 2L bioreactor systems under RSM-optimized condition in which reproducibility of the procedure and r-HA expression level were reconfirmed. The SRID potency assay revealed that r-HA protein expressed under RSM-optimized conditions is 2.6-fold more potent than HA protein produced under control conditions. This improvement could be due to the optimization and supplementation of the cell culture medium. Further characterization studies represented comparable results of r-HA protein expressed under un-supplemented and optimized conditions when the original commercially available recombinant protein was used as the reference protein. The observed comparability revealed that manipulation of culture conditions and its supplementation had no negative effect on the qualitative properties of the protein.

Conclusions

Our study showed that the design of experiments (DOE) resulted in increased r-HA protein expression and potency compared to the control condition where supplementation of culture medium did not occur. Due to the importance of HA potency for its application, any increase in potency would reduce the amount of protein required for filling, leading to cost reduction in industrial manufacturing. Besides, an important issue in optimization studies is the identification of effective parameters for sensitivity and uncertainty analysis to ensure the feasibility and robustness of the studies, which should be considered in the next complementary experiments.

Supporting information

S1 table. plackett-burman experimental design..

https://doi.org/10.1371/journal.pone.0308547.s001

S2 Table. Box-Behnken designed experiments.

https://doi.org/10.1371/journal.pone.0308547.s002

Daily effect of studied parameters on A) viability, B) viable cell count within 23 Placket-Burman-designed experiments. The gray cells do not have data due to the early harvesting.

https://doi.org/10.1371/journal.pone.0308547.s003

Effect of studied parameters on A) Viable cell count, B) Viability within designed 54 experiments. The gray cells do not have data due to the early harvesting.

https://doi.org/10.1371/journal.pone.0308547.s004

S1 Fig. Characterization of r-HA drug substance.

Glycosylation pattern: (A) Reference HA protein, (B) r-HA produced under control condition, (C) r-HA produced under optimized condition. Size Heterogeneity: (D) Reference HA protein, (E) r-HA produced under control condition, (F) r-HA produced under optimized condition. Native folding (reduced SDS-PAGE): (A) Reference HA protein; (B) r-HA produced under control condition; (C) r-HA produced under optimized condition: #1, 6, 9: HA protein treated with high concentration of Trypsin (1050 μg/ml); #2, 5, 8: HA protein treated with low concentration of Trypsin (210 μg/ml); # 3, 4, 7: HA protein without Trypsin digestion.

https://doi.org/10.1371/journal.pone.0308547.s005

Acknowledgments

The authors express their gratitude to Maryam Mirfakhraei for her assistance with the statistics in this study. Special thanks are also extended to Mohammad Saffarioun (CEO of AryoGen Pharmed Inc.) for his unwavering support throughout this endeavor.

• 1. Bartfai T, Lees GV. The future of drug discovery: who decides which diseases to treat? 1 st ed. USA: Academic Press; Elsevier Sceinces; 2013.
• View Article
• PubMed/NCBI
• Google Scholar
• 4. Rohrmann GF. Baculovirus Molecular Biology. 3 rd ed. National Center for Biotechnology Information (US). Bethesda (MD); 2013.
• 5. van Oers MM, Lynn DE. Insect Cell Culture. In: Encyclopedia of Life Sciences Wiley; 2010.
• 7. Luckow V. Cloning and expression of heterologous genes in insect cells with baculovirus vectors. In: Prokop A, Bajpai RK, Ho C, editors. Recombinant DNA technology and applications. New York: McGraw Hill; 1991:97–152.
• 10. Stein R. Process Intensification of Spodoptera frugiperda (Sf) Cell Growth via Multi-Parallel Bioreactor System. MSc. Thesis, Massachusetts Institute of Technology: Massachusetts Institute of Technology; 2020. Available from https://dspace.mit.edu/handle/1721.1/127258 .
• 13. Cheng Y-L. Investigation of Sf-9 cell metabolism before and after baculovirus infection using biovolume: A case for the improvement of adeno-associated viral vector production. MSc. Thesis. The University of Waterloo. 2009. Available from https://uwspace.uwaterloo.ca/handle/10012/4540 .
• 20. Lamidi S, Olalere R, Yekinni A, Adesina K. Design of Experiments (DOE): Applications and benefits in quality control and assurance. In: Qamar SZ, Al-Hinai N, editors. Quality Control and Quality Assurance—Techniques and Applications. Rijeka: IntechOpen; 2024. Available from: http://dx.doi.org/10.5772/intechopen.113987 .
• 39. Beg S. Design of experiments for pharmaceutical product development: Volume II: Applications and practical case studies. 1 st ed. Springer; 2021.

File(s) under embargo

until file(s) become available

Design, Implementation, and Evaluation of a Quantum-Infused Middle-School Level Science Teaching and Learning Sequence

This dissertation explores the integration of Quantum Information Science and Engineering (QISE) into formal K-12 curriculum through a Design-Based Research (DBR) approach. The overarching purpose is to develop a NGSS-aligned quantum-infused science curriculum unit for middle school students, aiming to enhance student understanding and engagement in quantum randomness. The study emphasizes the sequential introduction of concepts (from radioactive decay to quantum computing), interdisciplinary inquiry-based learning, and alignment of content and assessment strategies by leveraging Learning Progressions (LPs) and Hypothetical Learning Trajectories (LTs). Methods employed in this DBR study included iterative design processes, teacher feedback, and teaching experiments with 10 participant in-service middle school science teachers as well as quantitative assessment and evaluation of students’ learning and engagement data. Also, it is aimed to focus on professional development for teachers, incorporating NGSS and the Framework as the foundational guidelines. Findings highlighted the importance of teacher feedback in refining educational strategies, the challenges of teaching advanced quantum concepts at the middle school level, and the benefits of using classical physics as a gateway to introduce quantum concepts. This study is also manifestation of a structured teaching-learning pathway, guided by validation and hypothetical LPs, to support students' progression of understanding towards more sophisticated knowledge in QISE. Implications included the potential for enhancing coordination and sequencing of QISE teaching at the K-12 level, contributing to the cultivation of a diverse and quantum-savvy workforce. This DBR study hoped to set a foundation for future research endeavors, emphasizing the need for comprehensive teacher training in K-12 QISE education and the transformative power of education in fostering deeper comprehension and engagement with complex subjects.

HQ0034-21-1-0014

Degree type.

• Doctor of Philosophy
• Curriculum and Instruction

Campus location

• West Lafayette

Advisor/Supervisor/Committee Chair

Advisor/supervisor/committee co-chair, additional committee member 2, additional committee member 3, additional committee member 4, additional committee member 5, usage metrics.

• Curriculum and pedagogy theory and development