Introduction¶

Planning experiment to promote good decision making, with minimum cost

Counter-intuitive point: the best time to design experiment is after experiment is finished, especially the first time when you don’t know the underlying data-generating response process

Experiment¶

Deliberate variation of one/more process variables while observing effect on one/more response variables

Experimental Design¶

• Maximize information gain
• Minimize resources (time and cost)

DOE is a procedure to plan experiments to efficiently provide valid conclusions

Does the experiment have enough statistical power to answer research questions?

Process¶

1. Define objectives of experiment
2. Define inputs to the process

Types of Input Vars¶

Uses¶

• Exploratory work
• Comparison: Choosing between alternatives
• Screening: Selecting key factors that affect a response
• Response surface modeling: Optimize given process
• Hitting and control target response with minimum variability
• Maximize/minimize response/goal
• Increase process robustness
• Prediction

RSM¶

Response Surface Methodology

Looks for quadratic/higher order trends

Assumes all variables significant

Quadratic response always has a stationary point (min/max/saddle point)

Can be used to optimize a process $$ y_i = \beta_0 + \sum_{i=1}^k \beta_i x_i + \sum_{i=1}^k \beta_{ii} x_i^2 + \sum_{i} \sum_{j>i} \beta_{ij} x_i x_j + u_i $$ One at a time

Properties¶

Notes¶

• Beware of extrapolation
• Multiple responses
• Overlapping response contour plots
• Combined cost function
• PCA can be useful

Pitfalls of DOE & RSM¶

• Optimization of process without understanding the process
• Design assumes a model, so there is no way to test model error as the model always fits
• A quadric model will always show you an optimum point, but its accuracy depends on the accuracy of the model
• Exponential data fir with a quadratic model will show an optimum that does not exist

DOE for Prediction¶

Goal: equalize the leverage of every point

Optimal design¶

Algorithmic approach to searching the design space and pick values of input vars in experiment to produce desired statistical properties

• Smallest SE of model parameters and of predictions, for a given \(n\)
• Smallest \(n\), for a given SE of model parameters & predictions

Limitations¶

• Model must be specified ahead of time
• Range of each input var must be specified ahead of time
• With multiple input vars, there can be tradeoffs b/w parameter variances

Types¶

Design		Use when	Advantage	Disadvantage		\(\text{SE}(b_1)\) For SLR
Space-Filling	Evenly spaced out \(x\) or \(y\)	check whether the model is correct	Intuitive Can verify model	Wasted opportunity cost of SE		\(\sqrt{3 \times \dfrac{n-1}{n+1}} \times \dfrac{\text{RMSE}}{\sqrt{n} \left( \dfrac{x_\max - x_\min}{2} \right)}\)
Dumbbell	- half data points at the lowest \(x\) value - half data points at the highest \(x\) value	structural model already known	Lowest SE of parameters	Cannot verify structural model		\(1 \times \dfrac{\text{RMSE}}{\sqrt{n} \left( \dfrac{x_\max - x_\min}{2} \right)}\)
Equal-Thirds	- ⅓ highest - ⅓ middle - ⅓ lowest	Compromise between space-filling and dumbbell				\(\sqrt{\dfrac{3}{2}} \times \dfrac{\text{RMSE}}{\sqrt{n} \left( \dfrac{x_\max - x_\min}{2} \right)}\)

Principles¶

1. Capacity for primary model
2. Capacity for alternate model
3. Minimum variance of estimated model parameters or predict values
4. Except for simple cases, must search for optimal design
5. Sample where the variation is
6. For non-constant variance, \(n_i \propto \sigma_{yi}^2\)
7. For curves, sample more in steep regions: Think about evenly-space \(y\) values rather than evenly-space \(x\) values
8. Repeats and replication: To compute a model-independent estimate of process standard deviation
9. Randomization and blocking
10. allows detection of drift
11. reduces influence of effect modifiers

What to optimize¶

Information matrix \(= X^T X\)

IDK¶

Comments