Model Interpretation¶
Association \(\ne\) Causation
Classification of Inference Techniques¶
- IDK
- Model-Specific
- Model-Agnostic
- Scope
- Global: Explanation for entire dataset
- Local: Explanation for single data point
Inference Techniques¶
| IDK | Scope | ||
|---|---|---|---|
| Simple Linear Regression \(y = \beta_0 + \beta_j x_j + \beta_\text{ind} G + \beta_\text{int} x_j G\) | Model-Specific | Global | \(\beta_0\) is the baseline value of \(y\) when \(x_j=0\) \(\beta_j\) is the change in \(y\) for every unit increase in \(x_j\) \(\beta_\text{ind}\) is the change in baseline for group \(G\), ie baseline will now be \((\beta_0 + \beta_\text{ind})\) \(\beta_\text{int}\) is the additional change in \(y\) in group \(G\), ie for every unit increase in \(x_j\), \(y\) changes by \((\beta_j + \beta_\text{int})\) units for group \(G\) |
| \(\ln \vert y \vert = \beta_0 + \beta_j x_j\) | Model-Specific | Global | For every unit increase in \(x_j\), percentage change in \(y\) is \(\beta_j\) units |
| \(\ln \vert y \vert = \beta_0 + \beta_j \ln \vert x_j \vert\) | Model-Specific | Global | Elasticity of \(y\) wrt \(x_j\) is given by \(\beta_j\) \(\beta_j = \dfrac{\% \Delta y}{\% \Delta x_j}\) |
| SAGE | Model-Agnostic | Global | |
| Variable/Feature Importance | Model-Agnostic | Global | Decrease of in-sample error due to splits over \(x\), averaged over all trees of ensemble |
| Partial Dependence | Model-Agnostic | Global | Partial derivative of \(y\) wrt \(x\): Marginal effect of \(x\) on \(y\) after integrating out all other vars |
| SHAP | Model-Agnostic | Local | |
| LIME | Model-Agnostic | Local |