> ## Documentation Index
> Fetch the complete documentation index at: https://sourcetable.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Feature engineering

> Transform raw data into model-ready features with encoding, scaling, and automated creation.

Feature engineering turns raw data into inputs that improve model performance. Sourcetable automates common transformations through the AI assistant.

## Missing value handling

```
"Handle missing values in the dataset — use median for numeric and mode for categorical"
```

| Strategy | When to use |
| - | - |
| Mean/median imputation | Numeric columns, few missing values |
| Mode imputation | Categorical columns |
| Forward/backward fill | Time series data |
| KNN imputation | Values depend on other features |
| Drop rows | Small percentage missing, large dataset |
| Indicator column | Missingness itself is informative |

## Encoding categorical variables

```
"Encode all categorical columns for the ML model"
```

| Method | When to use |
| - | - |
| One-hot encoding | Nominal categories (color, city) with low cardinality |
| Label encoding | Ordinal categories (low/medium/high) |
| Target encoding | High-cardinality categories (zip code, product ID) |
| Binary encoding | High-cardinality with many categories |
| Frequency encoding | When category frequency matters |

## Scaling and normalization

```
"Normalize all numeric features to 0-1 range"
```

| Method | When to use |
| - | - |
| StandardScaler (Z-score) | Most ML algorithms, normally distributed data |
| MinMaxScaler (0-1) | Neural networks, distance-based algorithms |
| RobustScaler | Data with outliers |
| Log transform | Right-skewed distributions (income, prices) |
| Box-Cox transform | Make data more normal, various skew levels |

## Feature creation

### Date/time features

```
"Extract year, month, day of week, and hour from the timestamp column"
```

Creates: year, quarter, month, week, day of week, day of month, hour, minute, is\_weekend, is\_holiday, days\_since\_start.

### Aggregation features

```
"Create customer-level features: total purchases, average order value, days since last order"
```

Group-by aggregations: sum, mean, median, count, min, max, std, first, last.

### Interaction features

```
"Create interaction features between price and quantity"
```

Products, ratios, and polynomial features between numeric columns.

### Text features

```
"Extract features from the product description column"
```

Creates: word count, character count, average word length, sentiment score, TF-IDF vectors.

## Feature selection

### Importance-based

```
"Rank features by importance using a random forest"
```

Trains a model and ranks features by their contribution to predictions.

### Variance Inflation Factor (VIF)

```
"Check for multicollinearity and drop highly correlated features"
```

Calculates VIF for each feature. Values above 5-10 indicate problematic multicollinearity.

### Recursive Feature Elimination (RFE)

```
"Use RFE to find the optimal set of features for the model"
```

Iteratively removes the least important feature until performance stops improving.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.