23 Experiments

Run tracking, metrics, and tuning integration

Keywords

snowflake, R, RStudio, Posit, VS Code, workspace notebooks, snowflakeR, RSnowflake, mlops

23.1 Overview

Experiment tracking on Snowflake captures training exploration — parameters, metrics, artifacts, and optionally logged models — alongside the governed Model Registry used for deployment.

snowflakeR exposes experiments via a small R API on top of snowflake-ml-python. Use experiments for hyperparameter search and model comparison; promote winners to the Registry with sfr_log_model().

Requires a recent snowflake-ml-python build (see ?sfr_experiment for version checks in your session).

23.2 Learning Objectives

Create experiments and runs from R
Log parameters, metrics, and artifacts
Integrate with tidymodels tune
List and download artifacts for review

23.3 Basic workflow

library(snowflakeR)
conn <- sfr_connect()

exp <- sfr_experiment(conn, name = "FORECAST_TUNING")

sfr_start_run(exp, name = "arima_001")

sfr_exp_log_params(exp, p = 2L, d = 1L, q = 1L)
sfr_exp_log_metric(exp, "mape", 0.12, step = 1L)
sfr_exp_log_metric(exp, "rmse", 1450.3, step = 1L)

# Optional: log files or interim models
# sfr_exp_log_artifact(exp, "/tmp/forecast_plot.png", "plots/forecast.png")
# sfr_exp_log_model(exp, fitted_workflow, ...)

sfr_end_run(exp)

Each run appears in Snowflake ML experiment UI alongside Python-logged runs — same experiment namespace.

23.4 Multiple runs (grid search pattern)

exp <- sfr_experiment(conn, name = "XGB_GRID")

for (depth in c(3, 6, 9)) {
  sfr_start_run(exp, name = sprintf("depth_%d", depth))
  sfr_exp_log_params(exp, max_depth = depth, eta = 0.1)
  # ... train ...
  sfr_exp_log_metric(exp, "auc", auc_value)
  sfr_end_run(exp)
}

For systematic resampling grids, prefer tune integration below.

23.5 tidymodels `tune` integration

Attach an experiment to a tuning workflow:

exp <- sfr_experiment(conn, "MY_EXPERIMENT")

# After defining workflow + resamples:
# sfr_experiment_from_tune(exp, <tune_workflow_or_last_fit_object>)

# ... tune::tune_grid() or fit_resamples() ...

# sfr_experiment_log_best(exp, <tune_results>)

Exact signatures depend on your tune version — see ?sfr_experiment_from_tune and ?sfr_experiment_log_best.

Typical flow:

Define tidymodels workflow with tunable parameters.
sfr_experiment_from_tune() links the Snowflake experiment to the workflow.
Run tuning; metrics log per candidate.
sfr_experiment_log_best() records the winning configuration.
Register final model via Model Registry.

23.6 Artifacts

sfr_exp_list_artifacts(exp, run_name = "arima_001")

sfr_exp_download_artifact(exp,
  run_name       = "arima_001",
  artifact_path  = "plots/forecast.png",
  target_path    = "forecast.png"
)

Use artifacts for diagnostic plots, serialized preprocessors, or audit trails — not for production model binaries (use Registry).

23.7 Cleanup

sfr_delete_run(exp, run_name = "arima_001")
sfr_delete_experiment(exp, name = "FORECAST_TUNING")

23.8 Experiments vs Model Registry

	Experiments	Model Registry
Purpose	Exploration, comparison	Versioned deployment
Lifecycle	Many ephemeral runs	Promoted model versions
Serving	No	Yes (SPCS / warehouse)
R API	`sfr_experiment()`, `sfr_exp_*`	`sfr_log_model()`, `sfr_deploy_*`

23.9 Companion

Vignette: experiments
Related: Model Registry, MLOps on Snowflake

23.10 Next steps

Model Registry — promote a winning run to a versioned model.

Parallel doSnowflake — when training must scale beyond one notebook.

--- title: "Experiments" subtitle: "Run tracking, metrics, and tuning integration" --- ## Overview **Experiment tracking** on Snowflake captures training exploration — parameters, metrics, artifacts, and optionally logged models — alongside the governed **Model Registry** used for deployment. snowflakeR exposes experiments via a small R API on top of `snowflake-ml-python`. Use experiments for hyperparameter search and model comparison; promote winners to the Registry with `sfr_log_model()`. Requires a recent **`snowflake-ml-python`** build (see `?sfr_experiment` for version checks in your session). ## Learning Objectives - Create experiments and runs from R - Log parameters, metrics, and artifacts - Integrate with tidymodels **`tune`** - List and download artifacts for review --- ## Basic workflow {#sec-basic} ```r library(snowflakeR) conn <- sfr_connect() exp <- sfr_experiment(conn, name = "FORECAST_TUNING") sfr_start_run(exp, name = "arima_001") sfr_exp_log_params(exp, p = 2L, d = 1L, q = 1L) sfr_exp_log_metric(exp, "mape", 0.12, step = 1L) sfr_exp_log_metric(exp, "rmse", 1450.3, step = 1L) # Optional: log files or interim models # sfr_exp_log_artifact(exp, "/tmp/forecast_plot.png", "plots/forecast.png") # sfr_exp_log_model(exp, fitted_workflow, ...) sfr_end_run(exp) ``` Each run appears in Snowflake ML experiment UI alongside Python-logged runs — same experiment namespace. --- ## Multiple runs (grid search pattern) {#sec-grid} ```r exp <- sfr_experiment(conn, name = "XGB_GRID") for (depth in c(3, 6, 9)) { sfr_start_run(exp, name = sprintf("depth_%d", depth)) sfr_exp_log_params(exp, max_depth = depth, eta = 0.1) # ... train ... sfr_exp_log_metric(exp, "auc", auc_value) sfr_end_run(exp) } ``` For systematic resampling grids, prefer **`tune`** integration below. --- ## tidymodels `tune` integration {#sec-tune} Attach an experiment to a tuning workflow: ```r exp <- sfr_experiment(conn, "MY_EXPERIMENT") # After defining workflow + resamples: # sfr_experiment_from_tune(exp, <tune_workflow_or_last_fit_object>) # ... tune::tune_grid() or fit_resamples() ... # sfr_experiment_log_best(exp, <tune_results>) ``` Exact signatures depend on your `tune` version — see `?sfr_experiment_from_tune` and `?sfr_experiment_log_best`. Typical flow: 1. Define tidymodels workflow with tunable parameters. 2. `sfr_experiment_from_tune()` links the Snowflake experiment to the workflow. 3. Run tuning; metrics log per candidate. 4. `sfr_experiment_log_best()` records the winning configuration. 5. Register final model via [Model Registry](../18_model_registry/index.qmd). --- ## Artifacts {#sec-artifacts} ```r sfr_exp_list_artifacts(exp, run_name = "arima_001") sfr_exp_download_artifact(exp, run_name = "arima_001", artifact_path = "plots/forecast.png", target_path = "forecast.png" ) ``` Use artifacts for diagnostic plots, serialized preprocessors, or audit trails — not for production model binaries (use Registry). --- ## Cleanup {#sec-cleanup} ```r sfr_delete_run(exp, run_name = "arima_001") sfr_delete_experiment(exp, name = "FORECAST_TUNING") ``` --- ## Experiments vs Model Registry {#sec-vs-registry} | | Experiments | Model Registry | |--|-------------|----------------| | Purpose | Exploration, comparison | Versioned deployment | | Lifecycle | Many ephemeral runs | Promoted model versions | | Serving | No | Yes (SPCS / warehouse) | | R API | `sfr_experiment()`, `sfr_exp_*` | `sfr_log_model()`, `sfr_deploy_*` | --- ## Companion - Vignette: `experiments` - Related: [Model Registry](../18_model_registry/index.qmd), [MLOps on Snowflake](../15_mlops_on_snowflake/index.qmd) --- ## Next steps [Model Registry](../18_model_registry/index.qmd) — promote a winning run to a versioned model. [Parallel doSnowflake](../22_parallel_dosnowflake/index.qmd) — when training must scale beyond one notebook.