optimize-shiny-performance

# Optimize Shiny Performance

Profile, diagnose, and optimize Shiny application performance through caching, async operations, and reactive graph optimization.

## When to Use

- Shiny app feels slow or unresponsive during user interaction
- Server resources are exhausted under concurrent user load
- Specific operations (data loading, plotting, computation) create bottlenecks
- Preparing an app for production deployment with many users

## Inputs

- **Required**: Path to the Shiny application
- **Required**: Description of the performance problem (slow load, laggy interaction, high memory)
- **Optional**: Number of expected concurrent users
- **Optional**: Available server resources (RAM, CPU cores)
- **Optional**: Whether the app uses a database or external API

## Procedure

### Step 1: Profile the Application

```r
# Profile with profvis
profvis::profvis({
  shiny::runApp("path/to/app", display.mode = "normal")
})

# Or profile specific operations
profvis::profvis({
  result <- expensive_computation(data)
})
```

Identify the top bottlenecks:
1. **Data loading**: How long does initial data fetch take?
2. **Reactive recalculation**: Which reactives fire most often?
3. **Rendering**: Which outputs take the longest to render?
4. **External calls**: Database queries, API requests, file I/O?

Use the reactive log for reactive graph analysis:

```r
# Enable reactive logging
options(shiny.reactlog = TRUE)
shiny::runApp("path/to/app")
# Press Ctrl+F3 in the browser to view the reactive graph
```

**Got:** Clear identification of the 2-3 biggest bottlenecks.

**If fail:** If profvis does not show useful detail, wrap specific sections with `profvis::profvis()`. If reactlog is overwhelming, focus on one interaction at a time.

### Step 2: Optimize Reactive Graph

Reduce unnecessary reactive invalidations:

```r
# BAD: Recomputes on ANY input change
output$plot <- renderPlot({
  data <- load_data()  # Runs every time
  filtered <- data[data$category == input$category, ]
  plot(filtered)
})

# GOOD: Isolate data loading from filtering
raw_data <- reactive({
  load_data()
}) |> bindCache()  # Cache the expensive part

filtered_data <- reactive({
  raw_data()[raw_data()$category == input$category, ]
})

output$plot <- renderPlot({
  plot(filtered_data())
})
```

Use `isolate()` to prevent unnecessary invalidations:

```r
# Only recompute when the button is clicked, not on every input change
output$result <- renderText({
  input$compute  # Take dependency on button
  isolate({
    paste("N =", input$n, "Mean =", mean(rnorm(input$n)))
  })
})
```

Use `debounce()` and `throttle()` for high-frequency inputs:

```r
# Debounce text input — wait 500ms after user stops typing
search_text <- reactive(input$search) |> debounce(500)

# Throttle slider — update at most every 250ms
slider_value <- reactive(input$slider) |> throttle(250)
```

**Got:** Reactive graph fires only necessary recalculations.

**If fail:** If removing a dependency breaks functionality, use `req()` to add explicit guards instead of relying on implicit reactive dependencies.

### Step 3: Implement Caching

#### bindCache for Shiny Outputs

```r
output$plot <- renderPlot({
  create_expensive_plot(filtered_data())
}) |> bindCache(input$category, input$date_range)

output$table <- renderDT({
  expensive_query(input$filters)
}) |> bindCache(input$filters)
```

`bindCache` uses input values as cache keys. When the same inputs occur again, the cached result is returned immediately.

#### memoise for Functions

```r
# Cache expensive function results
load_reference_data <- memoise::memoise(
  function(dataset_name) {
    readr::read_csv(paste0("data/", dataset_name, ".csv"))
  },
  cache = cachem::cache_disk("cache/", max_age = 3600)
)
```

#### App-level Data Pre-computation

```r
# In global.R or outside server function — computed once at app startup
reference_data <- readr::read_csv("data/reference.csv")
model <- readRDS("models/trained_model.rds")

server <- function(input, output, session) {
  # reference_data and model are available to all sessions
  # without reloading
}
```

**Got:** Repeated operations use cached results; response time drops significantly.

**If fail:** If cache grows too large, set `max_age` or `max_size` limits. If cached values are stale, reduce `max_age` or add a cache-clear button. If `bindCache` causes errors, ensure cache key inputs are serializable.

### Step 4: Add Async for Long Operations

Use `ExtendedTask` (Shiny >= 1.8.1) for long-running computations:

```r
server <- function(input, output, session) {
  # Define the extended task
  analysis_task <- ExtendedTask$new(function(data, params) {
    promises::future_promise({
      # This runs in a background process
      run_heavy_analysis(data, params)
    })
  }) |> bind_task_button("run_analysis")

  # Trigger the task
  observeEvent(input$run_analysis, {
    analysis_task$invoke(dataset(), input$params)
  })

  # Use the result
  output$result <- renderTable({
    analysis_task$result()
  })
}
```

For apps on Shiny < 1.8.1, use promises directly:

```r
library(promises)
library(future)
plan(multisession, workers = 4)

server <- function(input, output, session) {
  result <- eventReactive(input$compute, {
    future_promise({
      Sys.sleep(5)  # Simulate long computation
      expensive_analysis(isolate(input$params))
    })
  })

  output$table <- renderTable({
    result()
  })
}
```

**Got:** Long operations do not block the UI; other users can interact while computation runs.

**If fail:** If `future_promise` errors, check that `plan(multisession)` is set. If variables are not available in the future, pass them explicitly — futures run in separate R processes.

### Step 5: Optimize Rendering

Reduce rendering overhead:

```r
# Use plotly for interactive plots instead of re-rendering
output$plot <- plotly::renderPlotly({
  plotly::plot_ly(filtered_data(), x = ~x, y = ~y, type = "scatter")
})

# Use server-side DT for large tables
output$table <- DT::renderDataTable({
  DT::datatable(large_data(), server = TRUE, options = list(
    pageLength = 25,
    processing = TRUE
  ))
})

# Conditional UI to avoid rendering hidden elements
output$details <- renderUI({
  req(input$show_details)
  expensive_details_ui()
})
```

**Got:** Rendering operations are faster and do not block the UI.

**If fail:** If plotly is slow with large datasets, use `toWebGL()` for WebGL rendering or downsample data before plotting.

### Step 6: Validate Performance Improvements

```r
# Before/after benchmarking
system.time({
  shiny::testServer(myModuleServer, args = list(...), {
    session$setInputs(category = "A")
    session$flushReact()
  })
})

# Load testing with shinyloadtest
shinyloadtest::record_session("http://localhost:3838")
shinyloadtest::shinycannon(
  "recording.log",
  "http://localhost:3838",
  workers = 10,
  loaded_duration_minutes = 5
)
shinyloadtest::shinyloadtest_report("recording.log")
```

**Got:** Measurable improvement in response times and/or concurrent user capacity.

**If fail:** If performance did not improve, re-profile to find the next bottleneck. Performance optimization is iterative — fix the biggest bottleneck first, then re-measure.

## Validation

- [ ] Profiling identifies specific bottlenecks (not guessing)
- [ ] Reactive graph has no unnecessary invalidation chains
- [ ] Expensive operations use caching (bindCache or memoise)
- [ ] Long-running computations use async (ExtendedTask or promises)
- [ ] High-frequency inputs use debounce/throttle
- [ ] Large datasets use server-side processing
- [ ] Performance improvement is measurable (before/after timing)

## Pitfalls

- **Premature optimization**: Profile first. The bottleneck is rarely where you think it is.
- **Cache invalidation bugs**: If users see stale data, the cache key does not include all relevant inputs. Add missing dependencies to `bindCache()`.
- **Future variable scoping**: `future_promise` runs in a separate process. Global variables, database connections, and reactive values must be captured explicitly.
- **Reactive spaghetti**: If the reactive graph is too complex to understand, the app needs architectural refactoring (modules), not just caching.
- **Over-caching**: Caching everything wastes memory. Only cache operations that are expensive AND have repeated input patterns.

## Related Skills

- `build-shiny-module` — modular architecture for maintainable reactive code
- `scaffold-shiny-app` — choose the right app framework from the start
- `deploy-shiny-app` — deploy optimized apps with appropriate server resources
- `test-shiny-app` — performance regression tests