Maintaining fallback values

Fallback values are critical to application resilience. They ensure your application continues functioning when LaunchDarkly's service is unavailable or flag data cannot be retrieved. However, fallback values can become stale over time, leading to incorrect behavior during outages. This guide explains how to choose appropriate fallback values and maintain them effectively.

Choosing Fallback Values

The fallback value you choose depends on the risk and impact of the feature being unavailable. Consider these strategies:

Failing Closed

Definition: Turn off the feature when flag data is unavailable.

When to use:

• New features that haven't been fully validated
• Features that have not been released to all users
• Features that could introduce significant load or stability issues if released to everyone at once

Example:

// New checkout flow - fail closed if flag unavailable
const useNewCheckout = ldClient.variation('new-checkout-flow', false);
if (useNewCheckout) {
  return <NewCheckoutComponent />;
}
return <LegacyCheckoutComponent />;

Failing Open

Definition: Enable the feature for everyone when flag data is unavailable.

When to use:

• Features that have been generally available, also known as GA, for a while and are stable
• Circuit breakers/operational flags where the on state is the norm
• Features that would have significant impact if disabled for everyone at once

Example:

// Enable caching when flag is unavailable. Failing closed would cause a significant performance degradation.
const enableCache = ldClient.variation('enable-caching', true);

For temporary flags that we intend to remove, consider cleaning up and archiving the flag instead of updating the fallback value to true.

Dynamic Fallback Values

Definition: Implement logic to provide different fallback values to different users based on context.

When to use:

• Configuration/operational flags that override values from another source (environment variables, configuration, etc.)
• Advanced scenarios requiring sophisticated fallback logic

Example:

function getRateLimit(request: RequestContext): number {
   // dynamic rate limit based on the request method
   return ldclient.variation('config-rate-limit', request, request.method === 'GET' ? 100 : 10)
}

Methods for Maintaining Fallback Values

Fallback values can become stale as flags evolve. Use these methods to ensure fallback values remain accurate and up-to-date:

Create a Formal Process

Establish a formal process for defining and updating fallback values:

Process steps:

1. Define fallback values at flag creation - Require fallback values when creating flags
2. Document fallback strategy - Document why each fallback value was chosen (failing closed vs. failing open)
3. Review fallback values during flag lifecycle - Review fallback values when:
   • Flags are promoted from development to production
   • Flags are modified or targeting rules change
   • Flags are deprecated or removed
4. Update fallback values as flags mature - Update fallback values when flags become GA or stable
5. Test fallback values - Include fallback value testing in your testing strategy

Documentation template:

Flag: new-checkout-flow
Fallback value: false - failing closed
Rationale: New feature not yet validated in production. Safer to use legacy checkout during outages.
Review date: 2024-01-15
Next review: When flag reaches 50% rollout

Automated Reports on Fallback Values

Use automated reporting to identify stale or incorrect fallback values:

Runtime Monitoring (Data Export or SDK Hooks)

Monitor fallback value usage in real-time:

Data Export:

• Export evaluation events to your data warehouse
• Feature events expose the fallback value as the default property on the event
• Compare fallback values to current flag state and desired behavior

SDK Hooks:

• Implement a before evaluation hook
• Record the fallback value for each evaluation in a telemetry system
• Compare fallback values to current flag state and desired behavior

Example SDK Hook:

class FallbackMonitoringHook implements Hook {
  beforeEvaluation(seriesContext: EvaluationSeriesContext, data: EvaluationSeriesData) {
    // Always log the fallback value being used
    this.logFallbackValue(
      seriesContext.flagKey,
      seriesContext.defaultValue,
    );
    return data;
  }
}

API-Based Reporting

Use the LaunchDarkly API to generate reports on fallback values:

Example: This fallback-report script demonstrates how to:

• Retrieve flag definitions from the LaunchDarkly API
• Compare flag fallthrough/off variations with fallback values in code
• Generate reports identifying mismatches

Use cases:

• Scheduled reports comparing flag definitions with code fallback values
• CI/CD integration to detect fallback value mismatches
• Periodic audits of fallback value accuracy

Note: This approach relies on telemetry from SDKs generated when variation/variationDetail are called. The API only reports one fallback value and cannot reliably handle situations where different fallback values are used for different users or applications.

Static Analysis and AI Tools

Use static analysis or AI tools to analyze fallback values:

Static analysis:

• Scan codebases for variation() calls
• Extract fallback values from source code
• Compare with flag definitions

AI tools:

• Use AI to analyze code and suggest fallback value updates. You can find an example prompt in the LaunchDarkly Labs Agent Prompts repository.
• Identify patterns in fallback value usage
• Generate recommendations based on flag lifecycle stage
• Use ldcli or the LaunchDarkly MCP to enable the agent to compare fallback values to flag definitions

Centralize Fallback Management

Centralize fallback value management to ensure consistency and simplify updates:

Wrapper Functions

Create wrapper functions around variation() and variationDetail() that load fallback values from a centralized configuration:

Example:

// fallback-config.json
{
  "new-checkout-flow": false,
  "cache-optimization-v2": true,
  "experimental-feature": false
}

// wrapper.ts
import fallbackConfig from './fallback-config.json';

export function variationWithFallback(
  client: LDClient,
  flagKey: string,
  context: LDContext
): LDEvaluationDetail {
  let fallbackValue = fallbackConfig[flagKey];
  if (fallbackValue === undefined) {
    
    // you may want to make this an error in preproduction environments to catch missing fallback values early
    console.warn(`No fallback value defined for flag: ${flagKey}`);
    // you can use naming convention or other logic to determine a default fallback value
    // for example, release flags may default to failing closed 
    if (flagKey.startsWith('release-')) {
      fallbackValue = false;
    }
  }
  return client.variationDetail(flagKey, context, fallbackValue);
}

Benefits:

• Single source of truth for fallback values
• Easier to automate fallback value updates
• Logic can be shared across applications and services

Tradeoffs:

• Difficult to implement dynamic fallback values (e.g., different fallback values for different users or applications)
• Loss of locality: fallback values are no longer present in the variation call and require checking the fallback definition file

Automatic Fallback Updates During Build

Automatically update fallback values during your build process using flag fallthrough or off variations:

Process:

1. During build, query LaunchDarkly API for flag definitions
2. Extract fallthrough or off variations for each flag
3. Update fallback configuration files with current values
4. Validate that fallback values match expected types

Example build script:

#!/bin/bash
# Update fallback values from LaunchDarkly flags
curl -H "Authorization: ${LD_API_KEY}" \
  "https://app.launchdarkly.com/api/v2/flags/${PROJECT_KEY}" \
  | jq '.items[] | {key: .key, fallback: .environments.production.fallthrough.variation}' \
  > fallback-config.json

Benefits:

• Ensures fallback values match current flag definitions
• Reduces manual maintenance overhead
• Catches drift between code and flag definitions
• Supports automated flag lifecycle management

Considerations:

• Additional logic may be required to determine which value to use. For example, if the flag is off, you may use the off variation
• Ensure your team is aware of how fallback values are generated from targeting state