Hybrid Provider

The hybrid provider combines the low latency of in-process state with the distributed consistency of Redis. It maintains a local counter per key and periodically flushes accumulated increments to Redis in batches via a background actor (the RedisCommitter). Between flushes, admission decisions are served from local state without any Redis I/O.

Access: rl.hybrid()

When to use

• High-throughput distributed APIs where per-request Redis round-trips are too expensive
• When you can tolerate up to sync_interval_ms of lag between local state and Redis
• When you want sub-microsecond admission latency on the fast-path

If you need decisions that always reflect the latest Redis state, use the Redis Provider.

Requirements

• Redis >= 7.2
• One async runtime feature: redis-tokio or redis-smol

Setup

The setup is identical to the Redis provider. The only difference is calling rl.hybrid() instead of rl.redis():

use std::sync::Arc;

use trypema::{
    HardLimitFactor, RateGroupSizeMs, RateLimit, RateLimitDecision,
    RateLimiter, RateLimiterOptions, SuppressionFactorCacheMs, WindowSizeSeconds,
};
use trypema::hybrid::SyncIntervalMs;
use trypema::local::LocalRateLimiterOptions;
use trypema::redis::{RedisKey, RedisRateLimiterOptions};

#[tokio::main]
async fn main() -> Result<(), trypema::TrypemaError> {
    let client = redis::Client::open("redis://127.0.0.1:6379/").unwrap();
    let connection_manager = client.get_connection_manager().await.unwrap();

    let rl = Arc::new(RateLimiter::new(RateLimiterOptions {
        local: LocalRateLimiterOptions {
            window_size_seconds: WindowSizeSeconds::try_from(60).unwrap(),
            rate_group_size_ms: RateGroupSizeMs::try_from(10).unwrap(),
            hard_limit_factor: HardLimitFactor::try_from(1.5).unwrap(),
            suppression_factor_cache_ms: SuppressionFactorCacheMs::default(),
        },
        redis: RedisRateLimiterOptions {
            connection_manager,
            prefix: None,
            window_size_seconds: WindowSizeSeconds::try_from(60).unwrap(),
            rate_group_size_ms: RateGroupSizeMs::try_from(10).unwrap(),
            hard_limit_factor: HardLimitFactor::try_from(1.5).unwrap(),
            suppression_factor_cache_ms: SuppressionFactorCacheMs::default(),
            sync_interval_ms: SyncIntervalMs::try_from(10).unwrap(),
        },
    }));

    rl.run_cleanup_loop();

    let key = RedisKey::try_from("user_123".to_string())?;
    let rate = RateLimit::try_from(10.0)?;

    // Absolute: served from local state
    let decision = rl.hybrid().absolute().inc(&key, &rate, 1).await?;

    // Suppressed: probabilistic admission from local state
    let decision = rl.hybrid().suppressed().inc(&key, &rate, 1).await?;

    // Query suppression factor
    let factor = rl.hybrid().suppressed().get_suppression_factor(&key).await?;

    Ok(())
}

How it works

1. First call for a key: The hybrid provider reads from Redis to initialise local state.
2. Subsequent calls (fast-path): Admission decisions are made from local counters. No Redis I/O.
3. Background sync: The RedisCommitter periodically (every sync_interval_ms) batches local increments and flushes them to Redis, then reads back the updated Redis state to refresh the local view.
4. State exhaustion: When local state indicates the key is at/over capacity, the provider reads from Redis to refresh and may transition to a different state.

State machine

Each key transitions through internal states. Understanding these helps reason about the hybrid provider's behaviour.

Absolute strategy

Suppressed strategy

Thundering herd prevention

Each key has its own tokio::sync::Mutex. When local state is exhausted and a Redis read is needed, only one task performs the read; others wait on the mutex and then re-check the (now-refreshed) state. This prevents N concurrent tasks from all hitting Redis simultaneously for the same key.

The sync_interval_ms option

sync_interval_ms controls how often the background actor flushes local increments to Redis:

Default: 10ms. Validated newtype: SyncIntervalMs::try_from(value) fails if value == 0.

A good rule of thumb: sync_interval_ms <= rate_group_size_ms.

Hybrid vs Redis: trade-offs

Key constraints

Hybrid uses RedisKey with the same validation rules as the Redis provider: non-empty, <= 255 bytes, no :. See Redis Provider for details.

Next steps

• Redis Provider -- the pure Redis alternative
• Local Provider -- in-process only
• Configuration & Tuning -- tune sync_interval_ms and other options