Benchmark Results

Local Benchmark Comparison

Local provider comparisons (stress/load + Criterion) on one host.

Local backend performance (comparison)

Results from make stress-local-compare.

Environment + machine details for interpreting results: see Benchmarking & Load Testing (Results).

All runs used:

--provider local --threads 16 --duration-s 30 --window-s 10
sample_every=100
trypema commit: b4bd2459217fe65b2f38fe423e6ff5bb5cfe8657 (chore: refactor benches)

Note: make stress-local-compare runs both trypema strategy variants (local + absolute and local + suppressed).

Hot key (enforced limit)

Workload:

--key-dist hot --rate-limit-per-s 1000
--key-dist hot: always uses a single key (user_0); key_space is ignored (effectively 1 key).

Backend	ops/s	p50 (us)	p95 (us)	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (SlidingWindowLog, absolute)	387,819	8	204	442	925	6,983	30,000	11,606,906	0	0
`governor` (GCRA, absolute)	3,883,509	1	5	25	120	15,687	40,000	116,484,081	0	0
`trypema` (local + absolute)	3,488,403	1	5	33	132	19,151	30,014	104,631,741	0	0
`trypema` (local + suppressed)	3,650,574	1	3	9	74	89,279	30,765	0	5,003	109,485,649

Uniform keys (100k keys, effectively unlimited)

Workload:

--key-dist uniform --key-space 100000 --rate-limit-per-s 1000000000
--key-dist uniform: picks a key uniformly at random from key_space keys (user_0..user_{key_space-1}).

Backend	ops/s	p50 (us)	p95 (us)	p99 (us)	p99.9 (us)	max (us)	allowed
`burster` (SlidingWindowLog, absolute)	66,825	65	923	1,527	4,607	96,255	2,009,681
`governor` (GCRA, absolute)	6,280,022	1	1	1	1	13,295	188,583,923
`trypema` (local + absolute)	9,641,276	1	1	2	215	10,023	289,569,227
`trypema` (local + suppressed)	7,609,677	1	2	42	137	6,287	228,537,450

For more comprehensive uniform key results across varying key spaces and rate limits, see Local uniform keys (matrix) below.

Results (Stress / Load harness)

These results are copied from a make stress run (the trypema-stress harness). Units:

Throughput: ops_per_s (ops/sec)
Latency: lat_us in microseconds (sampled every sample_every=100 ops)

key_dist controls how the stress test chooses rate-limiter keys (users) on each operation:

hot: always uses a single key (user_0). key_space is ignored (effectively 1 key).
uniform: picks a key uniformly at random from key_space keys (user_0..user_{key_space-1}).
skewed: with probability hot_fraction uses user_0, otherwise picks uniformly from the remaining keys (user_1..). (key_space controls how many exist.)

Target	Backend	Mode	Threads	Duration (s)	Key dist	Key space	Offered load	ops/s	p50 (us)	p95 (us)	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_denied
`make stress-local-hot`	`trypema` (local + absolute)	max	16	30	hot	100000	max	3,581,224	1	5	30	107	24,239	30,014	107,419,455	0
`make stress-local-uniform`	`trypema` (local + absolute)	max	16	60	uniform	100000	max	8,354,304	1	1	5	145	1,470	501,590,138	0	0
`make stress-local-burst`	`trypema` (local + suppressed)	target-qps	16	120	skewed	100000	`target=20k, burst=200k`	683,740	1	6	23	484	9,031	16,625,173	0	65,450,386

Local uniform keys (matrix)

These runs come from make stress-local-uniform-matrix (stress harness) and vary key_space and rate_limit_per_s while keeping the rest fixed:

--threads 16 --duration-s 30 --window-s 10 --group-ms 10 --mode max --key-dist uniform

The harness prints results in run groups of four (one per limiter variant). To keep it readable, the matrix is split by (key_space, rate_limit_per_s).

key_space=10, rate_limit_per_s=1

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,462,821	101	197	2,057	300	43,891,804	0	0
`governor` (absolute)	11,539,282	1	1	22,671	400	346,240,036	0	0
`trypema` (local + absolute)	11,444,162	1	1	19,039	309	343,337,760	0	0
`trypema` (local + suppressed)	9,693,284	1	3	26,671	1,020	0	44	290,851,987

key_space=10, rate_limit_per_s=10

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,489,697	102	220	12,311	3,000	44,699,153	0	0
`governor` (absolute)	11,565,990	1	1	17,007	4,000	347,044,913	0	0
`trypema` (local + absolute)	11,509,856	1	1	16,175	3,004	345,350,593	0	0
`trypema` (local + suppressed)	8,850,885	1	9	27,023	3,952	0	495	265,534,712

key_space=10, rate_limit_per_s=100

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,491,944	100	207	1,729	30,000	44,743,976	0	0
`governor` (absolute)	11,404,419	1	1	23,519	40,000	342,157,543	0	0
`trypema` (local + absolute)	11,387,056	1	2	20,127	30,011	341,646,817	0	0
`trypema` (local + suppressed)	8,996,526	1	8	24,687	38,953	0	4,996	269,873,751

key_space=10, rate_limit_per_s=10,000

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,478,059	101	214	2,897	3,000,000	41,352,200	0	0
`governor` (absolute)	11,374,129	1	1	15,287	4,000,010	337,289,423	0	0
`trypema` (local + absolute)	11,496,426	1	1	30,079	3,000,012	341,955,743	0	0
`trypema` (local + suppressed)	8,639,026	1	10	22,863	3,000,893	0	87,211	256,119,575

key_space=10, rate_limit_per_s=100,000

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,461,870	100	192	981	30,000,000	13,867,219	0	0
`governor` (absolute)	11,230,705	1	1	23,071	40,000,075	296,935,126	0	0
`trypema` (local + absolute)	11,122,116	1	3	23,359	30,000,013	303,687,477	0	0
`trypema` (local + suppressed)	8,639,499	1	10	35,295	30,000,793	0	547,347	228,677,471

key_space=1,000, rate_limit_per_s=1

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,469,712	73	1,199	14,583	30,007	44,074,915	0	0
`governor` (absolute)	6,261,221	1	1	20,463	39,220	187,830,595	0	0
`trypema` (local + absolute)	6,510,131	1	1	13,591	30,007	195,310,397	0	0
`trypema` (local + suppressed)	6,254,484	1	52	16,863	34,583	0	3,999	187,616,542

key_space=1,000, rate_limit_per_s=10

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,480,674	71	1,133	19,935	300,008	44,132,533	0	0
`governor` (absolute)	11,380,136	1	1	21,519	399,199	341,077,110	0	0
`trypema` (local + absolute)	11,923,162	1	1	20,719	300,013	357,471,864	0	0
`trypema` (local + suppressed)	9,637,790	1	26	11,999	319,895	0	49,000	288,779,483

key_space=1,000, rate_limit_per_s=100

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,456,029	81	1,159	16,639	3,000,008	40,695,894	0	0
`governor` (absolute)	6,259,911	1	1	14,311	3,999,244	183,826,239	0	0
`trypema` (local + absolute)	11,931,078	1	1	24,895	3,000,019	355,000,796	0	0
`trypema` (local + suppressed)	6,269,134	1	69	11,095	2,999,759	0	62,035	185,026,236

key_space=1,000, rate_limit_per_s=10,000

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,520,512	71	1,066	16,343	45,631,249	0	0	0
`governor` (absolute)	6,258,429	1	1	10,087	187,771,834	0	0	0
`trypema` (local + absolute)	11,175,804	1	1	11,743	300,000,018	35,324,813	0	0
`trypema` (local + suppressed)	10,165,153	1	13	15,063	213,629,662	0	89,715,697	1,671,019

key_space=1,000, rate_limit_per_s=100,000

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed
`burster` (absolute)	1,436,225	73	1,167	31,679	43,094,650
`governor` (absolute)	6,306,821	1	1	19,023	189,232,299
`trypema` (local + absolute)	6,411,487	1	6	21,119	192,376,662
`trypema` (local + suppressed)	10,078,774	1	13	14,207	302,399,696

key_space=10,000, rate_limit_per_s=1

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	1,000,018	34	2,109	19,903	300,020	29,714,952	0	0
`governor` (absolute)	11,508,773	1	1	34,879	390,007	344,961,238	0	0
`trypema` (local + absolute)	11,832,581	1	1	21,375	300,020	354,794,617	0	0
`trypema` (local + suppressed)	8,984,415	2	224	8,295	277,835	0	39,903	269,296,789

key_space=10,000, rate_limit_per_s=10

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	992,127	36	2,107	23,327	3,000,019	26,775,548	0	0
`governor` (absolute)	6,362,268	1	1	24,191	3,990,008	186,927,711	0	0
`trypema` (local + absolute)	6,509,860	1	1	26,191	3,000,026	192,360,589	0	0
`trypema` (local + suppressed)	8,689,741	2	206	3,505	2,973,493	0	83,118	257,693,351

key_space=10,000, rate_limit_per_s=100

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed	rejected	suppressed_allowed	suppressed_denied
`burster` (absolute)	997,518	33	1,902	17,295	28,348,447	1,588,838	0	0
`governor` (absolute)	11,452,847	1	1	30,175	39,989,991	303,689,537	0	0
`trypema` (local + absolute)	11,784,103	1	1	23,839	30,000,028	323,564,650	0	0
`trypema` (local + suppressed)	8,786,166	2	258	8,375	29,970,036	0	563,427	233,103,367

key_space=10,000, rate_limit_per_s=10,000

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed
`burster` (absolute)	1,053,736	23	2,044	36,191	31,621,672
`governor` (absolute)	11,319,092	1	1	10,527	339,629,109
`trypema` (local + absolute)	11,152,981	1	12	8,391	334,715,739
`trypema` (local + suppressed)	9,949,948	1	206	9,703	298,589,748

key_space=10,000, rate_limit_per_s=100,000

Limiter	ops/s	p99 (us)	p99.9 (us)	max (us)	allowed
`burster` (absolute)	997,546	36	2,369	22,351	29,935,825
`governor` (absolute)	11,123,170	1	1	23,775	333,740,832
`trypema` (local + absolute)	10,987,323	1	17	9,591	329,685,281
`trypema` (local + suppressed)	9,600,009	1	217	6,875	288,083,537

Benchmark Results

Recorded Criterion and stress/load results (local + Redis + Hybrid).

Redis + Hybrid Benchmark Comparison

Redis + Hybrid provider comparisons (stress/load + Criterion) under shared client settings.