Biscuit authentication intermittently returns Unauthorized for valid tokens under modest RPC load

[sunchengzhu]

Symptom: valid tokens are misclassified as Unauthorized

Summary

With a valid Biscuit token, FNN intermittently returns:

RPC error (code -32999): Unauthorized

The corresponding server logs show the actual cause:

authorize failed: Reached Datalog execution limits

The same token succeeds for other requests, so this is not a persistent signature, expiration, or permission issue.

Environment and result

• FNN: v0.9.0-rc7
• Load: 5 complete transactions per second for 5 minutes, 1500 total
• Result: 1497 succeeded, 3 failed (0.2% failure rate)
• Failed job: https://github.com/sunchengzhu/cch-daily-smoke/actions/runs/29346453027/job/87131324455

Sequence	Node	RPC
441	node2	get_payment
455	node1	send_btc
630	node1	get_cch_order

Key logs

# CI
15:43:24 ERROR FAILED seq=441 ... Unauthorized
15:43:24 ERROR FAILED seq=455 ... Unauthorized
15:44:01 ERROR FAILED seq=630 ... Unauthorized
15:46:55 INFO RESULT FAIL succeeded=1497/1500 failed=3 failure_rate=0.200%

# node2: seq 441
2026-07-14T15:43:24.631334Z DEBUG fnn::rpc::biscuit:
authorize failed: Reached Datalog execution limits

# node1: seq 455
2026-07-14T15:43:24.647755Z DEBUG fnn::rpc::biscuit:
authorize failed: Reached Datalog execution limits

# node1: seq 630
2026-07-14T15:44:01.506089Z DEBUG fnn::rpc::biscuit:
authorize failed: Reached Datalog execution limits

The three server-side errors match the three CI failures exactly.

Source analysis

Fiber calls Biscuit's default authorize() without setting custom limits:

self.build_rule()?
    .fact(...)?
    .build(token)?
    .authorize()?;

Source: crates/fiber-lib/src/rpc/biscuit.rs

The resolved biscuit-auth 6.0.0 defaults are:

max_facts: 1000
max_iterations: 100
max_time: 1ms

The authorization rules are simple read(...)/write(...) checks, and the same token succeeds thousands of times. The most likely cause is the default 1 ms time limit under load. The current log does not expose whether the exact subtype is Timeout, TooManyFacts, or TooManyIterations.

The middleware also maps every authorization failure to -32999 Unauthorized, so clients cannot distinguish invalid credentials from a temporary execution limit.

Suggestions

1. Make Biscuit limits configurable and consider increasing the default max_time from 1 ms to 10 ms.
2. Log the exact limit subtype and RPC method.
3. Return a distinct, retryable RPC error for Datalog execution limits instead of Unauthorized.

---

中文版

现象：有效 Token 被误判为 Unauthorized

概述

使用有效 Biscuit Token 运行稳定性测试时，FNN 偶发返回：

RPC error (code -32999): Unauthorized

但相同时间的服务端日志显示真实原因是：

authorize failed: Reached Datalog execution limits

相同 Token 的其他请求均正常，因此不是固定的签名、过期或权限配置问题。

环境与结果

• FNN：v0.9.0-rc7
• 测试强度：每秒发起 5 笔完整交易，连续测试 5 分钟，共 1500 笔
• 结果：1497 笔成功，3 笔失败，失败率 0.2%
• 失败 Job：https://github.com/sunchengzhu/cch-daily-smoke/actions/runs/29346453027/job/87131324455

Sequence	节点	RPC
441	node2	get_payment
455	node1	send_btc
630	node1	get_cch_order

关键日志

# CI
15:43:24 ERROR FAILED seq=441 ... Unauthorized
15:43:24 ERROR FAILED seq=455 ... Unauthorized
15:44:01 ERROR FAILED seq=630 ... Unauthorized
15:46:55 INFO RESULT FAIL succeeded=1497/1500 failed=3 failure_rate=0.200%

# node2：对应 seq 441
2026-07-14T15:43:24.631334Z DEBUG fnn::rpc::biscuit:
authorize failed: Reached Datalog execution limits

# node1：对应 seq 455
2026-07-14T15:43:24.647755Z DEBUG fnn::rpc::biscuit:
authorize failed: Reached Datalog execution limits

# node1：对应 seq 630
2026-07-14T15:44:01.506089Z DEBUG fnn::rpc::biscuit:
authorize failed: Reached Datalog execution limits

三次服务端错误与 CI 中三次 Unauthorized 的时间完全对应。

源码分析

Fiber 直接调用 Biscuit 默认 authorize()，没有设置自定义 limits：

self.build_rule()?
    .fact(...)?
    .build(token)?
    .authorize()?;

源码：crates/fiber-lib/src/rpc/biscuit.rs

biscuit-auth 6.0.0 的默认值为：

max_facts: 1000
max_iterations: 100
max_time: 1ms

相关权限规则只是简单的 read(...)/write(...) 判断，并且相同 Token 已成功鉴权数千次，因此最可能是负载下偶发超过默认 1ms。当前日志没有输出具体 subtype，无法进一步区分 Timeout、TooManyFacts 或 TooManyIterations。

此外，middleware 会把所有鉴权错误统一转换为 -32999 Unauthorized，导致客户端无法区分 Token 无效、权限不足和临时 Datalog 执行限制。

建议

1. 支持配置 Biscuit limits，并考虑将默认 max_time 从 1ms 提高到 10ms。
2. 日志输出具体限制类型及 RPC method。
3. Datalog 执行限制使用独立、可重试的 RPC 错误，不要统一映射为 Unauthorized。