Author SHA1 Message Date
root 9faae09851 fix: reconcile v2 retries and parse all gallery search layouts 2026-10-07 23:50:52 +08:00
root 4c1a2d22e2 Fix gallery resolution parsing for split archive paragraphs 2026-10-07 23:09:47 +08:00
root e6c7dab12e Rewrite durable task lifecycle, gallery indexing and subscription publication 2026-10-07 17:46:32 +08:00
root 776eb8cf53 修复 GraalVM 25 原生构建中的 commons-logging 初始化 2026-09-26 12:45:30 +08:00
root e03a2e007b netty 按需声明,并记录 jar 运行期的低内存调优
netty-all 是聚合 pom,会拖进 5 个平台的 native-quic、各架构的 epoll/kqueue/
io_uring 传输以及 htt3/mqtt/redis 等一堆用不到的 codec。主站实际只用
Bootstrap/NioEventLoopGroup/NioSocketChannel/ByteBuf/ByteToMessageCodec/
LengthFieldBasedFrameDecoder/LoggingHandler/Promise,因此收敛为
transport + codec-base + handler(含 buffer/common/resolver 传递依赖)。

效果:jar 68.1MB -> 52.2MB,netty 模块 50 -> 7,进程 RSS 291MB -> 约 227MB。
433 个测试全绿。hutool 未动。

同时把 jar 运行期的 systemd JVM 参数与前后实测数据记入
docs/MEMORY_TUNING_2026-09-21.md,避免这些只存在于服务器 drop-in 的参数
在下次重新部署时丢失。
2026-09-21 17:27:17 +08:00
root 5f1066cdab 去掉订阅快照的周期性重发
节点侧已改为按「收到主站的任何消息」判定新鲜度(过期 ⇔ 主站失联),
主站每 30 分钟的可用性检查即可持续刷新,因此主站不需要任何为续期的周期性推送。

- 删除上一版加入的 scheduledSubscriptionKeepalive(6 小时一次)。
- 保留「内容变化即推」(各处 requestSubscriptionSync)与「节点上线即推」
  (initChannel),两者覆盖变更下发与节点冷启动两种场景。
- 移除对应的 standby 保活配置项。

最终推送时机:只在订阅内容变化时推送,另在节点上线时补推一次;
空闲时零快照传输(此前 60 秒一次时为约 486 MiB/天)。

测试:433 项全过。新增 SubscriptionSnapshotPushTest 锁住两条约定——
RemoteService 不得再有定时任务(续期改由节点存活信号承担),以及订阅刷新成功后
仍必须触发一次推送(用真实 refreshDueAccounts 调用验证)。
2026-09-21 16:49:55 +08:00
root e3d4220997 订阅快照改为变更即推,并加低频保活
原先主站每 60 秒无条件重建并推送整份快照:实测 12 账号时传输约 346 KiB/次、
约 486 MiB/天,节点每次都要 base64 解码、SHA-256、gunzip、JSON 校验后判
APPLY_OLD 丢弃。而 revision 是内容寻址的,订阅不常变动时这些推送纯属浪费。

改为:
- 内容变化即推:各处已有的 requestSubscriptionSync()(刷新成功、绑定/改绑、
  重置 Key、增删停用账号、过滤开关变更)保持原样,延迟仍在数秒内。
- 低频保活:新增 scheduledSubscriptionKeepalive(),默认 6 小时一次,仅用于刷新
  节点「最近收到快照」的时刻。节点的过期判定依据该时刻(见 storageNode 侧修复),
  因此保活周期必须远小于节点 SubscriptionMaxStaleSeconds(默认 7 天)。
- 节点上线即推:initChannel 保持推送一次,覆盖节点刚重启、本地尚无快照的情况。

空闲流量约 486 MiB/天 -> 约 1.4 MiB/天(约三个数量级)。

测试:432 项全过。新增 SubscriptionKeepaliveTest 锁定不变量——保活入口必须存在,
且周期小于节点有效期的 1/4、初始延迟小于一个周期;否则节点可能在两次保活之间判过期。
2026-09-21 16:36:13 +08:00
root 42921ba97b 修复多处隐患并做低风险性能优化
缺陷修复:
- GalleryUtil.parseGid 只捕 IndexOutOfBoundsException:非数字段抛 NumberFormatException、
  link 为 null 抛 NPE,都会穿透成 500(按链接查询直接把用户输入喂进来)。现统一返回 null,
  由调用方转成业务失败。
- 图片索引缓存不是整体生效:先写 gidToKey 再逐页写 ImageKeyCache,中途失败会留下
  「gidToKey 命中但页 key 缺失」的半截缓存,后续请求直接返回已缓存而永远取不到图。
  现任何异常都回滚已写入部分,并识别历史半截缓存自动重建。
- 全站请求日志把 AuthCode 明文记入 INFO,改为只记 present/absent。
- selectEnableAuthCode 的 SQL 与全量查询完全相同(都无 isEnable 条件),停用用户的
  授权码刷新后仍放行,isEnable 形同虚设;补齐条件并允许 null 按默认 true 处理。
- /validate 手工 String.format 拼 JSON,用户名含引号或反斜杠会产出非法 JSON,
  前端 JSON.parse 失败;改用 ObjectMapper 组装。

性能与并发:
- Netty IO 线程不再直接做 JDBC:节点上报的任务状态改由单线程顺序执行器落库与推送,
  既不打乱「按上报顺序覆盖状态」的语义,也不阻塞心跳与响应。
- 取订阅不再等待第三方归属地查询:先以 unknown 落库并立即分发,归属地由后台线程补齐。
- 搜索关键词与分页参数按 UTF-8 编码,&、#、中文不再破坏上游查询串。
- selectAllGallery 的收藏标记由 O(n*m) 嵌套循环改为集合查找。
- 复用 CustomUtil.objectMapper(8 处 new ObjectMapper);gid2MpvKey 改为有界 LRU(2048);
  缩略图清理排序不再每次比较都重读文件属性;FileDownload 关闭响应流,避免大文件下载的
  临时文件堆积到 Full GC;移除从未使用的 downloadThread 线程池。

测试:431 项全过(新增索引回滚、半截缓存重建、用户名特殊字符、畸形链接查询、
搜索编码、分发不等待定位、批量收藏标记等用例;异步状态处理用例加 awaitStatusApplied
屏障保持断言确定性)。
2026-09-21 16:17:17 +08:00
root 584db277a6 登录方案文档更新为全部完成并记录独立复验结论 2026-09-21 11:25:49 +08:00
root 628f3ad0a2 登录方案:确认复用 /login 并一次返回两条链接,密钥已入库 2026-09-21 10:33:00 +08:00
root dfb647bcd3 登录方案文档更新为已实施状态,补机器人侧票据规格 2026-09-21 10:23:26 +08:00
root 00e27152fe 新增一键登录:共享密钥自签票据换会话
- 新增 PanelLoginTicket:v1.<时间戳>.<HMAC-SHA256> 形状,只验签与时间窗,
  不发一次性状态。机器人用同一把密钥本地签发,因此不需要机器人到主站的网络调用,
  一端临时不可达也不影响生成链接。窗口内可重放(用户明确接受),过期即失效。
- PersonalController 新增 GET /login?t= 换 HttpSession(先作废旧会话防固定攻击)、
  /login/logout 退出、/personal/denied 提示页。
- PersonalInterceptor 放行「有效会话或 AuthCode=alone」,拒绝时回 401 供前端区分;
  InterceptorConfiguration 排除登录端点与 /personal/。
- server.servlet.session.cookie.path 固定为 /:nginx 会把 /user 改写成 /personal/user,
  沿用容器推导的 /personal 会让浏览器判定路径不匹配而丢会话。
- 密钥经 PERSONAL_LOGIN_SECRET 环境变量注入,为空时一律拒绝而非放行。
- 新增 PanelLoginTicketTest(7 项)与登录/退出/会话相关用例,共 423 项通过。
2026-09-21 10:20:51 +08:00
root 654181d2ea 登录方案对齐 PersonalHub 既有实现 2026-09-21 10:03:35 +08:00
root bee2f11fd6 移除文件管理与文件分享,接口范围收敛到订阅与用户管理
- 删除 /personal/files、uploadFile、private/**、compress、delete 与对应服务实现,
  连同压缩线程池、TAR 打包与 PersonalArchiveTest
- 删除 /personal/share、extendShareTime、cancelShare 与 /GetFile/{path},
  以及 ShareFile 实体、ShareFileMapper、PublicService.GetFile、每天四点的
  checkShareCode 定时任务和分享码失败黑名单
- CustomBean 去掉 ShareFile 的 Native 反射登记
- 同步收敛 PersonalServiceTest、PublicServiceTest、PersonalControllerTest、
  PublicControllerTest、LocalServiceTest 与三个画廊测试
- 文档同步,并附上 /login 方案 docs/login-command-plan.md

测试:mvn -o test 410 项通过
2026-09-21 10:02:07 +08:00
root 305036ddfe 新增下载人查询接口,并把管理员判定收敛到服务端
- Gallery 增加 downloaderName,仅在管理员查询任务列表时填充;普通用户
  既不获取也不下发他人身份
- 新增 GET /GalleryManage/downloader?gid=&AuthCode=,按 gid 返回实际下载人
  昵称,非管理员一律拒绝
- /validate 下发 isAdmin,前端不再各自硬编码 userId === 3
- 管理员判定统一走 UserService.ADMIN_USER_ID
- 补充列表填充、按 gid 查询的权限与缺任务分支测试
2026-09-20 22:54:23 +08:00
root d949415e4d 分散刷新后补上备机同步
原实现中「全量刷新 → 通知备机同步」是同一步的两个动作:refreshAll()
由调用方跟一次 requestSubscriptionSync()。改为按账号分散刷新后,调度器
自己没有触发同步,刷新出的新缓存要等到下一次周期同步才可能送到备机。

生产上每 60 秒有一次周期同步兜底,所以影响被限制在最长约一分钟的滞后;
但这属于重构时漏掉的语义,且一旦把周期同步间隔调大就会变成真实缺口,
因此仍按原语义在刷新到内容后立即触发同步。没有账号到期时不打扰备机。

测试 441 通过。
2026-09-15 16:27:29 +08:00
root 3adb42c538 订阅刷新改为每账号错峰,保证 24 小时内各刷一次
原先每 24 小时一次性刷新全部子账号:12 个账号在 16 秒内背靠背发完
24 个请求,且 @Scheduled(fixedRate) 未设 initialDelay,首次触发即立刻
执行,导致每次重启都重刷一遍全部账号——上游看到的请求密度实际取决于
部署频率。请求头也只带 HttpClient 默认值,上游收到的是
"Apache-HttpClient/5.6.4 (Java/25.0.4)"。

改为高频轻量 tick(5 分钟,延迟 2 分钟启动),只刷新已到期账号,单轮
最多 2 个。24 小时保证由三层构成:成功后按「成功时刻 + 窗口 − 一个 tick」
推进相位,使分散效果自我维持;到期时刻持久化在数据库,停机期间错过的
账号恢复后立即到期;兜底把被写到超过一个窗口之后的计划判为损坏并立即
刷新。失败用固定间隔重试(数据库没有连续失败次数字段,做不出真正的指数
退避),重试排在将来时被尊重,避免故障账号每个 tick 重打上游。从未成功过
的账号按 tick 间隔尽快刷,已有缓存的按窗口分散。超过两倍窗口仍未成功的
账号推送告警。

请求头按账号与订阅格式确定性地伪装成 mihomo/clash-verge/v2rayN 等真实
客户端身份;同一账号固定用同一身份,逐请求更换反而更像机器。实现前已
实测确认显式设置 Accept-Encoding 不会破坏 HttpClient5 的透明解压。注意
这只能伪装到 HTTP 头,TLS 指纹仍与真实客户端不同。

新增 next_refresh_at、last_success_epoch 两列,均为 Epoch 毫秒整数:SQLite
的 CURRENT_TIMESTAMP 写 UTC 文本而被 JDBC 按本地时区解释,实测偏差 8 小时,
不能用于时间比较。迁移为纯新增列,旧二进制忽略它们,故回滚程序无需回滚
数据库;回填 last_success_epoch 必须执行,否则既有账号会被当成从未成功过
而集中补刷,正好复现本次要消除的爆发。

同时移除 LocalService 的 24 小时全量入口与 refreshAll()(它们正是爆发式
写法),并将调度线程池由默认 1 调到 2,避免刷新阻塞连接自检。

测试 439 通过,指令覆盖率 82.1%。
2026-09-15 16:15:41 +08:00
root a0959618ca 修复 Jackson 3 迁移引入的看图 500,并大幅补充单元测试
生产缺陷(2026-09-14 Spring Boot 4.1.1 / Jackson 2→3 迁移引入):
- GalleryUtil.parseImageKeys 解析 mpv 页 imagelist 时,行尾分号属 JS 语法,
  Jackson 3 默认 FAIL_ON_TRAILING_TOKENS=true 会抛 StreamReadException。
  改为先由 parseImagelist 剥掉 JS 语句外壳(前缀 + 行尾分号)再交给 JSON 解析。
  已用线上真实页面验证:修复前必抛异常,修复后正确解析出 154 页 imgkey。
- GalleryManageService.cacheImagesKey 只 catch IOException,而 Jackson 3 的
  JacksonException 继承 RuntimeException,异常穿透 Controller(无 @ControllerAdvice)
  表现为 HTTP 500。改为同时兜 JacksonException 并按业务失败返回。
  同时用 parseGidFromLink 拦住段数不足的畸形链接(原先会抛
  ArrayIndexOutOfBoundsException 变成 500)。

既有行为修正:
- PersonalService.deleteFile 原先依据 hutool FileUtil.del 的返回值报成功,
  而该函数对不存在的目标返回 true,导致路径写错也显示「删除成功」;
  改为先校验存在性,不存在则回「文件不存在」。
- PersonalService.getFiles 的 listFiles()==null 分支原先报「文件夹为空」,
  但该返回值只在 I/O 出错时出现,会掩盖权限/磁盘问题,改为「读取文件夹失败」。

测试:143 → 379 用例,指令覆盖率 36.2% → 80.2%(分支 34.6% → 69.8%)。
新增控制器(MockMvc standaloneSetup)、服务层、拦截器/过滤器、WebSocket、
配置装配与 MyBatis AOT hints 的测试;全部离线可跑,不加载 Spring 上下文、不连库、不占端口。
2026-09-15 13:42:22 +08:00
123 changed files with 14616 additions and 1065 deletions
+3
View File
@@ -41,3 +41,6 @@ build/
### Mac OS ### ### Mac OS ###
.DS_Store .DS_Store
/.idea/encodings.xml /.idea/encodings.xml
# Migration script bytecode
scripts/__pycache__/
+150
View File
@@ -0,0 +1,150 @@
# 图片索引缓存(GalleryIndex)— 重写模块说明
## 问题
在线阅读的索引是「gid → 每一页 imgkey」,存在 `cache.db` 的两张表里:
| 表 | 列 | 含义 |
|---|---|---|
| `gidToKey` | `gid`, `key`, `pages` | 画廊头部:gid、画廊 key、总页数 |
| `ImageKeyCache` | `gid`, `imgkey`, `page` | 每页的 imgkey |
旧实现(`GalleryManageService.cacheImagesKey`)依次做四件事:查缓存、判断是否需要重建、
外呼抓 mpv 页、逐页 `INSERT`。由此产生的实际问题:
1. **没有唯一约束**。实测生产缓存库有 4 组重复 gid、97 组重复 `(gid,page)`。
2. **整组替换不是原子的**。先写头部再逐页写,中途失败会留下「头部命中、页 key 缺失」的
半截索引;后续请求直接判定为已缓存,永远取不到图。
3. **用行数判断完整性**。`count(*) == pages` 无法发现「页码 1、3 而 pages=2」这种跳号。
4. **没有并发合并**。同一本画廊被并发打开时,两个构建者会互相删除对方刚写入的行。
5. **逐页写入放大写事务**,抓取与写入之间没有事务边界。
## 交付物与职责
| 文件 | 职责 |
|---|---|
| `Service/GalleryIndexService.java` | 索引的唯一读写入口:同 gid 合并、抓取在事务外、校验后整组短事务替换 |
| `Exception/GalleryIndexException.java` | 分类失败原因:INVALID_LINK / UNAVAILABLE / MALFORMED / TRANSIENT |
| `Configuration/GalleryIndexConfiguration.java` | 缓存库的事务管理器与读写模板(独立文件,未改 SqlConfiguration) |
| `Dao/cache/ImageCacheMapper.java` | 新增页码查询、脏行计数、批量写入(原有方法全部保留) |
| `Maintenance/GalleryIndexMaintenance.java` | 离线体检与修复工具(只删不健康 gid 的整组索引) |
| `scripts/migrate_image_index_cache.{sh,sql}` | 迁移:默认 dry-run、备份、清理、建唯一索引 |
| `Service/GalleryIndexServiceTest.java` | 服务编排的行为约束(无 Spring、无网络) |
| `Maintenance/GalleryIndexMaintenanceTest.java` | 工具在真实临时 SQLite 上的判定与删除范围 |
| `Dao/cache/ImageCacheMapperSqlTest.java` | 注解 SQL 在真实 MyBatis + SQLite 上的验证 |
## 接口
```java
GidToKey ensureIndex(String url) throws GalleryIndexException;
```
返回该画廊的 `gid` / `key` / `pages`。三条不变量:
- **抓取在事务外**:`upstream.fetchPageKeys(url)` 先于任何写事务执行;写事务只在
`publish(...)` 里开启,删除旧索引与批量写入在同一个事务内整体生效。
- **校验先于写入**:行必须满足 gid 与请求一致、页码恰好为 1..N、不重复、imgkey 非空,
否则抛 MALFORMED,此时尚未开启写事务,库中旧索引原样保留。
- **读在快照内**:`lookup` 在同一个读事务里读头部与页码,避免读到整组替换的中间状态。
失败原因:
| Reason | 触发条件 | isPermanent() | 建议处置 |
|---|---|---|---|
| INVALID_LINK | 链接里取不到 gid 或 key | true | 直接回「链接错误」 |
| UNAVAILABLE | 上游返回空页列表(下架/删除) | true | 回「该图片已下架或已被删除」 |
| MALFORMED | 上游页 key 不自洽 | false | 可重试,回「网络波动或其他异常」 |
| TRANSIENT | 上游异常或写库失败 | false | 可重试,回「网络波动或其他异常」 |
## 接入方式(由主模型完成 GalleryManageService 侧改动)
构造函数注入:
```java
final GalleryIndexService galleryIndexService;
```
`cacheImagesKey` 改为:
```java
public String cacheImagesKey(String url) {
Response response = Response.generateResponse();
try {
GidToKey gidToKey = galleryIndexService.ensureIndex(url);
return response.success(objectMapper.valueToTree(gidToKey)).toJSONString();
} catch (GalleryIndexException e) {
response.failure(e.getReason() == GalleryIndexException.Reason.INVALID_LINK
? "链接错误"
: e.isPermanent() ? "该图片已下架或已被删除" : "网络波动或其他异常");
return response.toJSONString();
}
}
```
Spring 装配无需额外配置:`GalleryIndexConfiguration` 已把 `GalleryIndexService` 注册为 bean,
依赖 `ImageCacheMapper`(cache 数据源)与两个 TransactionTemplate。原来的
`isIndexComplete` / `rollbackImageIndex` 私有方法与 `ImageCacheMapper` 的
`countImageKeyCacheByGid` / `delete*ByGid` 仍保留,供旧调用方与非索引路径使用。
读取路径 `getCachedImage` 不需要改动:它按 `(gid,page)` 查 ImageKeyCache,现在会走新加的
`ux_ImageKeyCache_gid_page` 唯一索引。
## 迁移
```bash
# 1. dry-run(只读,不带 --apply 绝不写入)
scripts/migrate_image_index_cache.sh --db /path/to/copy.db
# 2. 先在 SQLite 备份副本上演练
sqlite3 -readonly /home/lionwebsite/cache.db ".backup '/tmp/cache-copy.db'"
# 3. 确认后执行(自动做 .backup 备份并校验)
scripts/migrate_image_index_cache.sh --db /tmp/cache-copy.db --apply
```
迁移只做两类写入:删除不健康 gid 的整组索引(重复头部、半截、跳号、空 imgkey)、
删除孤儿页 key;随后建立 `ux_gidToKey_gid` 与 `ux_ImageKeyCache_gid_page`。
若清理后仍有重复,建唯一索引会失败并整体回滚。真实的 `/home/lionwebsite/cache.db` 未被本次工作修改。
离线修复工具:
```bash
java -cp sqlite-jdbc.jar Maintenance/GalleryIndexMaintenance.java --db /tmp/cache-copy.db # dry-run
java -cp sqlite-jdbc.jar Maintenance/GalleryIndexMaintenance.java --db /tmp/cache-copy.db --apply # 执行
```
对 `/home/lionwebsite` 下的库运行 `--apply` 会被拒绝,除非显式加 `--allow-production`。
## 测试
在 `/tmp` 的独立源码副本中运行(未复制任何生产 runtime config、未改生产库):
```
mvn -o -Djacoco.skip=true \
-Dtest='GalleryIndexServiceTest,GalleryIndexMaintenanceTest,ImageCacheMapperSqlTest' \
-DfailIfNoSpecifiedTests=false test
Tests run: 9, Failures: 0, Errors: 0 -- GalleryIndexMaintenanceTest
Tests run: 21, Failures: 0, Errors: 0 -- GalleryIndexServiceTest
Tests run: 7, Failures: 0, Errors: 0 -- ImageCacheMapperSqlTest
Tests run: 37, Failures: 0, Errors: 0
BUILD SUCCESS
```
覆盖的关键场景:
- 同 gid 24 线程并发只抓取一次、只发布一次;失败结果也被等待者共享;失败后可重试。
- 抓取发生在事务之外;所有删除与写入都在事务内;整组替换只有一个写事务。
- 行数相同但页码跳号、空 imgkey、空 key、pages<=0 均判定为不完整并重建。
- 写入失败整体回滚,旧索引与失败前逐字段一致。
- 上游畸形数据(gid 不符、页码重复/跳号/越界、imgkey 为空)在开启写事务之前被拒绝。
- 大规模索引按 BATCH_SIZE 分片写入;重复发布替换旧行而非累积。
- 注解 SQL 在真实 SQLite 上验证,含唯一约束确实拒绝重复写入。
## 已知边界
- 迁移后 gidToKey 433 条、ImageKeyCache 43704 条(原 443 / 43918)。
被删除的是 4 个冲突或不完整 gid 的整组索引与 214 行关联页 key,由下一次在线阅读重抓。
- 未做性能压测,不承诺加索引后的具体延迟收益,只确认查询计划已由全表扫描变为索引查找。
- lookup 的读事务只是为了拿到一致快照;实测 sqlite-jdbc 不允许在已建立的连接上设置
readOnly,因此读模板只设超时、不设只读标志。
+78
View File
@@ -0,0 +1,78 @@
# 2026-09-21 低内存调优记录(jar 运行期)
编译机未上线期间,两个后端都以 jar 方式运行。本机为 2 核 / 2.9G,活跃堆很小但 RSS 偏高,
做了一轮只改 JVM 参数与依赖声明的低内存调优。全程未改业务代码。
## 实测前后对照
| 服务 | 项目 | 调优前 | 调优后 |
| --- | --- | --- | --- |
| 主站 lionwebsite | RSS | 291 MB | 约 227 MB |
| 主站 lionwebsite | jar 体积 | 68.1 MB | 52.2 MB |
| 主站 lionwebsite | netty 模块 | 50 个 | 7 个 |
| 主站 lionwebsite | 线程数 | 40 | 37 |
| 存储节点 storageNode | RSS | 171 MB | 约 155-162 MB |
| 存储节点 storageNode | netty 模块 | 35 个 | 5 个 |
| 存储节点 storageNode | 线程数 | 24 | 20-22 |
调优前后堆使用都在 40-52 MB 量级,RSS 的主要成分是元空间、代码缓存与各类映射,
因此收益主要来自收敛预留和减少类加载面,而不是堆。
## 一、JVM 参数(systemd drop-in,不在版本控制内)
主站 `/etc/systemd/system/lionwebsite.service.d/jar-run.conf`:
/usr/local/jdk-25/bin/java -Xmx256m -Xss512k -XX:MaxMetaspaceSize=160m \
-XX:+UseSerialGC -XX:+UseCompactObjectHeaders \
-XX:ReservedCodeCacheSize=96m -XX:+ExitOnOutOfMemoryError \
-jar /home/lionwebsite/lionwebsite.jar
存储节点 `/etc/systemd/system/storageNode.service.d/jar-run.conf`:
/usr/local/jdk-25/bin/java -Xmx192m -Xss512k -XX:MaxMetaspaceSize=96m \
-XX:+UseSerialGC -XX:+UseCompactObjectHeaders \
-XX:ReservedCodeCacheSize=96m -XX:+ExitOnOutOfMemoryError \
-cp /root/gallery/storageNode/storageNode-jar.jar:/root/gallery/storageNode/lib/* lion.Main
取值依据:
- `UseSerialGC`:小堆、低吞吐场景,省掉 G1 的分区元数据与并发标记线程。存储节点原本就是默认 SerialGC,主站原本是默认 G1,本次对齐。
- `UseCompactObjectHeaders`:对象头 12 到 8 字节,堆内以 String 与 map 节点为主时收益直接。
- `ReservedCodeCacheSize=96m`:原先 240m 预留,实测主站只用约 14 MB、节点约 6 MB。
- `-Xmx` 收敛:主站由 384m 到 256m,节点由 256m 到 192m,相对活跃堆仍留 4-6 倍余量。
回滚:恢复同目录 `jar-run.conf.bak.20260921T090605Z`(主站)或
`jar-run.conf.bak.20260921T090746Z`(节点)后 `daemon-reload` 加 `restart`。
## 二、依赖精简(已入库)
`netty-all` 是聚合 pom,会拖进全部模块。两个后端只用到 NIO 传输与基础编解码,
因此改为按需声明:
- 主站(netty 4.2.17):`netty-transport` 加 `netty-codec-base` 加 `netty-handler`。
`handler` 用于 `RemoteService` 的 `LoggingHandler` 协议调试日志。
- 存储节点(netty 4.1.138):`netty-transport` 加 `netty-codec`。
节点侧未使用 `handler` 模块的类,故不声明。4.1 无独立 `codec-base`,编解码基类在 `netty-codec`。
剔除的内容包括 5 个平台的 native-quic、aarch64 与 riscv64 与 osx 的 epoll 与 kqueue 与 io_uring 传输、
以及 codec-http3 与 mqtt 与 redis 与 smtp 与 stomp 与 xml 与 protobuf 等。
`hutool-all` 未调整。
## 三、已评估但未采用:AOT 缓存
JDK 25 的 `-XX:AOTCache` 可免原生编译生成类缓存。实测交替 A/B 各两轮:
| | 启动耗时 | RSS |
| --- | --- | --- |
| 无 AOT | 6.37s / 6.74s | 220.9 MB / 223.1 MB |
| 带 AOT | 6.27s / 6.36s | 221.4 MB / 222.7 MB |
启动约快 0.2s(噪声量级),RSS 与元空间均无改善,代价是 80 MB 缓存文件且每次发版需重训。
按降低内存的目标不值得,故未在生产启用。
## 四、遗留与后续
- 原生编译机恢复后应重新评估原生镜像:内存会显著低于 jar,届时上述 JVM 参数失去意义。
- 存储节点目录内仍保留 `lib.bak.20260921T091617Z`(47 jar)与 `storageNode.jar.bak.*`,为本次回滚路径。
- 两个 `jar-run.conf` 的 JVM 参数只存在于服务器 drop-in,未纳入版本控制;重新部署时需按本文恢复。
+61
View File
@@ -0,0 +1,61 @@
# 订阅缓存分代化 — 重写模块说明
## 问题
一个订阅账号对应两份内容:`v2ray.txt`(Base64 节点)与 `clash.yaml`(Clash 配置)。
旧实现先 `atomicWrite` 写 v2ray,再 `atomicWrite` 写 clash,两次替换彼此独立:
1. **两份文件可以不同版本**。第二次写入失败(磁盘满、进程被杀)时,用户会拿到
「新 v2ray + 旧 clash」的混合版本,且没有任何状态能表明它已经撕裂。
2. **读取方无法保证配对**。分发路径与备机快照各自 `Files.readAllBytes` 两个路径,
在这两次读之间完成一次刷新,就会读到跨版本组合。
3. **过期刷新可能复活**。在途刷新返回时只比较上游 key 与过滤开关;一次
`invalidateCache` 之后,迟到的响应仍会把内容写回被清空的路径。
4. **失败留半成品**。第一次写成功、第二次失败时,磁盘上就是只有一份文件的账号。
## 交付物与职责
| 文件 | 职责 |
|---|---|
| `Util/SubscriptionCacheStore.java` | 分代存储:两份文件同一分代写入 → 校验 → 指针原子发布;读取始终来自指针指向的同一分代 |
| `Service/SubscriptionRefreshService.java` | 抓取与解析在协调器锁外进行,发布短持锁;`invalidateCache` 递增版本使在途刷新失效 |
| `Service/SubscriptionStandbySnapshotService.java` | 快照构建在读锁内通过同一 store 解析成对内容,保证同版本 |
| `Util/ClashSubscriptionParser.java` | 按 YAML 语义过滤高倍节点并清理 `proxy-groups` 引用(由主模型提供) |
## 目录布局
```
<cacheRoot>/<accountId>/current 指针(Properties 文本,原子替换)
<cacheRoot>/<accountId>/generations/<generation>/v2ray.txt
<cacheRoot>/<accountId>/generations/<generation>/clash.yaml
<cacheRoot>/<accountId>/v2ray.txt + clash.yaml 旧结构,仅在两份都在时按 legacy 读
```
`current` 记录分代名与两份文件的 SHA-256。分代名是定宽 `时间戳-序号`,
因此字典序即时间序;默认只保留最近两个分代(当前与上一个)。
## 不变量
- **发布最后一步才切指针**:先写两份文件,读回校验长度与内容,再原子替换 `current`。
抓取失败、解析失败、任一份写入失败、指针写入失败,都会回滚新分代目录并保留上一个完整版本。
- **读取配对**:`read` / `resolve` / `hasComplete` 走同一份 `load` 契约,每个调用只读一次
指针快照,再按该快照校验双文件哈希。只有两份齐全且匹配才算一个有效版本;
指针缺失才回退 legacy(两份齐全时),指针存在但读不出完整版本(半写、篡改、缺字段)
时按无可用缓存处理,不回退到更旧的半份内容。
- **分发不会发出半份缓存**:`cachedPath` 在无有效版本时返回保证不存在的哨兵路径
(`.unavailable/`),使 `SubService` 判定为 503,而不会被单份 legacy 文件骗过。
`SubService.updateSub` 在读锁内完成 resolve 与 `FileDownload.export`,与 publish/invalidate 互斥。
- **失效不可复活**:`invalidateCache` 在写锁内删除指针/分代/旧结构文件,并移除该账号的
刷新版本标记;在途响应返回时因版本标记对不上而被丢弃,不会重新发布。
- **兼容旧结构**:只有当账号根下 `v2ray.txt` 与 `clash.yaml` 都在时才读为 legacy 版本;
新分代发布成功后旧结构文件被移除,避免两份来源并存。
## 测试
| 测试 | 锁定行为 |
|---|---|
| `Util/SubscriptionCacheStoreTest` | 发布后两份同分代;第二份写入失败 / 指针发布失败均保留旧版本;legacy 双文件才有效;半份 legacy、缺一份分代、内容被篡改、坏指针、指针指向缺失分代均判定不可用;每次查询只读一次指针;哨兵路径保证不存在 |
| `Service/SubscriptionCacheGenerationTest` | 在途旧刷新在失效与新版发布后被丢弃;被丢弃的刷新不留任何缓存 |
| `Service/SubscriptionFilteringTest` | 过滤规则;失败保留上次完整版本;`cachedPath` / `hasCompleteCache` 新契约;legacy 路径定位 |
| `Service/SubscriptionStandbySnapshotServiceTest` | 快照从单一分代读成对内容;不完整 legacy 账号不入快照 |
| `Util/ClashSubscriptionParserTest` | YAML 语义过滤(内联节点与引用)与畸形正文拒绝 |
+80
View File
@@ -0,0 +1,80 @@
# 任务生命周期与协议 v2
范围:主站下载任务的持久化生命周期、与存储节点的可靠上报协议。对应评估报告第 2、6 节。
## 为什么重写
原实现里创建先`POST`上游、后落库,删除先删库、后等节点,节点上报异步写完后立刻清队列:上下游任一侧失败都会留下「主站与节点说法不一致」的状态,且没有任何东西能把它们对回来。
复现过的具体后果:模拟状态消息发送失败后,完成任务已从节点队列移除、消息却未发出,于是「归档已存在、主站仍显示未完成」。
## 不变量
1. `gid` 只有一个有效生命周期意图。创建先落唯一 `CREATE` 意图(并写一条可见的占位任务行),靠 `gallery(gid)` 主键唯一把并发创建收敛成一个赢家;输家直接得到业务失败。
2. 上游副作用只在意图持久化之后发生。顺序固定为:登记意图 → 只读解析(`prepare`,无 POST)→ 持久化 `SUBMITTING` → 真正 `submit`。
3. `SUBMITTING` 之后绝不自动重发。上游不支持幂等,因此超时/断连/落库失败/重启中断一律落 `UNKNOWN` 等待人工对账处理,而不是再 `POST` 一次。
4. 任务行与周用量在同一短事务里写入;重复落库不重复计量。数据库失败时任务行与意图保持一致(要么都在,要么明确为 `UNKNOWN`),不允许出现「有任务没计量」。
5. 删除失败保留记录。先落 `DELETE_PENDING` 与稳定 `deleteOperationId`,只有节点确认(或文件已不存在)后才在同一事务里清收藏、删任务行并留 `DELETED` 墓碑。
6. 上报按 `revision` 单调推进,按 `operationId` 归属。迟到的旧世代上报被丢弃且不改变当前任务;墓碑(`DELETED`)永久忽略所有上报,已删除任务只能由显式 `CREATE` 重新建立新世代。
7. 回执只在提交之后发送。数据库失败不回 ACK,节点保留快照重投;重复上报也回 ACK 以停止重放,但不会重复通知。
8. 通知与 WebSocket 推送在事务提交之后、由独立执行器发送;慢通知不阻塞状态提交。
9. 只有确认已提交(`SUBMITTED`)或迁移前确实提交过的旧任务才允许下发给节点。`UNKNOWN`、`RESERVED`、`SUBMITTING`、`DELETE_PENDING`、`DELETED` 一律不下发:上游不支持幂等,把未确认/未计量的任务重投会在节点侧变成「下载完成却 0 额度」。
10. 启动时先绑定上报处理器与判定解析器、执行恢复(`SUBMITTING → UNKNOWN`、释放 `RESERVED`),完成之后才允许连接与自动重投;恢复失败时只保持连接,禁止自动派发。
## 状态表
| 层 | 取值 |
| --- | --- |
| `task_operation.state` | `RESERVED` `SUBMITTING` `SUBMITTED` `UNKNOWN` `DELETE_PENDING` `DELETED` |
| 任务展示状态 | `提交中` `已提交` `下载中` `等待压缩` `压缩中` `下载完成` `提交未知` `待删除` `删除失败` |
| 节点上报状态码 | `1` 下载中 `2` 等待压缩 `3` 压缩中 `4` 下载完成(沿用旧码) |
状态码到中文文案只在 `TaskStatus` 一处映射。
## 线协议
帧格式不变:`[messageType(1B)][length(4B, 大端)][JSON]`。旧类型 `0..9` 的字段一个都不动(旧节点 Jackson 会拒绝未知字段)。
| 类型 | 消息 | 关键字段 |
| --- | --- | --- |
| 10 | `ProtocolHelloMessage` | `protocolVersion=2`, `features=["reliable-task-v2"]` |
| 11 | `TaskCommandMessage` | `operationId`, `action=DOWNLOAD\|DELETE`, `gid`, `name` |
| 12 | `TaskReportMessage` | `reportId`, `revision`, `galleryTasks[]`, `operationIds{gid→op}` |
| 13 | `TaskReportAckMessage` | `reportId`, `result` |
流程:主站先发旧 `Identity`,再发类型 10;节点回类型 10 声明 v2 后,才允许使用 `11..13`。未协商成功时自动退化为旧的 `DownloadPostMessage`/`DeleteGalleryMessage`(此时不承诺可靠 ACK,属已知兼容边界)。
## 部署与迁移
1. 用 `scripts/migrate_task_operation_v2.sql` 建表。脚本只有独立语句、不含 `BEGIN/COMMIT`,由外层 wrapper 包事务并在执行前备份。
2. 该脚本还会在 `gallery(gid)` 上建唯一索引。历史库若已有重复 gid,这条语句会失败:先人工确认重复来源再处理,不要直接删行。
3. 主库连接需要 busy 等待(`SqlConfiguration` 已为主数据源设置 `PRAGMA busy_timeout=5000`)。单文件 SQLite 并发开短事务时,没有等待窗口会直接抛出 `SQLITE_BUSY`。生产库建议同时启用 WAL。
4. 旧任务行没有操作记录,首次收到上报时会按显式策略补建一次「世代」(优先采纳上报里的 `operationId`),不会改动其已有状态。
## 验证
`mvn -o test`:549 项,0 失败/错误/跳过。
新增关键用例(`TaskLifecycleIntegrationTest`,真实 SQLite 文件 + 真实事务/触发器/唯一约束,落在 `/tmp` 临时目录):
- 意图与占位行先于上游 `POST`;`SUBMITTING` 先于 `submit`。
- 同 `gid` 6 线程并发创建:上游只被提交一次、任务行不超过一条。
- 上游超时后落 `UNKNOWN`;再次点击创建也不会第二次 `POST`。
- 重启恢复把中断的 `SUBMITTING` 判为 `UNKNOWN`(任务行同步显示「提交未知」),安全释放 `RESERVED` 占位,且不改写 `operationId`、不重发。
- 额度写入失败时落库事务回滚,任务保持可对账的 `提交未知` + `UNKNOWN`。
- 重投/重试复用同一 `operationId`。
- 判定为拒绝的任务(`UNKNOWN` 等)绝不下发,也不退化为无 `operationId` 的旧命令。
- 删除失败保留任务行与意图,重试复用同一删除 ID;成功后清收藏、删行、留墓碑。删除意图 ID 不匹配时不得完成删除。
- 正在提交(`RESERVED`/`SUBMITTING`)的任务拒绝删除;释放预占只清匹配世代的记录与占位行。
- 落库严格匹配本次创建世代(expectedOperationId),被删除/换代后不得复活已删 gid。
- 墓碑永久忽略所有上报(含同一 `operationId`),只允许显式 `CREATE` 用新 `operationId` 重建。
- 旧节点不带 `operationId` 的连续上报保持可接受;新 v2 已知世代缺归属的上报仍被拒。
- 过期 `revision`、缺 `operationId`、待删除任务的上报都被丢弃;未知 gid 忽略但回执。
- 重复上报不重复通知;回执写入失败时 `apply` 返回不可回执结果。
`TaskProtocolV2CodecTest` 锁定类型 10..13 的帧头与字段名,并断言旧 `DownloadPostMessage` 没有混入新字段。
## 已知限制
未做双机真实故障演练。`UNKNOWN` 目前只提供状态与持久化记录:任务行会显示「提交未知」并拒绝自动重投/重试,对账与人工解决的接口由后续批次补齐。
+137
View File
@@ -0,0 +1,137 @@
# LionWebsite 一键登录(/login):实现说明
状态:**已全部完成并上线**。主站侧提交 `00e2715`(文档 `dfb647b`、`628f3ad`),
桌面端 `9b5bc94`;机器人侧由 `debian-qq-chatgpt` 于 host-vm103-debian-qq 完成
(见共享盘 `tasks/2026-09-21-lionwebsite-login-bot-command/`)。
关联待办:`td-20260921-a1b2c3`(已归档为
`todos/archive/done-td-20260921-a1b2c3-lionwebsite-login-tg.md`)。
用户已明确选择**接法 B:共享密钥自签票据**,理由是机器人到主站的网络调用不可靠。
因此主站**不提供**任何票据签发端点,也不需要机器人侧发起 HTTP 请求——机器人本地
用共享密钥算出票据即可。
## 1. 已经上线的东西(主站侧)
### 1.1 票据格式
```
v1.<签发时间的 Unix 秒级时间戳>.<HMAC-SHA256 十六进制小写>
```
签名原文为 `v1.<时间戳>`(固定前缀 `v1.` 加时间戳本身),密钥为共享密钥的 UTF-8 字节。
等价于 `hex(HMAC_SHA256(key, "v1." + timestamp))`。
### 1.2 校验规则
- 必须恰好三段,第一段必须是 `v1`;
- 签名比较用常量时间(`MessageDigest.isEqual`);
- 时间戳只接受 `[now - 300s, now]`,**未来时间戳一律拒绝**(防伪造者用远期时间换长期有效);
- 窗口内**允许重放**(用户明确接受),过期即失效;
- 默认 300 秒有效期,可用 `personal.login.ticket-ttl-seconds` 调整,下限 30 秒。
### 1.3 端点
| 端点 | 作用 |
| --- | --- |
| `GET /login?t=<票据>` | 校验票据 → 作废旧会话(防会话固定)→ 建新会话 → 302 跳 `/index`;失败 302 跳 `/denied` |
| `GET /login/logout` | 销毁会话 → 302 跳 `/denied` |
| `GET /denied` | 静态提示页:「请在机器人里发送 `/login`」 |
nginx 无需改动:`location /` 会把 `/login` 改写成后端 `/personal/login`。
### 1.4 会话与鉴权
- 登录成功下发 `JSESSIONID`,参数为 `Path=/`、`HttpOnly`、`SameSite=Lax`,14 天滑动过期。
**`Path=/` 是必须的**:nginx 会把 `/user` 改写成后端 `/personal/user`,若沿用容器按
请求路径推导的 `/personal`,浏览器判定 `/user` 不匹配就不会带会话,面板会一直 401。
- `PersonalInterceptor` 放行「有效会话 **或** `AuthCode=alone`」,两者都拒绝时返回 **401**。
`alone` 是留给下载器前端与存储节点 `/message2me` 推送的,本次**没有**退役它。
### 1.5 密钥
- 主站从环境变量 `PERSONAL_LOGIN_SECRET` 读取(`personal.login.secret`),
由 systemd drop-in `/etc/systemd/system/lionwebsite.service.d/login-secret.conf`
加载 `/etc/lionwebsite/login-secret.env`(`600` 权限,不在仓库里)。
- 密钥为空时**一律拒绝**票据,不会退化成放行。
- 轮换方式:改两侧配置并重启。轮换后旧票据在 300 秒内自然失效。
## 2. 机器人侧待实施(host-vm103-debian-qq)
### 2.1 生成票据
密钥必须与主站一致。**已由主人批准写入 PersonalHub 保险库**,引用为
`secret://services/service-lionwebsite/panel-login-secret`;机器人侧按既有秘密读取流程
申请取值,不要写进仓库或聊天记录。Python 侧计算方式:
```python
import hashlib, hmac, time
def lionwebsite_login_link(secret: str, base: str = "https://personal.lionwebsite.xyz") -> str:
stamp = str(int(time.time()))
signed = f"v1.{stamp}"
digest = hmac.new(secret.encode(), signed.encode(), hashlib.sha256).hexdigest()
return f"{base}/login?t=v1.{stamp}.{digest}"
```
Shell 等价写法(`openssl`)供人工验证用:
```sh
SIG=$(printf 'v1.%s' "$STAMP" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $NF}')
```
### 2.2 命令接入(已实施:复用既有 `/login`)
**复用既有 `/login`,不新建命令名;一条回复里同时给出两个登录链接。**
实现方式(vm103 实际做法,比原计划更稳):**机器人进程不持有密钥**。
签发走保险库固定动作 `CollabSecretService.use_secret`,值只交给进程内签名函数,
不返回、不落盘、不进日志;审计只记引用与动作名 `lionwebsite-login-link`。
QQ 侧 `plugins/login.py` 只经内部令牌 POST `/internal/panel-login`,
由 PersonalService 统一产出文案,因此 QQ 与 Telegram 不会漂移。
- Telegram:`telegram_panel_login()` 的返回文案改为同时给出 PersonalHub 面板链接与
LionWebsite 面板链接,`TELEGRAM_COMMANDS` 里 `login` 的描述同步更新。
- QQ(LionQQBot):既有 `/login`(别名 `/登录`)返回同样两条链接,共用同一套生成逻辑。
- PersonalHub 未完整配置(缺 password / session_secret / TOTP)时,现有逻辑会回一句
配置提示;此时**仍要发出 LionWebsite 链接**,不要让一条链接的失败带掉另一条。
- 帮助菜单(Telegram `/help` 与 QQ 菜单)中 `/login` 的描述同步更新。
- 文案不要写成「两条都只能用一次」:PersonalHub 链接是一次性的,
LionWebsite 票据是 300 秒窗口内可重放。
示例文案:
```
已生成登录链接:
· Personal Hub:[打开面板](< PersonalHub 链接 >)
· LionWebsite:[打开面板](< LionWebsite 链接 >)
两条链接 5 分钟内有效,请勿转发。
```
### 2.3 与 PersonalHub 现有 /login 的关系
PersonalHub 的 `/login` 原本只签发**它自己面板**的票据
(`OneTimePanelLoginTickets` + `/panel/telegram-login`)。改造时保留这条,
再在同一回复里附加 LionWebsite 链接;两者的票据机制与有效期互不影响。
## 3. 交付与复验
- 机器人侧提交状态:**两个机器人仓库的改动尚未提交 Git**(`v103` 侧待主人确认后
按项目约定提交,注意 PersonalHub 工作区另有前一轮未提交改动,勿混提)。
- `us9929main-chatgpt` 的独立复验(2026-09-21):
- 票据边界实测:`now` 与 `now-290s` 放行,`now-301s`、`now-400s`、`now+60s` 均落 `/denied`;
篡改签名与畸形串同样落 `/denied`。
- 密钥明文扫描:抓取两个任务目录全部正文与线程消息共 17 KiB,未出现密钥明文,
只有 4 处 `secret://` 引用。
- 内部接口鉴权:`POST /internal/panel-login` 无令牌与错令牌均返回 403。
- PersonalHub `/health` 返回 200;共享盘 waiting 指针已消费,队列已清空。
## 4. 可选收尾(未做,需要时另开任务)
- 移动端 `PrivateMainForMobile` 仍写死 `authCode: "alone"`。后端两种方式都接受,
所以它能用;要一并切到会话,照搬 `PrivateMain/src/store/index.js` 的改法即可。
- `sourcecode/storageNode` 有两处硬编码 `alone`(`CustomUtil.java` 的 `/message2me`
与 `MultiThreadedHTTPServer.java` 的本机鉴权),属另一仓库,需单独发布后才能退役字面量。
- 概览页的「本机订阅」链接 `https://personal.lionwebsite.xyz/sub/self` 目前是 404,
后端没有对应映射,属历史遗留,与本次登录改造无关。
+12 -18
View File
@@ -6,8 +6,8 @@
────────────────────────────────────────────────────────────────────────────── ──────────────────────────────────────────────────────────────────────────────
名称: LionWebsite 名称: LionWebsite
技术栈: Spring Boot 4.1.1, Java 21 字节码(生产 JDK 25), Maven, SQLite, MyBatis 4, Netty 4.2 技术栈: Spring Boot 4.1.1, Java 21 字节码(生产 JDK 25), Maven, SQLite, MyBatis 4, Netty 4.2
定位: 个人/私有 Web 应用,兼具 E-Hentai 画廊下载管理、个人文件服务、 定位: 个人/私有 Web 应用,兼具 E-Hentai 画廊下载管理、代理订阅
代理订阅管理等功能。 管理等功能。
运行端口: 8888 运行端口: 8888
数据库: 双 SQLite 数据库 — LionWebsite.db (主库) + cache.db (缓存库) 数据库: 双 SQLite 数据库 — LionWebsite.db (主库) + cache.db (缓存库)
构建目标: 当前以 JVM/JAR 运行;保留 GraalVM Native Image 配置但尚未在 JDK 25 完成原生验证 构建目标: 当前以 JVM/JAR 运行;保留 GraalVM Native Image 配置但尚未在 JDK 25 完成原生验证
@@ -29,8 +29,8 @@ src/main/java/com/lion/lionwebsite/
├── Controller/ ├── Controller/
│ ├── GalleryManageController.java 画廊任务 CRUD、收藏、图片在线缓存 /GalleryManage │ ├── GalleryManageController.java 画廊任务 CRUD、收藏、图片在线缓存 /GalleryManage
│ ├── QueryController.java E-Hentai 搜索代理 /query │ ├── QueryController.java E-Hentai 搜索代理 /query
│ ├── PublicController.java 根路由、IP、订阅、文件分享、验证 /、/ip、/sub/、/GetFile/、/validate │ ├── PublicController.java 根路由、IP、订阅、验证 /、/ip、/sub/、/validate
│ ├── PersonalController.java 个人文件管理 /personal/ │ ├── PersonalController.java 个人管理 (订阅更新/最后更新时间/IP/留言) /personal/
│ ├── SubController.java 订阅绑定管理 /personal/subBind/ │ ├── SubController.java 订阅绑定管理 /personal/subBind/
│ └── UserController.java 用户管理 /personal/user │ └── UserController.java 用户管理 /personal/user
│ │
@@ -41,8 +41,8 @@ src/main/java/com/lion/lionwebsite/
│ ├── PushService.java Telegram Bot 通知 (admin 告警) │ ├── PushService.java Telegram Bot 通知 (admin 告警)
│ ├── QueryService.java E-Hentai 搜索 + 缩略图代理缓存 (转 AVIF) │ ├── QueryService.java E-Hentai 搜索 + 缩略图代理缓存 (转 AVIF)
│ ├── LocalServiceImpl.java 定时任务 (连接检测/额度重置/Cookie验证/订阅更新/缩略图清理) │ ├── LocalServiceImpl.java 定时任务 (连接检测/额度重置/Cookie验证/订阅更新/缩略图清理)
│ ├── PublicServiceImpl.java IP 记录、分享码文件获取、授权码修改 │ ├── PublicServiceImpl.java IP 记录、授权码修改、身份查询
│ ├── PersonalServiceImpl.java 文件管理 (浏览/上传/下载/分享/压缩/删除/TAR打包) │ ├── PersonalServiceImpl.java 订阅最后更新时间、家里 IP、留言转发
│ ├── SubService.java 代理订阅绑定/重置/查询/更新记录 │ ├── SubService.java 代理订阅绑定/重置/查询/更新记录
│ ├── CollectService.java 画廊收藏/取消收藏 │ ├── CollectService.java 画廊收藏/取消收藏
│ └── UserServiceImpl.java 用户 CRUD + 授权码管理 │ └── UserServiceImpl.java 用户 CRUD + 授权码管理
@@ -52,7 +52,6 @@ src/main/java/com/lion/lionwebsite/
│ │ ├── GalleryMapper.java gallery 表 CRUD │ │ ├── GalleryMapper.java gallery 表 CRUD
│ │ ├── UserMapper.java user 表 CRUD │ │ ├── UserMapper.java user 表 CRUD
│ │ ├── CollectMapper.java collect 表 CRUD │ │ ├── CollectMapper.java collect 表 CRUD
│ │ ├── ShareFileMapper.java ShareFile 表 CRUD
│ │ ├── CustomConfigurationMapper.java 配置键值对读写 │ │ ├── CustomConfigurationMapper.java 配置键值对读写
│ │ └── SubMapper.java 订阅绑定 & 更新记录 │ │ └── SubMapper.java 订阅绑定 & 更新记录
│ └── cache/ 缓存库 Mapper │ └── cache/ 缓存库 Mapper
@@ -66,7 +65,6 @@ src/main/java/com/lion/lionwebsite/
│ ├── GidToKey.java 画廊 GID → MPV Key 映射 │ ├── GidToKey.java 画廊 GID → MPV Key 映射
│ ├── ImageKeyCache.java 图片 key 缓存 (gid, page, imgkey) │ ├── ImageKeyCache.java 图片 key 缓存 (gid, page, imgkey)
│ ├── CustomConfiguration.java 配置键常量定义 │ ├── CustomConfiguration.java 配置键常量定义
│ ├── ShareFile.java 文件分享 (ShareCode, FilePath, ExpireTime)
│ ├── SubBind.java 订阅绑定 (key, user) │ ├── SubBind.java 订阅绑定 (key, user)
│ ├── SubUpdateRecord.java 订阅更新记录 (ip, UA, time, location) │ ├── SubUpdateRecord.java 订阅更新记录 (ip, UA, time, location)
│ └── PageNameCache.java 页面名缓存 (gid, page, pageName) │ └── PageNameCache.java 页面名缓存 (gid, page, pageName)
@@ -115,13 +113,11 @@ src/main/java/com/lion/lionwebsite/
- 支持图片在线预览: 缓存 MPV key → 按需下载单页 → 转为 AVIF 格式 - 支持图片在线预览: 缓存 MPV key → 按需下载单页 → 转为 AVIF 格式
- 画廊收藏/取消收藏 - 画廊收藏/取消收藏
2. 个人文件服务 2. 个人管理服务
- 文件浏览器: 按路径浏览文件/文件夹,显示大小、分享状态 - 手动触发订阅更新,返回最近更新时间与已记录的家庭 IP
- 上传: MultipartFile 上传到指定路径 - 管理员留言经 PushService 转发到 Telegram
- 下载: 支持 HTTP Range 断点续传
- 分享: 生成 8 位随机分享码,设置过期时间,可延长/取消 (文件浏览/上传/下载/分享/压缩/删除已在 td-20260921-a1b2c3 中整体下线)
- 压缩: 异步 TAR 打包目录
- 删除文件/文件夹
3. E-Hentai 搜索代理 3. E-Hentai 搜索代理
- 代理搜索 exhentai.org,返回格式化结果 (含缩略图 URL) - 代理搜索 exhentai.org,返回格式化结果 (含缩略图 URL)
@@ -141,7 +137,6 @@ src/main/java/com/lion/lionwebsite/
- 每 30 分钟: 检测存储节点连接,断开则自动重连 - 每 30 分钟: 检测存储节点连接,断开则自动重连
- 每周一 4:00: 重置每周下载额度 - 每周一 4:00: 重置每周下载额度
- 每天 0:00: 验证 E-Hentai Cookie 有效性 - 每天 0:00: 验证 E-Hentai Cookie 有效性
- 每天 4:00: 清理过期分享码
- 每 24 小时: 更新代理订阅配置文件 - 每 24 小时: 更新代理订阅配置文件
- 每周一 4:00: 清理缩略图缓存 (保留最近 10000 个) - 每周一 4:00: 清理缩略图缓存 (保留最近 10000 个)
@@ -151,13 +146,12 @@ src/main/java/com/lion/lionwebsite/
- /personal 和 /remote 路径限 AuthCode="alone" 用户 - /personal 和 /remote 路径限 AuthCode="alone" 用户
- HumanInterceptor 拒绝无 User-Agent 请求 - HumanInterceptor 拒绝无 User-Agent 请求
- AdaptorFilter 记录所有请求日志 (IP/路径/UA/时间) - AdaptorFilter 记录所有请求日志 (IP/路径/UA/时间)
- 分享码失败黑名单 (连续失败则屏蔽)
六、依赖 六、依赖
────────────────────────────────────────────────────────────────────────────── ──────────────────────────────────────────────────────────────────────────────
spring-boot-starter-webmvc, spring-boot-starter-websocket, mybatis-spring-boot-starter 4.1 spring-boot-starter-webmvc, spring-boot-starter-websocket, mybatis-spring-boot-starter 4.1
jsoup (HTML 解析), hutool-all (工具集), sqlite-jdbc (数据库) jsoup (HTML 解析), hutool-all (工具集), sqlite-jdbc (数据库)
httpclient5 (HTTP 请求), commons-compress (TAR 打包) httpclient5 (HTTP 请求)
commons-io, commons-lang3, netty-all (TCP 通信) commons-io, commons-lang3, netty-all (TCP 通信)
java-telegram-bot-api (Telegram Bot), graalvm native-maven-plugin (AOT) java-telegram-bot-api (Telegram Bot), graalvm native-maven-plugin (AOT)
+28 -2
View File
@@ -17,6 +17,11 @@
<java.version>21</java.version> <java.version>21</java.version>
</properties> </properties>
<dependencies> <dependencies>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>2.6</version>
</dependency>
<dependency> <dependency>
<groupId>org.springframework.boot</groupId> <groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc</artifactId> <artifactId>spring-boot-starter-webmvc</artifactId>
@@ -86,7 +91,28 @@
<dependency> <dependency>
<groupId>io.netty</groupId> <groupId>io.netty</groupId>
<artifactId>netty-all</artifactId> <!--
只保留实际用到的模块。原先的 netty-all 是聚合 pom,会拖进 50 个模块
(含 5 个平台的 native-quic、aarch64/riscv64/osx 的 epoll/kqueue/io_uring、
codec-http3/mqtt/redis/smtp/stomp/xml/protobuf 等),
而本项目只用 Bootstrap/NioEventLoopGroup/NioSocketChannel/ByteBuf/
ByteToMessageCodec/LengthFieldBasedFrameDecoder/LoggingHandler/Promise。
transport 提供 Bootstrap/EventLoop/Channel,codec-base 提供
ByteToMessageCodec 与 LengthFieldBasedFrameDecoder,handler 提供
LoggingHandler(RemoteService 的协议调试日志用)。
buffer/common/resolver 由它们传递引入。
-->
<artifactId>netty-transport</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-base</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
</dependency> </dependency>
<dependency> <dependency>
@@ -144,7 +170,7 @@
<imageName>lionwebsite</imageName> <imageName>lionwebsite</imageName>
<buildArgs> <buildArgs>
<arg>-H:+ReportExceptionStackTraces</arg> <arg>-H:+ReportExceptionStackTraces</arg>
<arg>--initialize-at-build-time=org.apache.commons.logging.LogFactory,org.apache.commons.logging.LogFactoryService,org.sqlite.util.ProcessRunner</arg> <arg>--initialize-at-build-time=org.apache.commons.logging.LogFactory,org.apache.commons.logging.LogFactoryService,org.apache.commons.logging.impl.WeakHashtable,org.sqlite.util.ProcessRunner</arg>
</buildArgs> </buildArgs>
<metadataRepository> <metadataRepository>
<enabled>true</enabled> <enabled>true</enabled>
+151
View File
@@ -0,0 +1,151 @@
#!/usr/bin/env bash
# 图片索引(cache.db)迁移:为缓存表补唯一约束与查询索引。
#
# 设计约束:
# * 默认 dry-run,只读检查;必须显式 --apply 才执行写入。
# * 必须显式 --db 指定数据库文件,不读取任何运行时配置,也不推断生产路径。
# * 写库用 sqlite3 -cmd '.bail on',任一语句出错立即以非零码退出,事务不提交。
# * 只以读写模式打开目标库(文件必须已存在,不会新建);dry-run 全用 -readonly。
# * 备份用 SQLite 自身的 .backup(online backup API),而不是 cp:
# cp 会漏掉 WAL 里尚未 checkpoint 的已提交内容。
# * 备份后立即校验备份可读且含两张缓存表,失败则中止并删除无效备份。
# * 不健康(不完整/冲突)的 gid 整组索引会被删除,由下一次在线阅读重新抓取。
# * 唯一约束建在唯一索引上:若清理后仍有重复,建索引会失败并整体回滚。
#
# 用法:
# scripts/migrate_image_index_cache.sh --db /tmp/cache-copy.db # dry-run(只读)
# scripts/migrate_image_index_cache.sh --db /tmp/cache-copy.db --apply # 执行迁移
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SQL_FILE="$SCRIPT_DIR/migrate_image_index_cache.sql"
db_path=""
apply="no"
usage() {
sed -n '2,30p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
}
while [[ $# -gt 0 ]]; do
case "$1" in
--db) db_path="${2:-}"; shift 2 ;;
--apply) apply="yes"; shift ;;
-h|--help) usage; exit 0 ;;
*)
echo "未知参数: $1" >&2
exit 2 ;;
esac
done
if [[ -z "$db_path" ]]; then
echo "必须显式指定缓存库: --db <cache.db>;本脚本不会自动推断数据库位置。" >&2
exit 2
fi
if [[ ! -f "$db_path" ]]; then
echo "数据库不存在: $db_path" >&2
exit 2
fi
if [[ ! -f "$SQL_FILE" ]]; then
echo "缺少 SQL: $SQL_FILE" >&2
exit 2
fi
if ! command -v sqlite3 >/dev/null 2>&1; then
echo "缺少 sqlite3 命令行工具" >&2
exit 2
fi
if [[ "$(sqlite3 -readonly "$db_path" \
"select count(*) from sqlite_master where type='table' and name in ('gidToKey','ImageKeyCache');")" != "2" ]]; then
echo "缓存表缺失:$db_path 中找不到 gidToKey 与 ImageKeyCache。" >&2
exit 2
fi
# 体检统计,与 SQL 里实际清理的范围一致:
# bad_gids 将被整组删除的 gid 个数(去重),覆盖:
# 重复头部 / 行数不符 / 页码去重后不足 / 页码区间异常 /
# 空 imgkey / 空 key / 非法 pages
# bad_gid_rows 这些 gid 占用的头部行数(重复头部的 gid 会多于 1 行)
# duplicate_pages 重复的 (gid,page) 组数,仅用于展示,会随所属 gid 一并清理
# orphan_gids 有页 key 但无头部记录的 gid 个数
read_stats() {
sqlite3 -readonly "$db_path" "
select 'header_rows', count(*) from gidToKey;
select 'page_rows', count(*) from ImageKeyCache;
select 'duplicate_gids', count(*) from (select gid from gidToKey group by gid having count(*) > 1);
select 'duplicate_pages', count(*) from (select gid,page from ImageKeyCache group by gid,page having count(*) > 1);
select 'bad_gids', count(distinct g.gid) from gidToKey g
where g.pages is null or g.pages <= 0
or g.key is null or trim(g.key) = ''
or (select count(*) from ImageKeyCache i where i.gid = g.gid) <> g.pages
or (select count(distinct i.page) from ImageKeyCache i where i.gid = g.gid) <> g.pages
or (select min(i.page) from ImageKeyCache i where i.gid = g.gid) <> 1
or (select max(i.page) from ImageKeyCache i where i.gid = g.gid) <> g.pages
or exists (select 1 from ImageKeyCache i
where i.gid = g.gid and (i.imgkey is null or trim(i.imgkey) = ''));
select 'bad_gid_rows', coalesce(sum(c), 0) from (
select count(*) as c from gidToKey g where g.gid in (
select g2.gid from gidToKey g2 where g2.pages is null or g2.pages <= 0
or g2.key is null or trim(g2.key) = ''
or (select count(*) from ImageKeyCache i where i.gid = g2.gid) <> g2.pages
or (select count(distinct i.page) from ImageKeyCache i where i.gid = g2.gid) <> g2.pages
or (select min(i.page) from ImageKeyCache i where i.gid = g2.gid) <> 1
or (select max(i.page) from ImageKeyCache i where i.gid = g2.gid) <> g2.pages
or exists (select 1 from ImageKeyCache i
where i.gid = g2.gid and (i.imgkey is null or trim(i.imgkey) = '')))
group by g.gid);
select 'orphan_gids', count(distinct gid) from ImageKeyCache
where gid not in (select gid from gidToKey);
select 'strict_index', count(*) from sqlite_master
where type='index' and name='ux_ImageKeyCache_gid_page';
"
}
mode_label="dry-run(只读)"
if [[ "$apply" == "yes" ]]; then mode_label="apply(读写)"; fi
echo "=== 缓存库图片索引迁移($mode_label) ==="
echo "数据库: $db_path"
echo "--- 迁移前体检 ---"
read_stats | while IFS='|' read -r name value; do
printf ' %-16s %s\n' "$name" "$value"
done
if [[ "$apply" != "yes" ]]; then
echo
echo "dry-run:未做任何写入。确认无误后加 --apply 执行迁移。"
echo "提示:请先在 SQLite 副本上演练,副本应这样生成(含 WAL 内容):"
echo " sqlite3 -readonly <生产库> \".backup '<副本路径>'\""
exit 0
fi
# ---- 备份:SQLite online backup,包含 WAL 中已提交但未 checkpoint 的内容 ----
backup="$db_path.before-image-index.$(date -u +%Y%m%dT%H%M%SZ)"
if ! sqlite3 "$db_path" ".backup '$backup'"; then
echo "备份失败,未做任何修改: $backup" >&2
exit 3
fi
if [[ ! -s "$backup" ]]; then
echo "备份文件为空,未做任何修改: $backup" >&2
exit 3
fi
if [[ "$(sqlite3 -readonly "$backup" \
"select count(*) from sqlite_master where type='table' and name in ('gidToKey','ImageKeyCache');")" != "2" ]]; then
echo "备份校验失败(缺少缓存表),未做任何修改,已删除无效备份。" >&2
rm -f -- "$backup"
exit 3
fi
echo "已备份(SQLite .backup): $backup"
# ---- 执行迁移:.bail on 保证任一语句失败即退出,事务不提交 ----
sqlite3 -cmd '.bail on' "$db_path" < "$SQL_FILE"
echo "--- 迁移后体检 ---"
read_stats | while IFS='|' read -r name value; do
printf ' %-16s %s\n' "$name" "$value"
done
echo "图片索引迁移完成: $db_path"
+67
View File
@@ -0,0 +1,67 @@
-- 图片索引缓存迁移:为 cache.db 的两张索引表补唯一约束与查询索引。
--
-- 目标(对应本次重写的三条不变量):
-- 1. gid 唯一 —— 同一本画廊只能有一条 gidToKey 记录;
-- 2. (gid,page) 唯一 —— 同一页只能有一个 imgkey,同时天然满足按页查询的索引需求;
-- 3. 只保留「完整」的索引 —— 半截或冲突的 gid 整组删除,让下一次在线阅读重新抓取。
--
-- 为什么用唯一索引而不是重建表:
-- * 唯一索引对 SQLite 与唯一约束等价,都会拒绝重复写入;
-- * 不需要搬表、改列,旧版本二进制仍可读写同一份数据;
-- * 回滚只需 drop index,最坏情况也只需回到脚本自动生成的备份文件。
--
-- 幂等:所有重建索引都带 IF NOT EXISTS;清理语句只按集合删除,重复执行结果相同。
-- 安全:本脚本只做 DELETE 与 CREATE INDEX,不建表、不改列、不碰 gidToKey/ImageKeyCache
-- 之外的任何对象。必须在事务中整体提交,调用方使用 sqlite3 -bail 保证任一语句
-- 失败即终止、事务回滚。
--
-- 执行前请务必先备份数据库文件(迁移脚本 migrate_image_index_cache.sh --apply 会自动备份)。
BEGIN IMMEDIATE;
-- ---------- 1. 冲突 gid:同一 gid 有多条头部记录 ----------
-- 无法判断哪一条正确,整组移除,由下次访问重抓。
CREATE TEMP TABLE __conflict_gid AS
SELECT gid FROM gidToKey GROUP BY gid HAVING COUNT(*) > 1;
DELETE FROM ImageKeyCache WHERE gid IN (SELECT gid FROM __conflict_gid);
DELETE FROM gidToKey WHERE gid IN (SELECT gid FROM __conflict_gid);
-- ---------- 2. 半截或非法 gid:头部与页 key 不一致 ----------
-- 判定同时覆盖:页码数量不符、页码不连续(跳号/重复)、区间不是 1..pages、
-- 头部字段为空、以及存在空的 imgkey 行。
CREATE TEMP TABLE __broken_gid AS
SELECT g.gid
FROM gidToKey g
WHERE g.pages IS NULL OR g.pages <= 0
OR g.key IS NULL OR trim(g.key) = ''
OR (SELECT COUNT(*) FROM ImageKeyCache i WHERE i.gid = g.gid) <> g.pages
OR (SELECT COUNT(DISTINCT i.page) FROM ImageKeyCache i WHERE i.gid = g.gid) <> g.pages
OR (SELECT MIN(i.page) FROM ImageKeyCache i WHERE i.gid = g.gid) <> 1
OR (SELECT MAX(i.page) FROM ImageKeyCache i WHERE i.gid = g.gid) <> g.pages
OR EXISTS (SELECT 1 FROM ImageKeyCache i
WHERE i.gid = g.gid AND (i.imgkey IS NULL OR trim(i.imgkey) = ''));
DELETE FROM ImageKeyCache WHERE gid IN (SELECT gid FROM __broken_gid);
DELETE FROM gidToKey WHERE gid IN (SELECT gid FROM __broken_gid);
-- ---------- 3. 孤儿页 key:没有对应头部记录 ----------
DELETE FROM ImageKeyCache WHERE gid NOT IN (SELECT gid FROM gidToKey);
-- ---------- 4. 唯一约束 ----------
-- 到这里若仍有重复,说明上面的判定有遗漏;CREATE UNIQUE INDEX 会失败,
-- 配合 -bail 让整个事务回滚,不会留下「一半约束」的库。
CREATE UNIQUE INDEX IF NOT EXISTS ux_gidToKey_gid
ON gidToKey (gid);
CREATE UNIQUE INDEX IF NOT EXISTS ux_ImageKeyCache_gid_page
ON ImageKeyCache (gid, page);
-- 按 gid 统计页数会用到 (gid,page) 前缀,按页取 key 也走同一索引,
-- 因此不再单独建 page 索引,避免重复占用写放大。
DROP TABLE __conflict_gid;
DROP TABLE __broken_gid;
COMMIT;
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
set -euo pipefail
db_path="${1:-LionWebsite.db}"
if [[ ! -f "$db_path" ]]; then
echo "数据库不存在: $db_path" >&2
exit 2
fi
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet lionwebsite 2>/dev/null; then
echo "检测到 lionwebsite.service 仍在运行;请先停止服务再迁移,避免写冲突。" >&2
exit 3
fi
if pgrep -f 'java .*lionwebsite\.jar' >/dev/null 2>&1; then
echo "检测到主站进程仍在运行;请先停止服务再迁移,避免写冲突。" >&2
exit 3
fi
backup="${db_path}.before-refresh-schedule.$(date -u +%Y%m%dT%H%M%SZ)"
cp "$db_path" "$backup"
echo "已备份: $backup"
sqlite3 "$db_path" < "$(dirname "$0")/migrate_subscription_refresh.sql"
echo "--- 校验 ---"
sqlite3 "$db_path" "select count(*) as accounts, sum(next_refresh_at is null) as missing_schedule from subscription_account;"
echo "订阅刷新计划迁移完成: $db_path"
+26
View File
@@ -0,0 +1,26 @@
-- 为每账号分散刷新增加计划字段。
-- 纯新增列:旧版本二进制忽略这两列,因此回滚程序时不必回滚数据库。
BEGIN;
-- next_refresh_at:该子账号下一次应刷新的时刻,Epoch 毫秒。
-- 用整数存时刻,避免 SQLite 文本时间戳被按本地时区解释(实证存在 8 小时偏差)。
-- last_success_epoch:最近一次成功刷新的真实时刻,Epoch 毫秒。
-- 既有的 last_success_at 保持原样,仅供页面展示,不参与调度判断。
ALTER TABLE subscription_account ADD COLUMN next_refresh_at INTEGER;
ALTER TABLE subscription_account ADD COLUMN last_success_epoch INTEGER;
-- 回填 last_success_epoch:SQLite 的 strftime 把文本时间按 UTC 解释,而
-- CURRENT_TIMESTAMP 写入的正是 UTC,因此这里能得到正确时刻(JDBC 读取则会偏 8 小时)。
-- 必须回填:否则所有既有账号都会被当成「从未成功过」,迁移后立刻集中补刷一遍,
-- 正好复现本次改造要消除的爆发。
UPDATE subscription_account
SET last_success_epoch = CAST(strftime('%s', last_success_at) AS INTEGER) * 1000
WHERE last_success_epoch IS NULL AND last_success_at IS NOT NULL;
-- 已有账号在窗口内按 id 错开,避免迁移后同一 tick 集中开火。
-- 以迁移时刻为基准按 37 分钟步长铺开(12 个账号约 7.4 小时排完)。
UPDATE subscription_account
SET next_refresh_at = (CAST(strftime('%s', 'now') AS INTEGER) * 1000) + (id * 37 * 60 * 1000)
WHERE next_refresh_at IS NULL;
COMMIT;
+31
View File
@@ -0,0 +1,31 @@
-- 任务操作记录表(协议 v2:先持久化意图,再产生外部副作用)
--
-- 这是显式迁移脚本,不会被应用自动执行,也不对生产库运行。
-- 执行前请备份:sqlite3 LionWebsite.db ".backup LionWebsite.db.before-task-operation"
--
-- 幂等:可重复执行。
create table if not exists task_operation
(
gid integer not null
constraint task_operation_pk
primary key,
operationId varchar(64),
deleteOperationId varchar(64),
state varchar(20) not null,
lastRevision integer not null default 0,
lastReportId varchar(64),
updateTime long not null default 0
);
-- 节点上报消费只按 gid 命中主键,这里额外为「按状态找待处理意图」提供查询入口。
create index if not exists idx_task_operation_state on task_operation (state);
-- gallery.gid 目前只有 table 级主键约束(部分历史库甚至是自增主键的衍生约束),
-- 这里显式建立唯一索引,保证同一 gid 不会出现两条任务行:
-- 「同一 gid 只有一个有效意图」是 v2 生命周期成立的前提。
-- 若该语句因历史重复数据失败,请先人工确认重复行来源再处理,不要直接删除。
create unique index if not exists idx_gallery_gid_unique on gallery (gid);
-- 旧任务行没有操作记录:启动/查询时按 gid 惰性补建为 SUBMITTED(已在上游提交过),
-- 不需要在迁移里批量写入,避免把正在进行的任务判成 UNKNOWN。
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env python3
"""Explicit task-v2 schema migration. Default is read-only; --apply backs up first."""
import argparse
import datetime
import json
from pathlib import Path
import sqlite3
def migrate(database: Path, apply: bool = False) -> dict:
if not database.is_file():
raise ValueError("database must already exist")
with sqlite3.connect(database.resolve().as_uri() + ("?mode=rw" if apply else "?mode=ro"), uri=True) as db:
if not db.execute("pragma table_info(gallery)").fetchall():
raise ValueError("required table missing: gallery")
duplicates = db.execute("select count(*) from (select gid from gallery group by gid having count(*)>1)").fetchone()[0]
if duplicates:
raise ValueError("duplicate gallery gid; reconcile before migration")
present = bool(db.execute("pragma table_info(task_operation)").fetchall())
result = {"apply": apply, "create_operation_table": not present}
if not apply:
return result
stamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
backup = database.with_name(database.name + ".before-task-v2." + stamp)
with sqlite3.connect(backup) as copy:
db.backup(copy)
backup.chmod(0o600)
sql = Path(__file__).with_name("migrate_task_operation_v2.sql").read_text()
try:
db.executescript("BEGIN IMMEDIATE;\n" + sql + "\nCOMMIT;")
except BaseException:
db.rollback()
raise
result["backup"] = str(backup)
result["operation_rows"] = db.execute("select count(*) from task_operation").fetchone()[0]
return result
if __name__ == "__main__":
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("database", type=Path)
parser.add_argument("--apply", action="store_true")
arguments = parser.parse_args()
print(json.dumps(migrate(arguments.database, arguments.apply), ensure_ascii=False))
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env python3
"""Add stable user identity to subscription bindings and access records.
The default invocation only reports schema changes. --apply creates a consistent
SQLite backup before the transaction. Run against an isolated database first.
"""
import argparse
import datetime
import json
from pathlib import Path
import sqlite3
def migrate(db_path: Path, apply: bool = False) -> dict:
if not db_path.is_file():
raise ValueError("database must already exist")
uri = db_path.resolve().as_uri() + ("?mode=rw" if apply else "?mode=ro")
with sqlite3.connect(uri, uri=True) as db:
changes = []
for table in ("sub_bind", "sub_update_record"):
columns = {row[1] for row in db.execute(f'PRAGMA table_info("{table}")')}
if not columns:
raise ValueError(f"required table missing: {table}")
if "user_id" not in columns:
changes.append(f"add {table}.user_id")
duplicates = db.execute(
"select count(*) from (select username from User group by username having count(*) > 1)"
).fetchone()[0]
if duplicates:
raise ValueError("ambiguous duplicate usernames; resolve identities before migration")
result = {"apply": apply, "schema_changes": changes}
if not apply:
return result
stamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
backup = db_path.with_name(db_path.name + ".before-user-links." + stamp)
with sqlite3.connect(backup) as snapshot:
db.backup(snapshot)
backup.chmod(0o600)
db.execute("BEGIN IMMEDIATE")
try:
for change in changes:
table = change.split()[1].split(".")[0]
db.execute(f'ALTER TABLE "{table}" ADD COLUMN user_id INTEGER REFERENCES User(id)')
for table in ("sub_bind", "sub_update_record"):
db.execute(f'UPDATE "{table}" SET user_id=(SELECT id FROM User WHERE username="{table}".user) WHERE user_id IS NULL')
db.execute("CREATE UNIQUE INDEX IF NOT EXISTS User_username_uindex ON User(username)")
db.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_sub_bind_user_id ON sub_bind(user_id) WHERE user_id IS NOT NULL")
db.execute("CREATE INDEX IF NOT EXISTS idx_sub_record_user_id ON sub_update_record(user_id, id)")
db.commit()
except BaseException:
db.rollback()
raise
result["backup"] = str(backup)
result["unresolved_bindings"] = db.execute("select count(*) from sub_bind where user_id is null").fetchone()[0]
result["unresolved_records"] = db.execute("select count(*) from sub_update_record where user_id is null").fetchone()[0]
return result
if __name__ == "__main__":
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("database", type=Path)
parser.add_argument("--apply", action="store_true")
args = parser.parse_args()
print(json.dumps(migrate(args.database, args.apply), ensure_ascii=False))
+55
View File
@@ -0,0 +1,55 @@
import importlib.util
from pathlib import Path
import sqlite3
import tempfile
import unittest
spec = importlib.util.spec_from_file_location("migration", Path(__file__).with_name("migrate_user_links.py"))
migration = importlib.util.module_from_spec(spec)
spec.loader.exec_module(migration)
class MigrationTest(unittest.TestCase):
def test_dry_run_idempotent_migration_and_rename(self):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "test.db"
with sqlite3.connect(path) as db:
db.executescript("""
CREATE TABLE User(id INTEGER PRIMARY KEY, username TEXT);
CREATE TABLE sub_bind(key TEXT PRIMARY KEY, user TEXT, subscription_account_id INTEGER);
CREATE TABLE sub_update_record(id INTEGER PRIMARY KEY, user TEXT, location TEXT);
INSERT INTO User VALUES(1,'old-name');
INSERT INTO sub_bind VALUES('example-key','old-name',2);
INSERT INTO sub_update_record VALUES(10,'old-name','unknown');
INSERT INTO sub_update_record VALUES(11,'old-name','unknown');
""")
before = path.read_bytes()
self.assertEqual(2, len(migration.migrate(path)["schema_changes"]))
self.assertEqual(before, path.read_bytes())
self.assertEqual(0, migration.migrate(path, True)["unresolved_bindings"])
self.assertEqual([], migration.migrate(path, True)["schema_changes"])
with sqlite3.connect(path) as db:
db.execute("update User set username='new-name' where id=1")
self.assertEqual((1,), db.execute("select user_id from sub_bind").fetchone())
# A slow first lookup must not overwrite the second request's row.
db.execute("update sub_update_record set location='first-location' where id=10")
self.assertEqual(("unknown",), db.execute("select location from sub_update_record where id=11").fetchone())
def test_ambiguous_names_fail_without_modifying_database(self):
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "test.db"
with sqlite3.connect(path) as db:
db.executescript("""
CREATE TABLE User(id INTEGER PRIMARY KEY, username TEXT);
CREATE TABLE sub_bind(user TEXT);
CREATE TABLE sub_update_record(user TEXT);
INSERT INTO User VALUES(1,'same'),(2,'same');
""")
before = path.read_bytes()
with self.assertRaises(ValueError):
migration.migrate(path, True)
self.assertEqual(before, path.read_bytes())
if __name__ == "__main__":
unittest.main()
@@ -2,6 +2,7 @@ package com.lion.lionwebsite.Configuration;
import com.lion.lionwebsite.Domain.*; import com.lion.lionwebsite.Domain.*;
import com.lion.lionwebsite.Message.*; import com.lion.lionwebsite.Message.*;
import com.lion.lionwebsite.Service.SubscriptionRefreshPlanner;
import com.lion.lionwebsite.Util.GalleryUtil; import com.lion.lionwebsite.Util.GalleryUtil;
import com.pengrad.telegrambot.TelegramBot; import com.pengrad.telegrambot.TelegramBot;
import com.pengrad.telegrambot.model.*; import com.pengrad.telegrambot.model.*;
@@ -14,14 +15,21 @@ import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import java.time.Clock;
import java.time.Duration;
import java.util.Random;
@Configuration @Configuration
@RegisterReflectionForBinding(classes = {CustomConfiguration.class, GidToKey.class, ImageKeyCache.class, @RegisterReflectionForBinding(classes = {CustomConfiguration.class, GidToKey.class, ImageKeyCache.class,
GalleryForQuery.class, Gallery.class, GalleryTask.class, HikariConfig.class, GalleryForQuery.class, Gallery.class, GalleryTask.class, HikariConfig.class,
PageNameCache.class, ShareFile.class, User.class, PageNameCache.class, User.class,
SendResponse.class, Message.class, com.pengrad.telegrambot.model.User.class, SendResponse.class, Message.class, com.pengrad.telegrambot.model.User.class,
Chat.class, MessageEntity.class, Chat.class, MessageEntity.class,
AbstractMethodError.class, DeleteGalleryMessage.class, DownloadPostMessage.class, DownloadStatusMessage.class, AbstractMethodError.class, DeleteGalleryMessage.class, DownloadPostMessage.class, DownloadStatusMessage.class,
IdentityMessage.class, MaintainMessage.class, ResponseMessage.class, AvailableCheckMessage.class, IdentityMessage.class, MaintainMessage.class, ResponseMessage.class, AvailableCheckMessage.class,
// 协议 v2 的消息与任务操作实体:原生镜像下 Jackson/MyBatis 都要反射构造它们。
ProtocolHelloMessage.class, TaskCommandMessage.class, TaskReportMessage.class, TaskReportAckMessage.class,
TaskOperation.class,
SubscriptionSnapshotMessage.class, SubscriptionSnapshotPayload.class, SubscriptionAccountSnapshot.class, SubscriptionSnapshotMessage.class, SubscriptionSnapshotPayload.class, SubscriptionAccountSnapshot.class,
SubscriptionBindingSnapshot.class, LinkPreviewOptions.class}) SubscriptionBindingSnapshot.class, LinkPreviewOptions.class})
public class CustomBean { public class CustomBean {
@@ -41,4 +49,22 @@ public class CustomBean {
public TelegramBot getTelegramBot(){ public TelegramBot getTelegramBot(){
return new TelegramBot(botToken); return new TelegramBot(botToken);
} }
/**
* 订阅分散刷新的排程器。
*
* <p>放在这里是为了让窗口、最小间隔、tick 周期与重试间隔集中由配置注入,
* 同时保持 {@link SubscriptionRefreshPlanner} 本身是可直接构造的纯对象
* (便于用固定时钟与固定随机种子做确定性单元测试)。
*/
@Bean
public SubscriptionRefreshPlanner subscriptionRefreshPlanner(
@Value("${subscription.refresh.window-hours:24}") long windowHours,
@Value("${subscription.refresh.min-gap-minutes:60}") long minGapMinutes,
@Value("${subscription.refresh.tick-interval-ms:300000}") long tickIntervalMs,
@Value("${subscription.refresh.retry-delay-minutes:60}") long retryDelayMinutes) {
return new SubscriptionRefreshPlanner(Clock.systemDefaultZone(), new Random(),
Duration.ofHours(windowHours), Duration.ofMinutes(minGapMinutes),
Duration.ofMillis(tickIntervalMs), Duration.ofMinutes(retryDelayMinutes));
}
} }
@@ -0,0 +1,83 @@
package com.lion.lionwebsite.Configuration;
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
import com.lion.lionwebsite.Domain.ImageKeyCache;
import com.lion.lionwebsite.Service.GalleryIndexService;
import com.lion.lionwebsite.Util.GalleryUtil;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import javax.sql.DataSource;
import java.util.List;
/**
* 图片索引服务的事务装配。
*
* <p>单独成文件是为了不触碰既有的 SqlConfiguration:那里的两个 SqlSessionFactory
* 保持原样,这里只额外声明缓存库的事务管理器与模板。
*
* <p>事务管理器必须绑定 datasource-cache —— 索引只写缓存库,
* 若误用主库事务管理器,整组替换就失去原子性(删除与写入落在不同连接上)。
*/
@Configuration
public class GalleryIndexConfiguration {
@Bean
public PlatformTransactionManager cacheTransactionManager(
@Qualifier("datasource-cache") DataSource cacheDataSource) {
return new DataSourceTransactionManager(cacheDataSource);
}
/** 只用于图片索引的短事务发布,超时留给上层请求超时控制,这里显式设为 30 秒。 */
@Bean
public TransactionTemplate cacheTransactionTemplate(
@Qualifier("cacheTransactionManager") PlatformTransactionManager cacheTransactionManager) {
TransactionTemplate template = new TransactionTemplate(cacheTransactionManager);
template.setTimeout(30);
return template;
}
/**
* 只读快照事务,用于一次性读取头部与页码。
*
* <p>读事务不写库,因此只读连接可以与写入并发;它的作用是让「读头部 + 读页码」
* 落在同一个快照上,避免恰好读到整组替换的中间状态。
*/
@Bean
public TransactionTemplate cacheReadTransactionTemplate(
@Qualifier("cacheTransactionManager") PlatformTransactionManager cacheTransactionManager) {
TransactionTemplate template = new TransactionTemplate(cacheTransactionManager);
template.setTimeout(10);
return template;
}
/**
* 这里显式不设置 {@code readOnly}:sqlite-jdbc 不允许在已建立的连接上改只读标志
* (实测抛 SQLException: Cannot change read-only flag after establishing a connection),
* 而事务模板拿到的是连接池里的既有连接,设置只读会让每次读取都直接失败。
* 快照一致性由「同一个事务」保证,不需要额外的只读标志。
*/
@Bean
public GalleryIndexService galleryIndexService(
ImageCacheMapper imageCacheMapper,
@Qualifier("cacheTransactionTemplate") TransactionTemplate cacheTransactionTemplate,
@Qualifier("cacheReadTransactionTemplate") TransactionTemplate cacheReadTransactionTemplate) {
return new GalleryIndexService(imageCacheMapper, cacheTransactionTemplate,
cacheReadTransactionTemplate,
GalleryIndexConfiguration::fetchPageKeys);
}
/**
* 生产环境的上游实现:沿用 GalleryUtil.parseImageKeys(HTTP + JS 解析)。
*
* <p>放在这里而不是服务内部,是为了让服务在测试中可以脱离静态工具类。
*/
static List<ImageKeyCache> fetchPageKeys(String url) throws Exception {
return GalleryUtil.parseImageKeys(url);
}
}
@@ -17,7 +17,12 @@ public class InterceptorConfiguration implements WebMvcConfigurer {
@Override @Override
public void addInterceptors(InterceptorRegistry registry) { public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(getPersonalInterceptor()).addPathPatterns("/personal/**", "/remote/**"); // 登录端点必须排除在闸门之外,否则校验票据的入口会被自己拦住。
// /personal/ 只是跳到静态入口 /index,也一并排除:未登录时让前端自己去提示,
// 否则用户直接访问根域名只会看到 401。
registry.addInterceptor(getPersonalInterceptor())
.addPathPatterns("/personal/**", "/remote/**")
.excludePathPatterns("/personal/", "/personal/login", "/personal/login/logout", "/personal/denied");
registry.addInterceptor(taskHandlerInterceptor).addPathPatterns("/GalleryManage", "/GalleryManage/**", "/validate"); registry.addInterceptor(taskHandlerInterceptor).addPathPatterns("/GalleryManage", "/GalleryManage/**", "/validate");
registry.addInterceptor(getHumanInterceptor()).addPathPatterns("/", "/mobile"); registry.addInterceptor(getHumanInterceptor()).addPathPatterns("/", "/mobile");
} }
@@ -8,6 +8,11 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder; import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import com.zaxxer.hikari.HikariDataSource;
import javax.sql.DataSource; import javax.sql.DataSource;
@@ -16,7 +21,9 @@ public class SqlConfiguration {
@Bean(name = "datasource-main") @Bean(name = "datasource-main")
@ConfigurationProperties(prefix = "spring.datasource-main") @ConfigurationProperties(prefix = "spring.datasource-main")
public DataSource dataSourceMain() { public DataSource dataSourceMain() {
return DataSourceBuilder.create().build(); DataSource dataSource = DataSourceBuilder.create().build();
applyBusyTimeout(dataSource);
return dataSource;
} }
@Bean(name = "datasource-cache") @Bean(name = "datasource-cache")
@@ -25,6 +32,18 @@ public class SqlConfiguration {
return DataSourceBuilder.create().build(); return DataSourceBuilder.create().build();
} }
/**
* 给主库连接打开 busy 等待。
*
* <p>主库是单文件 SQLite,任务生命周期会并发开启短事务(创建、状态回执、删除)。
* 默认没有 busy 超时,第二个写事务会立刻拿到 SQLITE_BUSY 并被当作 500;
* 打开等待后,写冲突会按序排队重试而不是失败。
*/
private void applyBusyTimeout(DataSource dataSource) {
if (dataSource instanceof HikariDataSource hikari)
hikari.setConnectionInitSql("PRAGMA busy_timeout=5000");
}
@Bean(name = "sqlSessionFactory-main") @Bean(name = "sqlSessionFactory-main")
public SqlSessionFactory sqlSessionFactoryMain(@Qualifier("datasource-main") DataSource dataSource) throws Exception { public SqlSessionFactory sqlSessionFactoryMain(@Qualifier("datasource-main") DataSource dataSource) throws Exception {
SqlSessionFactoryBean sessionFactoryBean = new SqlSessionFactoryBean(); SqlSessionFactoryBean sessionFactoryBean = new SqlSessionFactoryBean();
@@ -48,4 +67,30 @@ public class SqlConfiguration {
public SqlSessionTemplate sqlSessionTemplateCache(@Qualifier("sqlSessionFactory-cache") SqlSessionFactory sqlSessionFactory) { public SqlSessionTemplate sqlSessionTemplateCache(@Qualifier("sqlSessionFactory-cache") SqlSessionFactory sqlSessionFactory) {
return new SqlSessionTemplate(sqlSessionFactory); return new SqlSessionTemplate(sqlSessionFactory);
} }
/**
* 主库事务管理器。
*
* <p>必须显式绑定到 datasource-main:应用里有两个 DataSource,Spring Boot 的自动配置
* 会退避,若没有这个 bean 就没有任何事务管理器,@Transactional 会静默失效。
* @Primary 同时让「不指定名字就注入 TransactionManager」的旧代码继续可用。
*/
@Bean(name = "mainTransactionManager")
@Primary
public DataSourceTransactionManager mainTransactionManager(@Qualifier("datasource-main") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
/**
* 主库编程式事务模板。
*
* <p>任务生命周期需要把「短事务」放在网络调用之外,并且要求「回调里捕获异常后不再抛出」
* 不能错误提交,因此调用方必须用 {@code execute} 的返回状态或再次抛出决定回滚,
* 而不是在回调里吞掉异常(见 GalleryTaskLifecycleService)。
*/
@Bean(name = "mainTransactionTemplate")
@Primary
public TransactionTemplate mainTransactionTemplate(@Qualifier("mainTransactionManager") DataSourceTransactionManager transactionManager) {
return new TransactionTemplate(transactionManager);
}
} }
@@ -0,0 +1,57 @@
package com.lion.lionwebsite.Configuration;
import com.lion.lionwebsite.Domain.TaskOperation;
import com.lion.lionwebsite.Service.GalleryTaskLifecycleService;
import com.lion.lionwebsite.Service.RemoteService;
import com.lion.lionwebsite.Service.TaskReportHandler;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Component;
/**
* 装配协议 v2 的双向依赖,并做启动恢复。
*
* <p>RemoteService 消费上报时需要生命周期服务,而生命周期服务下发命令时需要 RemoteService。
* 直接互相注入构造参数会形成环,所以这里在两端都构造完成之后用 setter 接上;
* 同时也把「启动时把中断的 SUBMITTING 判为 UNKNOWN、释放 RESERVED 占位」这一步接到应用就绪事件上。
*
* <p>顺序是不可交换的:
* <ol>
* <li>先绑定上报处理器、下发判定解析器与 operationId 解析器;</li>
* <li>再执行启动恢复(SUBMITTING → UNKNOWN、释放 RESERVED);</li>
* <li>最后才允许建立连接与自动重投。连接由本类显式触发,不再由 RemoteService 的
* PostConstruct 在装配前发起,否则会把状态不明的任务盲目重投给节点。</li>
* </ol>
* 恢复失败时仍然允许连接(保持节点在线/心跳),但明确禁止自动派发。
*/
@Component
@Slf4j
@RequiredArgsConstructor
public class TaskLifecycleWiring {
final RemoteService remoteService;
final GalleryTaskLifecycleService lifecycleService;
@EventListener(ApplicationReadyEvent.class)
public void wire() {
TaskReportHandler handler = lifecycleService;
remoteService.setTaskReportHandler(handler);
remoteService.setDownloadOperationIdResolver(lifecycleService::currentOperationId);
remoteService.setDispatchResolver(lifecycleService::resolveDispatch);
boolean recoverySafe = false;
try {
int recovered = lifecycleService.recoverInterruptedSubmissions();
recoverySafe = true;
if (recovered > 0)
log.warn("启动恢复:{} 个中断的创建/预占任务已处理(提交中的标为提交未知)", recovered);
} catch (RuntimeException e) {
// 恢复失败说明库中任务真实状态未知:可以保持连接,但绝不能自动重投/派发。
log.error("启动恢复失败,本次启动将拒绝自动重投未完成任务", e);
}
// 绑定与恢复完成之后才连接:连接成功后的自动重投受 lifecycleReady/recoverySafe 双重约束。
remoteService.startAfterWiring(recoverySafe);
}
}
@@ -67,6 +67,14 @@ public class GalleryManageController {
return galleryManageService.deleteGalleryByGid(gid, AuthCode); return galleryManageService.deleteGalleryByGid(gid, AuthCode);
} }
/** 按 gid 查实际下载人昵称,仅管理员可用。 */
@GetMapping("/downloader")
public String selectDownloader(Integer gid, String AuthCode){
if(gid == null)
return Response._failure("参数不全");
return galleryManageService.selectDownloaderByGid(gid, AuthCode);
}
@PostMapping("/collect") @PostMapping("/collect")
public String collectGallery(Integer gid, String AuthCode){ public String collectGallery(Integer gid, String AuthCode){
@@ -2,16 +2,19 @@ package com.lion.lionwebsite.Controller;
import com.lion.lionwebsite.Service.LocalService; import com.lion.lionwebsite.Service.LocalService;
import com.lion.lionwebsite.Service.PersonalService; import com.lion.lionwebsite.Service.PersonalService;
import com.lion.lionwebsite.Util.PanelLoginTicket;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
import jakarta.servlet.http.HttpSession;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController; import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException; import java.io.IOException;
@@ -22,59 +25,99 @@ import java.io.IOException;
@RequestMapping("/personal") @RequestMapping("/personal")
public class PersonalController { public class PersonalController {
/** 会话有效期,与 PersonalHub 的面板会话保持一致:14 天滑动过期。 */
private static final int SESSION_MAX_INACTIVE_SECONDS = 60 * 60 * 24 * 14;
final PersonalService personalService; final PersonalService personalService;
final LocalService localService; final LocalService localService;
final PanelLoginTicket panelLoginTicket;
@GetMapping("/") @GetMapping("/")
public void index(HttpServletResponse resp) throws IOException { public void index(HttpServletResponse resp) throws IOException {
resp.sendRedirect("/index"); resp.sendRedirect("/index");
} }
@GetMapping("/files") /**
public String file(String path){ * 一键登录:校验机器人签发的 HMAC 票据,通过后建立会话并跳到个人面板。
return personalService.getFiles(path); *
* <p>票据一次性之外的防护全靠有效期,因此失败的票据不重定向到登录页,
* 而是直接回到提示页,避免把参数回显到浏览器历史里。
*/
@GetMapping("/login")
public void login(@RequestParam(value = "t", required = false) String ticket,
HttpServletRequest request,
HttpServletResponse response) throws IOException {
if (!panelLoginTicket.verify(ticket)) {
log.warn("一键登录票据无效或已过期,来自 {}", clientIp(request));
// 重定向目标用浏览器可见路径:nginx 的 location / 会把它改写成 /personal/…
response.sendRedirect("/denied");
return;
} }
@PostMapping("/uploadFile") // 先作废旧会话再建新的,避免会话固定攻击。
public String uploadFile(String path, String fileName, MultipartFile file){ HttpSession existing = request.getSession(false);
return personalService.uploadFile(path, fileName, file); if (existing != null)
existing.invalidate();
HttpSession session = request.getSession(true);
session.setAttribute("personalAuthenticated", Boolean.TRUE);
session.setMaxInactiveInterval(SESSION_MAX_INACTIVE_SECONDS);
log.info("一键登录成功,来自 {}", clientIp(request));
response.sendRedirect("/index");
} }
@GetMapping("/private/**") @GetMapping("/login/logout")
public void getFile(HttpServletRequest request, HttpServletResponse response, String path){ public void logout(HttpServletRequest request, HttpServletResponse response) throws IOException {
personalService.download(request, response, path); HttpSession session = request.getSession(false);
if (session != null)
session.invalidate();
response.sendRedirect("/denied");
} }
@PostMapping("/share") /** 未登录或票据失效时的提示页;纯静态文案,不含任何可推断的信息。 */
public String shareFile(String path, Integer expireHour) { @GetMapping(value = "/denied", produces = MediaType.TEXT_HTML_VALUE)
return personalService.shareFile(path, expireHour); public String denied() {
return """
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>需要登录 · LionWebsite</title>
<style>
body { margin:0; display:flex; min-height:100vh; align-items:center; justify-content:center;
background:#f6f7f9; color:#1f2328;
font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",sans-serif; }
.card { max-width:26rem; padding:2rem; background:#fff; border:1px solid #e2e5e9;
border-radius:10px; box-shadow:0 1px 2px rgba(0,0,0,.04); }
h1 { margin:0 0 .75rem; font-size:1.15rem; }
p { margin:0; line-height:1.7; color:#4a5259; }
code { background:#f0f2f5; padding:.1rem .35rem; border-radius:4px; }
</style>
</head>
<body>
<div class="card">
<h1>需要登录</h1>
<p>请在机器人里发送 <code>/login</code>,用返回的一次性链接打开个人面板。</p>
</div>
</body>
</html>
""";
} }
@PostMapping("/compress") private static String clientIp(HttpServletRequest request) {
public String compress(String path){ String forwarded = request.getHeader("X-Forwarded-For");
return personalService.compress(path); if (forwarded != null && !forwarded.isBlank())
} return forwarded.split(",")[0].trim();
return request.getRemoteAddr();
@PostMapping("/delete")
public String deleteFile(String path){
return personalService.deleteFile(path);
}
@PostMapping("/extendShareTime")
public String extendShareTime(String path, Integer extendHour) {
return personalService.extendShareTime(path, extendHour);
}
@PostMapping("/cancelShare")
public String cancelShare(String path){
return personalService.cancelShare(path);
} }
@PostMapping("/updateSub") @PostMapping("/updateSub")
public String updateSub() throws IOException { public String updateSub() throws IOException {
Response response = Response.generateResponse(); Response response = Response.generateResponse();
if(localService.updateSub(true)) if(localService.updateSub())
response.success(); response.success();
else else
response.failure(); response.failure();
@@ -6,7 +6,10 @@ import com.lion.lionwebsite.Service.PublicService;
import com.lion.lionwebsite.Service.QueryService; import com.lion.lionwebsite.Service.QueryService;
import com.lion.lionwebsite.Service.RemoteService; import com.lion.lionwebsite.Service.RemoteService;
import com.lion.lionwebsite.Service.SubService; import com.lion.lionwebsite.Service.SubService;
import com.lion.lionwebsite.Service.UserService;
import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import tools.jackson.databind.node.ObjectNode;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -15,16 +18,12 @@ import org.springframework.web.bind.annotation.*;
import java.io.IOException; import java.io.IOException;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.util.LinkedList;
import java.util.List;
@RestController @RestController
@Slf4j @Slf4j
@RequiredArgsConstructor @RequiredArgsConstructor
public class PublicController { public class PublicController {
final List<String> black_share_codes = new LinkedList<>();
final PublicService publicService; final PublicService publicService;
final RemoteService remoteService; final RemoteService remoteService;
@@ -59,32 +58,18 @@ public class PublicController {
subService.updateSub(response, request, client, key); subService.updateSub(response, request, client, key);
} }
@GetMapping("/GetFile/{path}")
public void getFile(HttpServletRequest request, HttpServletResponse response, String ShareCode, @PathVariable("path") String path) throws IOException {
synchronized (black_share_codes) {
if (black_share_codes.contains(ShareCode))
return;
}
log.info("ShareCode:{}", ShareCode);
log.info("Path:{}", path);
boolean result = publicService.GetFile(request, response, ShareCode);
if(!result)
black_share_codes.add(ShareCode);
if(black_share_codes.size() > 100)
black_share_codes.removeFirst();
}
@PostMapping("/validate") @PostMapping("/validate")
public String validate(String AuthCode){ public String validate(String AuthCode){
Response response = Response.generateResponse(); Response response = Response.generateResponse();
User user = publicService.getUserId(AuthCode); User user = publicService.getUserId(AuthCode);
String isAvailable = remoteService.isDead() ? "false": "true"; // 身份字段直接作为结构化 data 返回;共享前端解码器兼容旧版本字符串。
response.success(String.format("{\"userId\": %d, " + ObjectNode identity = CustomUtil.objectMapper.createObjectNode();
"\"username\": \"%s\", " + identity.put("userId", user.getId());
"\"isAvailable\": %s}", user.getId(), user.getUsername(), isAvailable)); identity.put("username", user.getUsername());
identity.put("isAvailable", !remoteService.isDead());
// 管理员标记随登录一起下发,前端据此决定是否显示下载人信息与筛选。
identity.put("isAdmin", user.getId() == UserService.ADMIN_USER_ID);
response.success(identity);
return response.toJSONString(); return response.toJSONString();
} }
@@ -5,8 +5,11 @@ import com.lion.lionwebsite.Domain.ImageKeyCache;
import org.apache.ibatis.annotations.Insert; import org.apache.ibatis.annotations.Insert;
import org.apache.ibatis.annotations.Mapper; import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Delete;
import org.apache.ibatis.annotations.Select; import org.apache.ibatis.annotations.Select;
import java.util.List;
@Mapper @Mapper
public interface ImageCacheMapper { public interface ImageCacheMapper {
@Insert("insert into ImageKeyCache values (#{gid}, #{imgkey}, #{page})") @Insert("insert into ImageKeyCache values (#{gid}, #{imgkey}, #{page})")
@@ -21,4 +24,41 @@ public interface ImageCacheMapper {
@Select("select * from gidToKey where gid=#{gid}") @Select("select * from gidToKey where gid=#{gid}")
GidToKey selectKeyByGid(String gid); GidToKey selectKeyByGid(String gid);
/** 已缓存的页 key 行数,用于判断索引是否完整(半截缓存需要重建)。 */
@Select("select count(*) from ImageKeyCache where gid=#{gid}")
int countImageKeyCacheByGid(String gid);
@Delete("delete from ImageKeyCache where gid=#{gid}")
void deleteImageKeyCacheByGid(String gid);
@Delete("delete from gidToKey where gid=#{gid}")
void deleteGidToKey(String gid);
/**
* 某个 gid 已落库的页码列表(升序)。
*
* <p>只按行数判断完整性会把「页码 1 和 3、共 2 行」当成完整索引,
* 因此完整性判定改为按页码集合比对——先取回页码,再校验它是否恰好是 1..pages。
*/
@Select("select page from ImageKeyCache where gid=#{gid} order by page")
List<Integer> selectPageNumbersByGid(String gid);
/** 该 gid 下 imgkey 为空的脏行数;任何一行脏行都说明索引不可用。 */
@Select("select count(*) from ImageKeyCache where gid=#{gid} and (imgkey is null or trim(imgkey)='')")
int countInvalidImageKeysByGid(String gid);
/**
* 批量写入页 key(单条多值 INSERT)。
*
* <p>整本画廊的索引可能上千页,逐行 INSERT 会拉长写事务并放大锁竞争;
* 调用方按固定大小分片后,一个写事务里只需少量语句即可写完。
*/
// MyBatis 只在注解值「以 <script> 开头」时才按 XML 动态 SQL 解析,因此这里不能有前导空白。
@Insert("<script>"
+ "insert into ImageKeyCache (gid, imgkey, page) values "
+ "<foreach collection='pages' item='p' separator=','>"
+ "(#{p.gid}, #{p.imgkey}, #{p.page})"
+ "</foreach>"
+ "</script>")
int insertImageKeyCacheBatch(@Param("pages") List<ImageKeyCache> pages);
} }
@@ -20,4 +20,8 @@ public interface CollectMapper {
@Select("select collector from collect where gid=#{gid}") @Select("select collector from collect where gid=#{gid}")
ArrayList<Integer> selectCollectorByGid(int gid); ArrayList<Integer> selectCollectorByGid(int gid);
/** 任务删除完成后清理该 gid 的全部收藏记录(与删任务行同一事务)。 */
@Delete("delete from collect where gid=#{gid}")
void deleteByGid(@Param("gid") int gid);
} }
@@ -36,9 +36,10 @@ public interface GalleryMapper {
@Update(""" @Update("""
update gallery set name=#{name}, link=#{link}, language=#{language}, update gallery set name=#{name}, link=#{link}, language=#{language}, downloader=#{downloader},
pages=#{pages}, status=#{status}, fileSize=#{fileSize}, createTime=#{createTime}, pages=#{pages}, status=#{status}, fileSize=#{fileSize}, createTime=#{createTime},
proceeding=#{proceeding}, resolution=#{resolution}, displayFileSize=#{displayFileSize}, is_download=#{is_download} proceeding=#{proceeding}, resolution=#{resolution}, displayFileSize=#{displayFileSize},
thumb_link=#{thumb_link}, is_download=#{is_download}
where gid=#{gid}""") where gid=#{gid}""")
void updateGallery(Gallery gallery); void updateGallery(Gallery gallery);
@@ -47,4 +48,8 @@ public interface GalleryMapper {
@Delete("delete from gallery where gid=#{gid}") @Delete("delete from gallery where gid=#{gid}")
void deleteGalleryByGid(Integer gid); void deleteGalleryByGid(Integer gid);
/** 只改状态列,用于「待删除」这类不牵动其他字段的意图标记。 */
@Update("update gallery set status=#{status} where gid=#{gid}")
int updateGalleryStatus(@Param("gid") int gid, @Param("status") String status);
} }
@@ -1,31 +0,0 @@
package com.lion.lionwebsite.Dao.normal;
import com.lion.lionwebsite.Domain.ShareFile;
import org.apache.ibatis.annotations.*;
import java.util.ArrayList;
import java.util.Date;
@Mapper
public interface ShareFileMapper {
@Insert("insert into ShareFile (ShareCode, FilePath, ExpireTime) values (#{ShareCode}, #{FilePath}, #{ExpireTime})")
void insertShareFile(@Param("ShareCode")String ShareCode, @Param("FilePath")String FilePath, @Param("ExpireTime") Date ExpireTime);
@Select("select * from ShareFile where ShareCode=#{ShareCode}")
ShareFile selectShareFileByShareCode(String ShareCode);
@Select("select * from ShareFile where FilePath=#{FilePath}")
ShareFile selectShareFileByFilePath(String FilePath);
@Select("select * from ShareFile where FilePath like '%' || #{FilePath} || '%'")
ArrayList<ShareFile> selectShareFilesByFilePath(String FilePath);
@Delete("delete from ShareFile where ShareCode=#{ShareCode}")
void deleteShareFile(String ShareCode);
@Select("select * from ShareFile")
ShareFile[] selectAllShareFile();
@Update("update ShareFile set ExpireTime=#{ExpireTime}, ShareCode=#{ShareCode} where FilePath=#{FilePath}")
void updateShareFile(ShareFile ShareFile);
}
@@ -13,10 +13,10 @@ public interface SubMapper {
@Options(useGeneratedKeys = true, keyProperty = "id") @Options(useGeneratedKeys = true, keyProperty = "id")
void insertSubscriptionAccount(SubscriptionAccount account); void insertSubscriptionAccount(SubscriptionAccount account);
@Select("select id, name, upstream_key as upstreamKey, filter_high_multiplier as filterHighMultiplier, enabled, last_success_at as lastSuccessAt, last_error as lastError, created_at as createdAt, updated_at as updatedAt, (select count(*) from sub_bind sb where sb.subscription_account_id = sa.id) as boundUserCount from subscription_account sa order by id") @Select("select id, name, upstream_key as upstreamKey, filter_high_multiplier as filterHighMultiplier, enabled, last_success_at as lastSuccessAt, last_error as lastError, created_at as createdAt, updated_at as updatedAt, next_refresh_at as nextRefreshAt, last_success_epoch as lastSuccessEpoch, (select count(*) from sub_bind sb where sb.subscription_account_id = sa.id) as boundUserCount from subscription_account sa order by id")
ArrayList<SubscriptionAccount> selectAllSubscriptionAccounts(); ArrayList<SubscriptionAccount> selectAllSubscriptionAccounts();
@Select("select id, name, upstream_key as upstreamKey, filter_high_multiplier as filterHighMultiplier, enabled, last_success_at as lastSuccessAt, last_error as lastError, created_at as createdAt, updated_at as updatedAt, (select count(*) from sub_bind sb where sb.subscription_account_id = sa.id) as boundUserCount from subscription_account sa where id=#{id}") @Select("select id, name, upstream_key as upstreamKey, filter_high_multiplier as filterHighMultiplier, enabled, last_success_at as lastSuccessAt, last_error as lastError, created_at as createdAt, updated_at as updatedAt, next_refresh_at as nextRefreshAt, last_success_epoch as lastSuccessEpoch, (select count(*) from sub_bind sb where sb.subscription_account_id = sa.id) as boundUserCount from subscription_account sa where id=#{id}")
SubscriptionAccount selectSubscriptionAccount(Integer id); SubscriptionAccount selectSubscriptionAccount(Integer id);
@Select("select count(*) from subscription_account where name=#{name}") @Select("select count(*) from subscription_account where name=#{name}")
@@ -28,54 +28,72 @@ public interface SubMapper {
@Update("update subscription_account set name=#{name}, upstream_key=#{upstreamKey}, filter_high_multiplier=#{filterHighMultiplier}, enabled=#{enabled}, updated_at=CURRENT_TIMESTAMP, last_error=null where id=#{id}") @Update("update subscription_account set name=#{name}, upstream_key=#{upstreamKey}, filter_high_multiplier=#{filterHighMultiplier}, enabled=#{enabled}, updated_at=CURRENT_TIMESTAMP, last_error=null where id=#{id}")
void updateSubscriptionAccount(SubscriptionAccount account); void updateSubscriptionAccount(SubscriptionAccount account);
@Update("update subscription_account set last_success_at=CURRENT_TIMESTAMP, last_error=null, updated_at=CURRENT_TIMESTAMP where id=#{id}") // last_success_at 仍写 SQLite 的 UTC 文本时间,仅供页面显示;
void markSubscriptionRefreshSuccess(Integer id); // last_success_epoch 是调度判断用的权威时刻,两者口径不同,不要互相推导。
@Update("update subscription_account set last_success_at=CURRENT_TIMESTAMP, last_success_epoch=#{epoch}, last_error=null, updated_at=CURRENT_TIMESTAMP where id=#{id}")
void markSubscriptionRefreshSuccess(@Param("id") Integer id, @Param("epoch") long epoch);
@Update("update subscription_account set last_error=#{error}, updated_at=CURRENT_TIMESTAMP where id=#{id}") @Update("update subscription_account set last_error=#{error}, updated_at=CURRENT_TIMESTAMP where id=#{id}")
void markSubscriptionRefreshFailure(@Param("id") Integer id, @Param("error") String error); void markSubscriptionRefreshFailure(@Param("id") Integer id, @Param("error") String error);
/** 排定下一次刷新时刻;失败重试与次日分槽都走这里。 */
@Update("update subscription_account set next_refresh_at=#{nextRefreshAt}, updated_at=CURRENT_TIMESTAMP where id=#{id}")
void updateNextRefreshAt(@Param("id") Integer id, @Param("nextRefreshAt") long nextRefreshAt);
@Delete("delete from subscription_account where id=#{id}") @Delete("delete from subscription_account where id=#{id}")
void deleteSubscriptionAccount(Integer id); void deleteSubscriptionAccount(Integer id);
@Insert("insert into sub_bind (key, user, subscription_account_id) values (#{key}, #{user}, #{subscriptionAccountId})") @Insert("insert into sub_bind (key, user, subscription_account_id, user_id) values (#{key}, #{user}, #{subscriptionAccountId}, (select id from User where username=#{user}))")
void insertSubBind(SubBind subBind); void insertSubBind(SubBind subBind);
@Select("select sb.key, sb.user, sb.subscription_account_id as subscriptionAccountId, sa.name as subscriptionAccountName, sa.enabled as subscriptionAccountEnabled, sa.filter_high_multiplier as filterHighMultiplier from sub_bind sb left join subscription_account sa on sa.id=sb.subscription_account_id order by sb.user") @Select("select sb.key, coalesce(u.username, sb.user) as user, sb.user_id as userId, sb.subscription_account_id as subscriptionAccountId, sa.name as subscriptionAccountName, sa.enabled as subscriptionAccountEnabled, sa.filter_high_multiplier as filterHighMultiplier from sub_bind sb left join User u on u.id=sb.user_id left join subscription_account sa on sa.id=sb.subscription_account_id order by sb.user")
ArrayList<SubBind> selectAllSubBind(); ArrayList<SubBind> selectAllSubBind();
@Select("select sb.key, sb.user, sb.subscription_account_id as subscriptionAccountId, sa.name as subscriptionAccountName, sa.enabled as subscriptionAccountEnabled, sa.filter_high_multiplier as filterHighMultiplier from sub_bind sb left join subscription_account sa on sa.id=sb.subscription_account_id where sb.key=#{key}") @Select("select sb.key, coalesce(u.username, sb.user) as user, sb.user_id as userId, sb.subscription_account_id as subscriptionAccountId, sa.name as subscriptionAccountName, sa.enabled as subscriptionAccountEnabled, sa.filter_high_multiplier as filterHighMultiplier from sub_bind sb left join User u on u.id=sb.user_id left join subscription_account sa on sa.id=sb.subscription_account_id where sb.key=#{key}")
SubBind selectSubBind(String key); SubBind selectSubBind(String key);
@Select("select count(key) from sub_bind where key=#{key}") @Select("select count(key) from sub_bind where key=#{key}")
boolean selectSubBindExist(String key); boolean selectSubBindExist(String key);
@Select("select count(*) from sub_bind where user=#{user}") @Select("select count(*) from sub_bind where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
int countSubBindByUser(String user); int countSubBindByUser(String user);
@Update("update sub_bind set subscription_account_id=#{accountId} where user=#{user}") @Update("update sub_bind set subscription_account_id=#{accountId} where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
int updateSubBindAccount(@Param("user") String user, @Param("accountId") Integer accountId); int updateSubBindAccount(@Param("user") String user, @Param("accountId") Integer accountId);
@Update("update sub_bind set key=#{key} where user=#{user}") @Update("update sub_bind set key=#{key} where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
int updateSubBindKey(@Param("user") String user, @Param("key") String key); int updateSubBindKey(@Param("user") String user, @Param("key") String key);
@Delete("delete from sub_bind where user=#{user}") @Delete("delete from sub_bind where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
void deleteSubBind(String user); void deleteSubBind(String user);
@Insert("insert into sub_update_record (user, ip, UA, time, location) values (#{user}, #{ip}, #{UA}, #{time}, #{location})") @Insert("insert into sub_update_record (user, ip, UA, time, location, user_id) values (#{user}, #{ip}, #{UA}, #{time}, #{location}, coalesce(#{userId}, (select id from User where username=#{user})))")
@Options(useGeneratedKeys = true, keyProperty = "id")
void insertSubUpdateRecord(SubUpdateRecord subUpdateRecord); void insertSubUpdateRecord(SubUpdateRecord subUpdateRecord);
@Select("select * from sub_update_record order by time desc") @Select("select id,user,ip,UA,time,location,user_id as userId from sub_update_record order by time desc")
ArrayList<SubUpdateRecord> selectAllSubUpdateRecord(); ArrayList<SubUpdateRecord> selectAllSubUpdateRecord();
@Delete("delete from sub_update_record where user=#{user}") @Delete("delete from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
void deleteSubUpdateRecord(String user); void deleteSubUpdateRecord(String user);
@Select("select count(user) from sub_update_record where user=#{user}") @Select("select count(user) from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
Integer selectUpdateRecordCount(String user); Integer selectUpdateRecordCount(String user);
@Select("select min(id) from sub_update_record where user=#{user}") @Select("select min(id) from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
Integer selectMinUpdateRecordId(String user); Integer selectMinUpdateRecordId(String user);
/** 后台补齐归属地:只更新该用户最近一条记录,避免覆盖更早的历史记录。 */
@Update("update sub_update_record set location=#{location} where id=(select max(id) from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user}))")
void updateLatestSubUpdateRecordLocation(@Param("user") String user, @Param("location") String location);
@Update("update sub_update_record set location=#{location} where id=#{id}")
void updateSubUpdateRecordLocation(@Param("id") int id, @Param("location") String location);
@Update("update sub_bind set user=#{newName} where user_id=#{userId} or (user_id is null and user=#{oldName})")
void renameBoundUser(@Param("userId") int userId, @Param("oldName") String oldName,
@Param("newName") String newName);
@Delete("delete from sub_update_record where id=#{id}") @Delete("delete from sub_update_record where id=#{id}")
void deleteSubUpdateRecordById(int id); void deleteSubUpdateRecordById(int id);
} }
@@ -0,0 +1,107 @@
package com.lion.lionwebsite.Dao.normal;
import com.lion.lionwebsite.Domain.TaskOperation;
import org.apache.ibatis.annotations.*;
/**
* 任务操作记录的持久化入口(主库)。
*
* <p>所有写操作都在调用方的事务里执行;这里的注解只描述 SQL,不包含业务判断。
* 主键为 gid,保证「同一 gid 只能有一条有效操作意图」。
*/
@Mapper
public interface TaskOperationRepository {
@Select("select * from task_operation where gid=#{gid}")
TaskOperation selectByGid(int gid);
/** 生成该 gid 的稳定下载 operationId:只有还没有记录时才写入。 */
@Insert("insert or ignore into task_operation (gid, operationId, deleteOperationId, state, lastRevision," +
" lastReportId, updateTime) values (#{gid}, #{operationId}, null, #{state}, 0, null, #{updateTime})")
int insertIfAbsent(@Param("gid") int gid, @Param("operationId") String operationId,
@Param("state") String state, @Param("updateTime") long updateTime);
@Insert("insert into task_operation (gid, operationId, deleteOperationId, state, lastRevision, lastReportId, updateTime)" +
" values (#{gid}, #{operationId}, #{deleteOperationId}, #{state}, #{lastRevision}, #{lastReportId}, #{updateTime})")
void insert(TaskOperation operation);
/** 推进创建意图的状态(RESERVED → SUBMITTING → SUBMITTED/UNKNOWN)。 */
@Update("update task_operation set state=#{state}, updateTime=#{updateTime} where gid=#{gid}")
int updateState(@Param("gid") int gid, @Param("state") String state, @Param("updateTime") long updateTime);
/**
* 带前置条件的创建状态推进:只有当前状态与 operationId 都匹配时才写入。
*
* <p>这是「提交前先确认这一代意图仍然有效」的 CAS:若期间被删除/重建,
* 返回值 0,调用方必须停下,绝不能再产生上游副作用。
*/
@Update("update task_operation set state=#{state}, updateTime=#{updateTime}" +
" where gid=#{gid} and operationId=#{operationId} and state=#{expectedState}")
int compareAndSetCreateState(@Param("gid") int gid, @Param("operationId") String operationId,
@Param("expectedState") String expectedState, @Param("state") String state,
@Param("updateTime") long updateTime);
/**
* 只按当前删除意图 ID 匹配的删除完成:期间若已被删除/重建,则不改动新世代记录。
*/
@Update("update task_operation set deleteOperationId=null, state=#{state}, updateTime=#{updateTime}" +
" where gid=#{gid} and deleteOperationId=#{deleteOperationId}")
int completeDeleteIfSameGeneration(@Param("gid") int gid, @Param("deleteOperationId") String deleteOperationId,
@Param("state") String state, @Param("updateTime") long updateTime);
/**
* 删除「本次创建意图」的预占记录:必须同时匹配 gid 与 operationId,且状态仍是
* RESERVED/SUBMITTING。
*
* <p>三重匹配保证释放流程只能清掉自己那一代:期间被删除/重建(新 operationId 或墓碑)
* 的记录一律不动,避免把新世代抹掉。
*/
@Delete("delete from task_operation where gid=#{gid} and operationId=#{operationId}" +
" and state in ('RESERVED','SUBMITTING')")
int deleteReservationIfSameGeneration(@Param("gid") int gid, @Param("operationId") String operationId);
/** 记录/替换当前下载 operationId 并推进状态。 */
@Update("update task_operation set operationId=#{operationId}, state=#{state}, updateTime=#{updateTime} where gid=#{gid}")
int updateOperationId(@Param("gid") int gid, @Param("operationId") String operationId,
@Param("state") String state, @Param("updateTime") long updateTime);
/**
* 给「有记录但缺 operationId」的历史任务补一个稳定 operationId。
* 只在确实缺失时写入,避免覆盖并发创建出来的新世代。
*/
@Update("update task_operation set operationId=#{operationId}, updateTime=#{updateTime}" +
" where gid=#{gid} and operationId is null")
int assignOperationIdIfMissing(@Param("gid") int gid, @Param("operationId") String operationId,
@Param("updateTime") long updateTime);
/**
* 墓碑复活:删除完成后的 gid 允许用全新 operationId 重新创建。
* revision 与回执一并清零,旧操作的迟到上报会因 operationId 不匹配被丢弃。
*/
@Update("update task_operation set operationId=#{operationId}, deleteOperationId=null, state=#{state}," +
" lastRevision=0, lastReportId=null, updateTime=#{updateTime} where gid=#{gid}")
int reactivate(@Param("gid") int gid, @Param("operationId") String operationId,
@Param("state") String state, @Param("updateTime") long updateTime);
/** 记录删除意图与其稳定的删除操作 ID(重试必须复用同一个 ID)。 */
@Update("update task_operation set deleteOperationId=#{deleteOperationId}, state=#{state}, updateTime=#{updateTime} where gid=#{gid}")
int updateDeleteIntent(@Param("gid") int gid, @Param("deleteOperationId") String deleteOperationId,
@Param("state") String state, @Param("updateTime") long updateTime);
/**
* 写入状态上报回执(revision + reportId)。
*
* <p>回执与任务行更新在同一个事务里提交,提交成功之后才回 ACK;
* 因此节点重复投递同一份上报时,只能看到已经提交过的回执,不会重复落库或重复通知。
*/
@Update("update task_operation set lastRevision=#{revision}, lastReportId=#{reportId}, updateTime=#{updateTime}" +
" where gid=#{gid}")
int updateReportReceipt(@Param("gid") int gid, @Param("revision") long revision,
@Param("reportId") String reportId, @Param("updateTime") long updateTime);
@Select("select * from task_operation where state=#{state}")
java.util.List<TaskOperation> selectByState(String state);
@Delete("delete from task_operation where gid=#{gid}")
int deleteByGid(int gid);
}
@@ -14,10 +14,20 @@ public interface UserMapper {
@Select("select * from User where username=#{username}") @Select("select * from User where username=#{username}")
User selectUserByUsername(String username); User selectUserByUsername(String username);
@Select("select * from User where id=#{id}")
User selectUserById(int id);
@Select("select AuthCode from User") @Select("select AuthCode from User")
String[] selectAllAuthCode(); String[] selectAllAuthCode();
@Select("select AuthCode from User") /**
* 仅返回仍启用的授权码。
*
* <p>此前这条 SQL 与 {@link #selectAllAuthCode()} 完全相同(都没有 isEnable 条件),
* 于是「停用用户」后刷新授权码集合依旧把该码放行,isEnable 形同虚设。
* isEnable 为空按 DDL 默认值 true 处理,避免历史行被锁死。
*/
@Select("select AuthCode from User where isEnable is null or isEnable = 1")
String[] selectEnableAuthCode(); String[] selectEnableAuthCode();
@Select("select * from User") @Select("select * from User")
@@ -48,6 +48,11 @@ public class Gallery {
@JsonProperty("downloader") @JsonProperty("downloader")
private int downloader; //下载人 private int downloader; //下载人
/** 下载人昵称。仅管理员查询时填充,普通用户拿不到,避免暴露他人信息。 */
@JsonProperty("downloaderName")
@JsonInclude(JsonInclude.Include.NON_EMPTY)
private String downloaderName;
@JsonProperty("collector") @JsonProperty("collector")
@JsonInclude(JsonInclude.Include.NON_EMPTY) @JsonInclude(JsonInclude.Include.NON_EMPTY)
private String collector; //收藏人 private String collector; //收藏人
@@ -1,12 +0,0 @@
package com.lion.lionwebsite.Domain;
import lombok.Data;
import java.util.Date;
@Data
public class ShareFile {
String ShareCode;
String FilePath;
Date ExpireTime;
}
@@ -14,4 +14,10 @@ public class SubBind {
String subscriptionAccountName; String subscriptionAccountName;
boolean subscriptionAccountEnabled; boolean subscriptionAccountEnabled;
boolean filterHighMultiplier; boolean filterHighMultiplier;
Integer userId;
public SubBind(String key, String user, Integer accountId, String accountName,
boolean accountEnabled, boolean filterHighMultiplier) {
this(key, user, accountId, accountName, accountEnabled, filterHighMultiplier, null);
}
} }
@@ -16,4 +16,9 @@ public class SubUpdateRecord {
String UA; String UA;
Date time; Date time;
String location; String location;
Integer userId;
public SubUpdateRecord(int id, String user, String ip, String UA, Date time, String location) {
this(id, user, ip, UA, time, location, null);
}
} }
@@ -1,6 +1,5 @@
package com.lion.lionwebsite.Domain; package com.lion.lionwebsite.Domain;
import lombok.AllArgsConstructor;
import lombok.Data; import lombok.Data;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
@@ -8,7 +7,6 @@ import java.util.Date;
@Data @Data
@NoArgsConstructor @NoArgsConstructor
@AllArgsConstructor
public class SubscriptionAccount { public class SubscriptionAccount {
private Integer id; private Integer id;
private String name; private String name;
@@ -22,4 +20,12 @@ public class SubscriptionAccount {
private Integer boundUserCount; private Integer boundUserCount;
private String v2Url; private String v2Url;
private String clashUrl; private String clashUrl;
/**
* 下一次应刷新的时刻(Epoch 毫秒),由分散调度器排定。
* 刻意用整数而非 DATETIME:SQLite 文本时间戳会被 JDBC 按本地时区解释,
* 实测偏差 8 小时,不能用于时间比较。
*/
private Long nextRefreshAt;
/** 最近一次成功刷新的真实时刻(Epoch 毫秒),用于「24 小时内必刷一次」的判断与陈旧告警。 */
private Long lastSuccessEpoch;
} }
@@ -0,0 +1,53 @@
package com.lion.lionwebsite.Domain;
import lombok.Data;
/**
* 单个 gid 的下载生命周期操作记录(协议 v2 的持久化意图)。
*
* <p>这张表是「先持久化意图、再产生外部副作用」的落点:
* <ul>
* <li>CREATE 意图先于上游 POST 落库,用来保证同一 gid 只有一个未完成意图;</li>
* <li>提交结果不确定时只能停在 {@link #STATE_UNKNOWN},绝不自动重复上游 POST;</li>
* <li>DELETE 意图先于节点删除落库,节点确认后(或文件已不存在)才在同一事务里删任务行并留墓碑。</li>
* </ul>
*
* <p>旧任务行没有对应操作记录,查询时按 gid 惰性补建(见 GalleryTaskLifecycleService)。
*/
@Data
public class TaskOperation {
/** 已登记创建意图,尚未向上游提交。 */
public static final String STATE_RESERVED = "RESERVED";
/** 正在向上游提交;此时不可再发第二次 POST。 */
public static final String STATE_SUBMITTING = "SUBMITTING";
/** 上游确认已接受下载请求。 */
public static final String STATE_SUBMITTED = "SUBMITTED";
/**
* 提交结果未知:超时、断连或在 SUBMITTING 期间重启。
* 上游不支持幂等,因此这里只能等对账(reconcile),不能自动重发。
*/
public static final String STATE_UNKNOWN = "UNKNOWN";
/** 已登记删除意图,等待节点确认。 */
public static final String STATE_DELETE_PENDING = "DELETE_PENDING";
/** 节点已确认删除,任务行也随之删除;本行作为墓碑保留,用于停止迟到状态的重放。 */
public static final String STATE_DELETED = "DELETED";
private int gid;
/** 当前有效的下载操作 ID;同一 gid 的 DOWNLOAD 重试必须复用它。 */
private String operationId;
/** 独立于下载操作 ID 的删除操作 ID,删除重试复用它。 */
private String deleteOperationId;
private String state;
/** 最近一次应用的状态上报 revision;用于丢弃乱序/重放的旧上报。 */
private long lastRevision;
/** 节点已确认收到的最新上报 ID;用于重复 ACK 的幂等判断。 */
private String lastReportId;
private long updateTime;
}
@@ -0,0 +1,44 @@
package com.lion.lionwebsite.Exception;
/**
* 图片索引构建失败。
*
* <p>调用方需要区分三类结果,才能给出正确的用户可见响应:
* 链接本身有问题({@link Reason#INVALID_LINK},属于请求错误)、
* 画廊已不存在({@link Reason#UNAVAILABLE},重试无意义)、
* 上游或数据库暂时不可用({@link Reason#TRANSIENT},可以重试)。
*/
public class GalleryIndexException extends Exception {
public enum Reason {
/** 链接无法解析出 gid 或 key。 */
INVALID_LINK,
/** 上游明确表示画廊不存在(下架/删除/解析不到页 key)。 */
UNAVAILABLE,
/** 上游返回的页 key 无法自洽(gid 不匹配、页码重复/跳号、imgkey 为空)。 */
MALFORMED,
/** 上游波动或索引写入失败,重试可能成功。 */
TRANSIENT
}
private final Reason reason;
public GalleryIndexException(Reason reason, String message) {
super(message);
this.reason = reason;
}
public GalleryIndexException(Reason reason, String message, Throwable cause) {
super(message, cause);
this.reason = reason;
}
public Reason getReason() {
return reason;
}
/** 重试没有意义(链接错误或画廊不可用);上游数据畸形与会话级故障仍可重试。 */
public boolean isPermanent() {
return reason == Reason.INVALID_LINK || reason == Reason.UNAVAILABLE;
}
}
@@ -7,11 +7,15 @@ import jakarta.servlet.http.HttpServletResponse;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import java.io.IOException; import java.io.IOException;
import java.util.Calendar; import java.time.LocalTime;
import java.time.format.DateTimeFormatter;
@WebFilter(filterName = "AdaptorFilter", urlPatterns = {"/", "/personal/"}) @WebFilter(filterName = "AdaptorFilter", urlPatterns = {"/", "/personal/"})
@Slf4j @Slf4j
public class AdaptorFilter implements Filter { public class AdaptorFilter implements Filter {
private static final DateTimeFormatter TIME_FORMATTER = DateTimeFormatter.ofPattern("HH:mm:ss");
@Override @Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException { public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
HttpServletRequest request = (HttpServletRequest) servletRequest; HttpServletRequest request = (HttpServletRequest) servletRequest;
@@ -22,14 +26,15 @@ public class AdaptorFilter implements Filter {
if(UserAgent == null) if(UserAgent == null)
return; return;
String AuthCode = request.getParameter("AuthCode") == null ? "null" : request.getParameter("AuthCode"); // 这里是全站请求日志,AuthCode 是真正的凭据;只记录是否携带,绝不落明文。
boolean hasAuthCode = request.getParameter("AuthCode") != null;
String ServletPath = request.getServletPath(); String ServletPath = request.getServletPath();
String ip = request.getHeader("X-Forwarded-For") == null ? request.getRemoteAddr(): request.getHeader("X-Forwarded-For"); String ip = request.getHeader("X-Forwarded-For") == null ? request.getRemoteAddr(): request.getHeader("X-Forwarded-For");
Calendar calendar = Calendar.getInstance(); String now = LocalTime.now().format(TIME_FORMATTER);
String now = String.format("%s:%s:%s", calendar.get(Calendar.HOUR_OF_DAY), calendar.get(Calendar.MINUTE), calendar.get(Calendar.SECOND));
//日志 //日志
log.info("{} ip:{} \tpath:{} \tAuthCode:{} ua:{}", now, ip, ServletPath, AuthCode, UserAgent.length() > 61 ? UserAgent.substring(0, 60): UserAgent); log.info("{} ip:{} \tpath:{} \tAuthCode:{} ua:{}", now, ip, ServletPath,
hasAuthCode ? "present" : "absent", UserAgent.length() > 61 ? UserAgent.substring(0, 60): UserAgent);
//如果是验证,则直接跳转 //如果是验证,则直接跳转
if(ServletPath.equals("/validate")) if(ServletPath.equals("/validate"))
@@ -37,7 +42,7 @@ public class AdaptorFilter implements Filter {
//如果不是,则根据UA判断是否跳转 //如果不是,则根据UA判断是否跳转
else if ((UserAgent.contains("Android") || UserAgent.contains("iPhone"))) else if ((UserAgent.contains("Android") || UserAgent.contains("iPhone")))
if (ServletPath.equals("/personal/") && AuthCode.equals("alone")) if (ServletPath.equals("/personal/") && "alone".equals(request.getParameter("AuthCode")))
response.sendRedirect("/mobile?AuthCode=alone"); response.sendRedirect("/mobile?AuthCode=alone");
else else
response.sendRedirect("/mobile"); response.sendRedirect("/mobile");
@@ -2,13 +2,40 @@ package com.lion.lionwebsite.Interceptor;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
import jakarta.servlet.http.HttpSession;
import org.springframework.web.servlet.HandlerInterceptor; import org.springframework.web.servlet.HandlerInterceptor;
/**
* 个人管理区的访问闸门:有效会话或合法 AuthCode。
*
* <p>会话来自机器人签发的登录链接(见 {@code PanelLoginTicket} 与
* {@code /personal/login});固定的 {@code alone} 授权码保留给下载器与存储节点推送,
* 因此这里两条路径都放行,等那些调用方切换完成后再退役字面量。
*/
public class PersonalInterceptor implements HandlerInterceptor { public class PersonalInterceptor implements HandlerInterceptor {
/** 会话标记:{@code /personal/login} 校验票据后写入。 */
public static final String SESSION_ATTRIBUTE = "personalAuthenticated";
@Override @Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler){ public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler){
return request.getParameter("AuthCode") != null && request.getParameter("AuthCode").equals("alone"); if (authenticatedSession(request))
return true;
String authCode = request.getParameter("AuthCode");
if (authCode != null && authCode.equals("alone"))
return true;
// 会话缺失或过期时回 401,前端据此提示「去机器人发 /login」而不是静默失败。
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
return false;
}
/** 只认服务端写入的会话标记,不信任客户端可伪造的 Cookie 内容。 */
public static boolean authenticatedSession(HttpServletRequest request) {
// 已有会话才可能已登录;不调用 getSession(),以免为匿名访问创建空会话。
HttpSession session = request.getSession(false);
return session != null && Boolean.TRUE.equals(session.getAttribute(SESSION_ATTRIBUTE));
} }
} }
@@ -16,7 +16,9 @@ public class TaskHandlerInterceptor implements HandlerInterceptor {
final UserMapper userMapper; final UserMapper userMapper;
String[] AuthCodes; // 由 PostConstruct 加载后仅被读(发布式更新)与被测试线程读取;
// volatile 保证 refresh 后其它线程立即看到新数组,避免停用的授权码短暂仍可用。
volatile String[] AuthCodes;
@PostConstruct @PostConstruct
void init() { void init() {
@@ -0,0 +1,319 @@
package com.lion.lionwebsite.Maintenance;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.time.Instant;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
/**
* 缓存库图片索引的离线体检与修复工具。
*
* <p>它是独立可执行类(只用 JDBC,不依赖 Spring / MyBatis),因此可以在服务停止后
* 直接对数据库文件运行,也可以先对生产库的副本演练。</p>
*
* <pre>
* java -cp sqlite-jdbc.jar GalleryIndexMaintenance.java --db /path/cache.db
* java -cp sqlite-jdbc.jar GalleryIndexMaintenance.java --db /path/cache.db --apply
* </pre>
*
* <p>它只做两类写入:删除不完整或有冲突的 gid 整组索引(gidToKey 头部加该 gid 的全部
* 页 key),以及删除没有头部记录的孤儿页 key。被删掉的 gid 由下一次在线阅读重新抓取,
* 因此不需要猜测保留哪一组 key。工具不建表、不改列、不碰其它业务表。</p>
*/
public final class GalleryIndexMaintenance {
/** 生产运行时数据库所在目录;默认拒绝直接写入,必须显式加 --allow-production。 */
private static final String PRODUCTION_DIR = "/home/lionwebsite";
private static final String PLAIN_REPAIR_NOTE =
"这些画廊将在下次被访问时重新抓取。";
public static void main(String[] args) {
Map<String, String> options = parseOptions(args);
if (options.containsKey("help") || !options.containsKey("db")) {
printUsage();
System.exit(options.containsKey("help") ? 0 : 2);
return;
}
Path database = Path.of(options.get("db")).toAbsolutePath().normalize();
boolean apply = options.containsKey("apply");
if (!Files.isRegularFile(database)) {
System.err.println("数据库不存在: " + database);
System.exit(2);
return;
}
if (apply && isProductionPath(database) && !options.containsKey("allow-production")) {
System.err.println("拒绝写入生产库路径 " + database
+ "。请先在副本上演练;确需在本机生产库执行时显式加 --allow-production,"
+ "并确保服务已停止。");
System.exit(3);
return;
}
try (Connection connection = open(database)) {
Report before = inspect(connection);
System.out.println(before.render(apply ? "修复前" : "体检结果"));
if (!apply) {
System.out.println();
System.out.println("dry-run:未做任何写入。确认无误后加 --apply 执行修复。");
System.exit(before.isHealthy() ? 0 : 1);
return;
}
if (before.isHealthy()) {
System.out.println();
System.out.println("索引已健康,无需修复。");
return;
}
connection.setAutoCommit(false);
int removed = repair(connection, before.badGids, before.orphanGids);
connection.commit();
System.out.println();
System.out.println("已删除 " + removed + " 个 gid 的索引整组," + PLAIN_REPAIR_NOTE);
Report after = inspect(connection);
System.out.println();
System.out.println(after.render("修复后"));
if (!after.isHealthy()) {
System.err.println("修复后仍未达到健康状态,请人工检查。");
System.exit(4);
}
} catch (SQLException e) {
System.err.println("执行失败: " + e.getMessage());
e.printStackTrace(System.err);
System.exit(5);
}
}
private static Connection open(Path database) throws SQLException {
Connection connection = DriverManager.getConnection("jdbc:sqlite:" + database);
try (Statement statement = connection.createStatement()) {
statement.execute("pragma busy_timeout=5000");
statement.execute("pragma foreign_keys=off");
}
return connection;
}
/** 是否位于生产运行时目录;工具据此拒绝在未显式确认时写入生产库。 */
static boolean isProductionPath(Path database) {
return database.startsWith(Path.of(PRODUCTION_DIR).toAbsolutePath().normalize());
}
// ---------- 体检 ----------
static Report inspect(Connection connection) throws SQLException {
Report report = new Report();
report.hasHeaderTable = tableExists(connection, "gidToKey");
report.hasPageTable = tableExists(connection, "ImageKeyCache");
if (!report.hasHeaderTable || !report.hasPageTable)
return report;
report.headerRows = count(connection, "select count(*) from gidToKey");
report.pageRows = count(connection, "select count(*) from ImageKeyCache");
report.duplicateHeaderGids = count(connection,
"select count(*) from (select gid from gidToKey group by gid having count(*) > 1)");
report.duplicatePageGroups = count(connection,
"select count(*) from (select gid, page from ImageKeyCache group by gid, page having count(*) > 1)");
try (PreparedStatement statement = connection.prepareStatement(BAD_GID_QUERY);
ResultSet rows = statement.executeQuery()) {
while (rows.next()) {
String gid = rows.getString(1);
report.badGids.add(gid);
report.badReasons.put(gid, describe(rows));
}
}
try (PreparedStatement statement = connection.prepareStatement(
"select distinct gid from ImageKeyCache where gid not in (select gid from gidToKey)");
ResultSet rows = statement.executeQuery()) {
while (rows.next())
report.orphanGids.add(rows.getString(1));
}
return report;
}
/**
* 一个 gid 触发的全部不健康条件集中在一句里检查,避免多次全表扫描。
*
* <p>必须按 gid 分组后再判断:重复 gid(同一 gid 有多条头部记录)本身就是要清除的对象,
* 而重复行里的某一条可能恰好与页 key 数一致,逐行判断会把它当成健康记录漏掉。
*/
private static final String BAD_GID_QUERY = """
select g.gid as gid,
count(*) as header_rows,
max(g.pages) as pages,
min(g.key) as key,
(select count(*) from ImageKeyCache i where i.gid = g.gid) as page_rows,
(select count(distinct i.page) from ImageKeyCache i where i.gid = g.gid) as distinct_pages,
(select coalesce(min(i.page), 0) from ImageKeyCache i where i.gid = g.gid) as min_page,
(select coalesce(max(i.page), 0) from ImageKeyCache i where i.gid = g.gid) as max_page,
(select count(*) from ImageKeyCache i where i.gid = g.gid
and (i.imgkey is null or trim(i.imgkey) = '')) as dirty_keys
from gidToKey g
group by g.gid
having count(*) > 1
or max(g.pages) is null or max(g.pages) <= 0
or min(g.key) is null or trim(min(g.key)) = ''
or (select count(*) from ImageKeyCache i where i.gid = g.gid) <> max(g.pages)
or (select count(distinct i.page) from ImageKeyCache i where i.gid = g.gid) <> max(g.pages)
or (select coalesce(min(i.page), 0) from ImageKeyCache i where i.gid = g.gid) <> 1
or (select coalesce(max(i.page), 0) from ImageKeyCache i where i.gid = g.gid) <> max(g.pages)
or (select count(*) from ImageKeyCache i where i.gid = g.gid
and (i.imgkey is null or trim(i.imgkey) = '')) > 0
order by g.gid
""";
private static String describe(ResultSet row) throws SQLException {
List<String> reasons = new ArrayList<>();
int headerRows = row.getInt("header_rows");
int pages = row.getInt("pages");
String key = row.getString("key");
int pageRows = row.getInt("page_rows");
int distinctPages = row.getInt("distinct_pages");
int minPage = row.getInt("min_page");
int maxPage = row.getInt("max_page");
int dirtyKeys = row.getInt("dirty_keys");
if (headerRows > 1) reasons.add("重复头部记录 " + headerRows + " 条");
if (key == null || key.isBlank()) reasons.add("key 为空");
if (pages <= 0) reasons.add("pages 非法(" + pages + ")");
if (pageRows != pages) reasons.add("页 key 行数 " + pageRows + " 与 pages " + pages + " 不符");
if (distinctPages != pages) reasons.add("页码去重后 " + distinctPages + " 个");
if (minPage != 1 || maxPage != pages) reasons.add("页码区间 " + minPage + "-" + maxPage);
if (dirtyKeys > 0) reasons.add("imgkey 为空 " + dirtyKeys + " 行");
return String.join("、", reasons);
}
// ---------- 修复 ----------
/** 删除不健康 gid 的整组索引与孤儿页 key,返回被移除索引的 gid 个数。 */
static int repair(Connection connection, List<String> badGids, List<String> orphanGids)
throws SQLException {
int touched = 0;
try (PreparedStatement pages = connection.prepareStatement("delete from ImageKeyCache where gid = ?");
PreparedStatement header = connection.prepareStatement("delete from gidToKey where gid = ?")) {
for (String gid : badGids) {
pages.setString(1, gid);
pages.executeUpdate();
header.setString(1, gid);
header.executeUpdate();
touched++;
}
for (String gid : orphanGids) {
pages.setString(1, gid);
if (pages.executeUpdate() > 0)
touched++;
}
}
return touched;
}
// ---------- 基础设施 ----------
private static boolean tableExists(Connection connection, String table) throws SQLException {
try (PreparedStatement statement = connection.prepareStatement(
"select count(*) from sqlite_master where type='table' and name=?")) {
statement.setString(1, table);
try (ResultSet rows = statement.executeQuery()) {
return rows.next() && rows.getInt(1) > 0;
}
}
}
private static int count(Connection connection, String sql) throws SQLException {
try (Statement statement = connection.createStatement();
ResultSet rows = statement.executeQuery(sql)) {
return rows.next() ? rows.getInt(1) : 0;
}
}
private static Map<String, String> parseOptions(String[] args) {
Map<String, String> options = new LinkedHashMap<>();
for (int i = 0; i < args.length; i++) {
String arg = args[i];
if (!arg.startsWith("--"))
continue;
String name = arg.substring(2);
if (name.equals("apply") || name.equals("help") || name.equals("allow-production")) {
options.put(name, "true");
continue;
}
if (i + 1 >= args.length) {
System.err.println("缺少参数值: " + arg);
System.exit(2);
}
options.put(name, args[++i]);
}
return options;
}
private static void printUsage() {
System.out.println("""
用法: java -cp <sqlite-jdbc.jar> GalleryIndexMaintenance.java --db <cache.db> [--apply] [--allow-production]
--db <path> 必填,缓存库文件
--apply 执行修复;不加则只做 dry-run 体检
--allow-production 允许写入 /home/lionwebsite 下的数据库(默认拒绝)
退出码: 0 健康或修复成功 / 1 存在待修复项(dry-run) / 2 参数错误 /
3 拒绝写生产路径 / 4 修复后仍不健康 / 5 执行异常
""");
}
private GalleryIndexMaintenance() { }
/** 一次体检结果。 */
static final class Report {
boolean hasHeaderTable;
boolean hasPageTable;
int headerRows;
int pageRows;
int duplicateHeaderGids;
int duplicatePageGroups;
final List<String> badGids = new ArrayList<>();
final List<String> orphanGids = new ArrayList<>();
final Map<String, String> badReasons = new LinkedHashMap<>();
boolean isHealthy() {
return hasHeaderTable && hasPageTable && badGids.isEmpty() && orphanGids.isEmpty();
}
String render(String title) {
StringBuilder out = new StringBuilder();
out.append("=== 图片索引").append(title).append("(").append(Instant.now()).append(")===\n");
if (!hasHeaderTable || !hasPageTable) {
out.append("缺少缓存表: gidToKey=").append(hasHeaderTable)
.append(" ImageKeyCache=").append(hasPageTable).append('\n');
return out.toString();
}
out.append("gidToKey 行数: ").append(headerRows).append('\n');
out.append("ImageKeyCache 行数: ").append(pageRows).append('\n');
out.append("重复 gid 组: ").append(duplicateHeaderGids).append('\n');
out.append("重复 (gid,page) 组: ").append(duplicatePageGroups).append('\n');
out.append("不完整或冲突 gid: ").append(badGids.size()).append('\n');
int limit = 20;
for (int i = 0; i < badGids.size() && i < limit; i++) {
String gid = badGids.get(i);
out.append(" - gid=").append(gid).append(' ')
.append(badReasons.getOrDefault(gid, "")).append('\n');
}
if (badGids.size() > limit)
out.append(" ... 其余 ").append(badGids.size() - limit).append(" 个略\n");
out.append("孤儿页 key 的 gid: ").append(orphanGids.size()).append('\n');
out.append(isHealthy() ? "结论: 健康\n" : "结论: 需要修复\n");
return out.toString();
}
}
}
@@ -21,6 +21,20 @@ public class AbstractMessage {
public static final byte SUBSCRIPTION_SNAPSHOT_MESSAGE = 9; public static final byte SUBSCRIPTION_SNAPSHOT_MESSAGE = 9;
// ---- 协议 v2(可靠任务)。旧类型 0..9 的字段保持不变,不能新增字段。 ----
/** 能力协商:{@code {protocolVersion: 2, features: ["reliable-task-v2"]}}。 */
public static final byte PROTOCOL_HELLO_MESSAGE = 10;
/** 带稳定 operationId 的任务命令(DOWNLOAD/DELETE)。 */
public static final byte TASK_COMMAND_MESSAGE = 11;
/** 带 reportId + revision 的状态上报,替代旧的无确认上报。 */
public static final byte TASK_REPORT_MESSAGE = 12;
/** 主站提交事务成功后的上报回执,节点据此清理快照。 */
public static final byte TASK_REPORT_ACK_MESSAGE = 13;
public byte messageType; public byte messageType;
public int messageId; public int messageId;
@@ -45,6 +45,10 @@ public class MessageCodec extends ByteToMessageCodec<AbstractMessage> {
case AbstractMessage.MAINTAIN_MESSAGE -> objectMapper.readValue(metadata, MaintainMessage.class); case AbstractMessage.MAINTAIN_MESSAGE -> objectMapper.readValue(metadata, MaintainMessage.class);
case AbstractMessage.AVAILABLE_CHECK_MESSAGE -> objectMapper.readValue(metadata, AvailableCheckMessage.class); case AbstractMessage.AVAILABLE_CHECK_MESSAGE -> objectMapper.readValue(metadata, AvailableCheckMessage.class);
case AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE -> objectMapper.readValue(metadata, SubscriptionSnapshotMessage.class); case AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE -> objectMapper.readValue(metadata, SubscriptionSnapshotMessage.class);
case AbstractMessage.PROTOCOL_HELLO_MESSAGE -> objectMapper.readValue(metadata, ProtocolHelloMessage.class);
case AbstractMessage.TASK_COMMAND_MESSAGE -> objectMapper.readValue(metadata, TaskCommandMessage.class);
case AbstractMessage.TASK_REPORT_MESSAGE -> objectMapper.readValue(metadata, TaskReportMessage.class);
case AbstractMessage.TASK_REPORT_ACK_MESSAGE -> objectMapper.readValue(metadata, TaskReportAckMessage.class);
default -> null; default -> null;
}; };
@@ -0,0 +1,30 @@
package com.lion.lionwebsite.Message;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.util.ArrayList;
import java.util.List;
/** 协议 v2 能力协商(类型 10)。双方都以对方声明了 version>=2 为前提才启用新消息。 */
@Data
@NoArgsConstructor
public class ProtocolHelloMessage extends AbstractMessage {
{
messageType = PROTOCOL_HELLO_MESSAGE;
}
public static final int PROTOCOL_V2 = 2;
public static final String FEATURE_RELIABLE_TASK_V2 = "reliable-task-v2";
private int protocolVersion = PROTOCOL_V2;
private List<String> features = new ArrayList<>(List.of(FEATURE_RELIABLE_TASK_V2));
/** 对端是否声明了 v2 可靠任务能力。 */
public boolean supportsReliableTasks() {
return protocolVersion >= PROTOCOL_V2
&& features != null && features.contains(FEATURE_RELIABLE_TASK_V2);
}
}
@@ -0,0 +1,29 @@
package com.lion.lionwebsite.Message;
import lombok.Data;
/**
* 任务命令(类型 11)。
*
* <p>只有确认对端支持 v2 之后才发送。同一逻辑操作在重试/重连后必须复用同一个
* {@link #operationId},节点据此返回原结果而不重复执行副作用。
*/
@Data
public class TaskCommandMessage extends AbstractMessage {
{
messageType = TASK_COMMAND_MESSAGE;
}
public static final String ACTION_DOWNLOAD = "DOWNLOAD";
public static final String ACTION_DELETE = "DELETE";
/** 稳定操作 ID:DOWNLOAD 与 DELETE 各自独立。 */
private String operationId;
private String action;
private int gid;
private String name;
}
@@ -0,0 +1,29 @@
package com.lion.lionwebsite.Message;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* 上报回执(类型 13)。
*
* <p>只在任务行与回执已经提交之后才发送;重复上报(同一 reportId)也会得到回执,
* 让节点停止重放。{@link #result} 0 表示已落库。
*/
@Data
@NoArgsConstructor
public class TaskReportAckMessage extends AbstractMessage {
{
messageType = TASK_REPORT_ACK_MESSAGE;
}
public static final byte RESULT_SUCCESS = 0;
private String reportId;
private byte result;
public TaskReportAckMessage(String reportId, byte result) {
this.reportId = reportId;
this.result = result;
}
}
@@ -0,0 +1,32 @@
package com.lion.lionwebsite.Message;
import com.lion.lionwebsite.Domain.GalleryTask;
import lombok.Data;
import java.util.LinkedHashMap;
import java.util.Map;
/**
* 节点状态上报(类型 12)。
*
* <p>相比旧的无确认上报,这里带上单调递增的 {@link #revision} 与唯一 {@link #reportId}:
* 主站据此丢弃乱序/重放的旧快照,并在事务提交后回 {@link TaskReportAckMessage},
* 节点只有在收到匹配回执后才清理自己的快照。
*/
@Data
public class TaskReportMessage extends AbstractMessage {
{
messageType = TASK_REPORT_MESSAGE;
}
/** 本次上报的唯一 ID,重复投递同一份快照时保持不变。 */
private String reportId;
/** 节点侧单调递增版本号;主站只接受比已记录更大的 revision。 */
private long revision;
private GalleryTask[] galleryTasks;
/** gid → 产生该状态的下载 operationId,用于丢弃属于旧操作的迟到上报。 */
private Map<Integer, String> operationIds = new LinkedHashMap<>();
}
@@ -0,0 +1,262 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
import com.lion.lionwebsite.Domain.GidToKey;
import com.lion.lionwebsite.Domain.ImageKeyCache;
import com.lion.lionwebsite.Exception.GalleryIndexException;
import com.lion.lionwebsite.Util.GalleryUtil;
import com.lion.lionwebsite.Util.SingleFlight;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.transaction.support.TransactionOperations;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
/**
* 图片索引(gid 到每页 imgkey)的唯一读写入口。
*
* <p>这个类替换掉原先「读缓存 / 写缓存 / 回滚」分散在业务服务里、并且逐页 INSERT
* 的写法。三件事在这里被固定下来:
*
* <ol>
* <li><b>同 gid 合并</b>:并发请求同一本画廊时只有一个线程真的去抓上游,
* 其余线程等待同一个结果(含失败),因此不会出现两个构建者互相删除对方写入的索引。</li>
* <li><b>抓取在事务外</b>:上游 HTTP 可能耗时十几秒,绝不能占着 SQLite 写事务,
* 否则整个缓存库的写入都会排队。</li>
* <li><b>整组短事务发布</b>:一个事务里先删该 gid 的旧索引,再批量写入新索引。
* 失败整体回滚,磁盘上仍保留旧的完整索引;只有写入成功这一个状态是对外可见的。</li>
* </ol>
*
* <p>依赖全部由构造函数注入:mapper 负责落库,operations 负责事务边界,
* upstream 负责把 mpv 页解析成页 key 列表。测试可以直接给假的实现,
* 不需要起 Spring 上下文,也不需要网络。
*/
public class GalleryIndexService {
private static final Logger log = LoggerFactory.getLogger(GalleryIndexService.class);
/** 单条 INSERT 携带的最大行数,避免 SQL 过长。 */
static final int BATCH_SIZE = 200;
private final ImageCacheMapper mapper;
private final TransactionOperations operations;
private final TransactionOperations readOperations;
private final GalleryKeyUpstream upstream;
private final SingleFlight<String, GidToKey> builds = new SingleFlight<>();
public GalleryIndexService(ImageCacheMapper mapper, TransactionOperations operations,
GalleryKeyUpstream upstream) {
this(mapper, operations, operations, upstream);
}
/**
* @param operations 发布索引的写事务(删除加批量写入必须整体生效)
* @param readOperations 读取索引的一致性快照事务;传 null 表示不额外开启读事务
*/
public GalleryIndexService(ImageCacheMapper mapper, TransactionOperations operations,
TransactionOperations readOperations, GalleryKeyUpstream upstream) {
if (mapper == null || operations == null || upstream == null)
throw new IllegalArgumentException("mapper/operations/upstream 不能为空");
this.mapper = mapper;
this.operations = operations;
this.readOperations = readOperations;
this.upstream = upstream;
}
/**
* 保证该画廊的索引可用,返回 gid 到 key 的映射。
*
* @param url 画廊链接,形如 https://exhentai.org/g/1596929/f08534d87d/
* @throws GalleryIndexException 链接错误 / 画廊不可用 / 暂时失败,见 GalleryIndexException.Reason
*/
public GidToKey ensureIndex(String url) throws GalleryIndexException {
return ensureIndex(url, this::buildAndPublish);
}
/**
* 同 ensureIndex,但允许调用方提供构建动作。
*
* <p>离线重建需要「先检查再决定是否重建」的不同策略,因此把构建步骤作为参数暴露出来;
* 生产调用使用默认实现即可。
*/
GidToKey ensureIndex(String url, IndexBuilder builder) throws GalleryIndexException {
GalleryLink link = GalleryLink.parse(url);
GidToKey cached = lookup(link);
if (cached != null)
return cached;
try {
return builds.run(link.gid(), () -> {
// 进入临界区后必须再看一眼:等待期间别的线程可能已经把索引建好了。
GidToKey existing = lookup(link);
return existing != null ? existing : builder.build(url);
});
} catch (GalleryIndexException e) {
throw e;
} catch (Exception e) {
throw new GalleryIndexException(GalleryIndexException.Reason.TRANSIENT,
"构建图片索引失败 gid=" + link.gid(), e);
}
}
/**
* 已缓存且完整则返回索引,否则返回 null。
*
* <p>完整性由页码集合判定,而不是行数:历史重复行与半截写入都会让行数与页数不符。
*/
GidToKey lookup(GalleryLink link) {
if (readOperations != null)
return readOperations.execute(status -> lookupInTransaction(link));
return lookupInTransaction(link);
}
/**
* 在同一个读事务里读头部与页码,得到一个一致的快照。
*
* <p>两个独立的 SELECT 之间可能夹着另一个线程的整组替换,于是头部是旧的、页码是新的
* (或相反),完整性判断会得出错误结论。SQLite 的读事务能看到固定快照,避免这种交错。
*/
private GidToKey lookupInTransaction(GalleryLink link) {
GidToKey header = mapper.selectKeyByGid(link.gid());
if (header == null)
return null;
if (header.getPages() <= 0 || header.getKey() == null || header.getKey().isBlank())
return null;
if (!isComplete(header))
return null;
return header;
}
private boolean isComplete(GidToKey header) {
List<Integer> pages = mapper.selectPageNumbersByGid(header.getGid());
if (pages == null || pages.size() != header.getPages())
return false;
if (mapper.countInvalidImageKeysByGid(header.getGid()) != 0)
return false;
// 页码必须是严格递增的 1..pages,重复或跳号都视为不完整。
for (int i = 0; i < pages.size(); i++) {
Integer page = pages.get(i);
if (page == null || page != i + 1)
return false;
}
return true;
}
/** 抓上游(事务外)后短事务整组替换。 */
private GidToKey buildAndPublish(String url) throws GalleryIndexException {
GalleryLink link = GalleryLink.parse(url);
List<ImageKeyCache> pageKeys;
try {
// 抓取必须发生在事务之外:上游可能耗时十几秒,占着 SQLite 写事务会拖垮整个缓存库。
pageKeys = upstream.fetchPageKeys(url);
} catch (Exception e) {
throw new GalleryIndexException(GalleryIndexException.Reason.TRANSIENT,
"抓取图片索引失败 gid=" + link.gid(), e);
}
if (pageKeys == null || pageKeys.isEmpty())
throw new GalleryIndexException(GalleryIndexException.Reason.UNAVAILABLE,
"该图片已下架或已被删除 gid=" + link.gid());
// 落库前先校验上游数据自洽:整组替换一旦开始就没有回头路,
// 与其把畸形页码写进唯一约束再回滚,不如在此之前就判为失败。
validatePageKeys(link, pageKeys);
GidToKey header = new GidToKey();
header.setGid(link.gid());
header.setKey(link.key());
header.setPages(pageKeys.size());
publish(header, pageKeys);
return header;
}
/**
* 校验上游返回的页 key 是否自洽。
*
* <p>四条要求缺一不可,否则索引会以「看起来完整」的样子写进库:
* gid 属于请求的那本画廊、页码恰好是 1..N、页码不重复、imgkey 非空。
* 任一不满足都抛出 {@link GalleryIndexException.Reason#MALFORMED},此时尚未开启写事务,
* 库中旧索引保持原样。
*/
void validatePageKeys(GalleryLink link, List<ImageKeyCache> pageKeys) throws GalleryIndexException {
int expected = pageKeys.size();
Set<Integer> seen = new HashSet<>(expected * 2);
for (ImageKeyCache row : pageKeys) {
if (row == null || row.getGid() == null || !row.getGid().equals(link.gid()))
throw malformed(link, "上游页 key 的 gid 与请求不一致");
if (row.getImgkey() == null || row.getImgkey().isBlank())
throw malformed(link, "上游页 key 的 imgkey 为空");
int page = row.getPage();
if (page < 1 || page > expected)
throw malformed(link, "上游页码越界: " + page + "(共 " + expected + " 页)");
if (!seen.add(page))
throw malformed(link, "上游页码重复: " + page);
}
if (seen.size() != expected)
throw malformed(link, "上游页码不连续,去重后 " + seen.size() + " 个,期望 " + expected + " 个");
}
private GalleryIndexException malformed(GalleryLink link, String detail) {
log.warn("上游图片索引非法 gid={} detail={}", link.gid(), detail);
return new GalleryIndexException(GalleryIndexException.Reason.MALFORMED,
detail + " gid=" + link.gid());
}
/**
* 在单个事务里整组替换索引。
*
* <p>失败时事务整体回滚:要么看到旧的完整索引,要么看到新的完整索引,
* 不存在「头部已更新、页 key 缺失」的中间态。
*/
void publish(GidToKey header, List<ImageKeyCache> pageKeys) {
operations.executeWithoutResult(status -> {
mapper.deleteImageKeyCacheByGid(header.getGid());
mapper.deleteGidToKey(header.getGid());
mapper.insertGidToKey(header);
writeBatches(pageKeys);
});
log.info("图片索引已整体替换 gid={} pages={}", header.getGid(), header.getPages());
}
private void writeBatches(List<ImageKeyCache> pageKeys) {
for (int start = 0; start < pageKeys.size(); start += BATCH_SIZE) {
int end = Math.min(start + BATCH_SIZE, pageKeys.size());
mapper.insertImageKeyCacheBatch(new ArrayList<>(pageKeys.subList(start, end)));
}
}
/** 便于日志与错误信息使用的解析结果。 */
record GalleryLink(String gid, String key) {
static GalleryLink parse(String url) throws GalleryIndexException {
String[] segments = url == null ? null : url.split("/");
Integer gid = GalleryUtil.parseGid(url);
if (gid == null || segments == null || segments.length <= 5 || segments[5].isBlank())
throw new GalleryIndexException(GalleryIndexException.Reason.INVALID_LINK,
"链接错误");
return new GalleryLink(String.valueOf(gid), segments[5].strip());
}
}
/**
* 上游取页 key 的端口。
*
* <p>生产实现只是把 URL 转交给 GalleryUtil.parseImageKeys;
* 单独抽出来是为了让本服务在测试里完全脱离网络与静态工具类。
*/
@FunctionalInterface
public interface GalleryKeyUpstream {
List<ImageKeyCache> fetchPageKeys(String url) throws Exception;
}
/**
* 构建索引的动作。允许抛出业务异常,因此不用 java.util.function.Function
* (后者不能抛受检异常,会让调用方被迫包装成运行时异常)。
*/
@FunctionalInterface
interface IndexBuilder {
GidToKey build(String url) throws GalleryIndexException;
}
}
@@ -11,7 +11,7 @@ import com.lion.lionwebsite.Util.ImageFileCache;
import java.nio.file.Path; import java.nio.file.Path;
import com.lion.lionwebsite.Util.GalleryUtil; import com.lion.lionwebsite.Util.GalleryUtil;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import tools.jackson.databind.ObjectMapper; import tools.jackson.databind.node.ObjectNode;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
import lombok.Data; import lombok.Data;
@@ -40,14 +40,22 @@ public class GalleryManageService {
final UserMapper userMapper; final UserMapper userMapper;
final ShareFileMapper shareFileMapper;
final ImageCacheMapper imageCacheMapper; final ImageCacheMapper imageCacheMapper;
final RemoteService remoteService; final RemoteService remoteService;
final PushService pushService; final PushService pushService;
/**
* 任务生命周期(协议 v2)。创建/删除都先走它做持久化意图,再产生外部副作用。
* 用字段注入是为了兼容现有构造式 mock 测试与旧调用点。
*/
@org.springframework.beans.factory.annotation.Autowired(required = false)
GalleryTaskLifecycleService lifecycleService;
@org.springframework.beans.factory.annotation.Autowired
GalleryIndexService galleryIndexService;
/** /**
* 创建任务 * 创建任务
* @param link 任务链接 * @param link 任务链接
@@ -81,6 +89,20 @@ public class GalleryManageService {
pushService.taskCreateReport(user.getUsername(), taskName, response); pushService.taskCreateReport(user.getUsername(), taskName, response);
return response.toJSONString(); return response.toJSONString();
} }
// 协议 v2 生命周期:gid 唯一意图先落库(占位行「提交中」),提交前持久化 SUBMITTING,
// 上游结果不确定时报 UNKNOWN 而不是自动重发;差额额度与任务行在同一短事务里写入。
if (lifecycleService != null) {
response = lifecycleService.createTask(gid, taskName, link, targetResolution, user.getId());
Gallery current = galleryMapper.selectGalleryByGid(gid);
taskName = current == null || current.getName() == null ? taskName : current.getName();
// 业务结果由生命周期服务给出,data 用结构化对象下发(前端解码器兼容旧字符串)。
if (response.isSuccess() && current != null)
response.success(objectMapper.valueToTree(current));
pushService.taskCreateReport(user.getUsername(), taskName, response);
return response.toJSONString();
}
Gallery gallery; Gallery gallery;
//判断数据库中是否有这个任务,有则返回状态 //判断数据库中是否有这个任务,有则返回状态
@@ -126,7 +148,7 @@ public class GalleryManageService {
// Do not overwrite an immediate node status with the original submitted state. // Do not overwrite an immediate node status with the original submitted state.
Gallery current = galleryMapper.selectGalleryByGid(gallery.getGid()); Gallery current = galleryMapper.selectGalleryByGid(gallery.getGid());
response.success((current == null ? gallery : current).toString()); response.success(objectMapper.valueToTree(current == null ? gallery : current));
pushService.taskCreateReport(user.getUsername(), taskName, response); pushService.taskCreateReport(user.getUsername(), taskName, response);
return response.toJSONString(); return response.toJSONString();
@@ -186,7 +208,7 @@ public class GalleryManageService {
try { try {
gallery = GalleryUtil.parse(link, false, null); gallery = GalleryUtil.parse(link, false, null);
if (gallery != null) if (gallery != null)
response.success(new ObjectMapper().valueToTree(gallery).toString()); response.success(objectMapper.valueToTree(gallery));
else else
response.failure("查询失败"); response.failure("查询失败");
} catch (Exception e) { } catch (Exception e) {
@@ -195,7 +217,7 @@ public class GalleryManageService {
} }
else else
response.success(new ObjectMapper().valueToTree(gallery).toString()); response.success(objectMapper.valueToTree(gallery));
return response.toJSONString(); return response.toJSONString();
} }
@@ -214,7 +236,7 @@ public class GalleryManageService {
response.failure("未找到该图片,请使用链接"); response.failure("未找到该图片,请使用链接");
else else
response.success(gallery.toString()); response.success(objectMapper.valueToTree(gallery));
return response.toJSONString(); return response.toJSONString();
} }
@@ -234,26 +256,62 @@ public class GalleryManageService {
} }
ArrayList<Integer> galleryIds = collectMapper.selectGidByCollector(userId); ArrayList<Integer> galleryIds = collectMapper.selectGidByCollector(userId);
Iterator<Integer> idIterator; // 原先对每个画廊线性扫描收藏列表(O(n*m))并就地删除迭代器元素;
// 改成集合查找后同样是「命中即标记」,复杂度降到 O(n+m)。
if (!galleryIds.isEmpty()) //如果该用户收藏了图片 if (!galleryIds.isEmpty()) {
galleryLoop:for (Gallery gallery : galleries) { //遍历图片 Set<Integer> collected = new HashSet<>(galleryIds);
idIterator = galleryIds.iterator(); for (Gallery gallery : galleries)
while (idIterator.hasNext()) { //遍历收藏的gid if (collected.contains(gallery.getGid()))
Integer id = idIterator.next();
if (id.equals(gallery.getGid())) { //如果找到对应的gid,修改对应图片的属性,删除当前gid,判断是否需要跳出或者结束循环
gallery.setCollect(true); gallery.setCollect(true);
idIterator.remove();
if (galleryIds.isEmpty())
break galleryLoop;
else
continue galleryLoop;
}
}
} }
response.success(new ObjectMapper().valueToTree(galleries).toString()); // 下载人昵称只对管理员下发;一次建表避免逐条查询。
if (userId == UserService.ADMIN_USER_ID)
fillDownloaderNames(galleries);
response.success(objectMapper.valueToTree(galleries));
return response.toJSONString();
}
/** 给任务列表补上下载人昵称,找不到用户时保留空值而不是报错。 */
private void fillDownloaderNames(Gallery[] galleries) {
User[] users = userMapper.selectAllUser();
Map<Integer, String> names = new HashMap<>();
if (users != null)
for (User user : users)
names.put(user.getId(), user.getUsername());
for (Gallery gallery : galleries)
gallery.setDownloaderName(names.get(gallery.getDownloader()));
}
/**
* 查询某个任务的实际下载人昵称。
* 只有管理员可调用,普通用户一律拒绝,避免暴露他人身份。
*
* @param gid 任务 gid
* @param AuthCode 调用方授权码
*/
public String selectDownloaderByGid(int gid, String AuthCode) {
Response response = Response.generateResponse();
User requester = userMapper.selectUserByAuthCode(AuthCode);
if (requester == null || requester.getId() != UserService.ADMIN_USER_ID) {
response.failure("无权查看下载人");
return response.toJSONString();
}
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
if (gallery == null) {
response.failure("任务不存在");
return response.toJSONString();
}
User downloader = userMapper.selectUserById(gallery.getDownloader());
ObjectNode node = objectMapper.createObjectNode();
node.put("gid", gid);
node.put("downloader", gallery.getDownloader());
node.put("downloaderName", downloader == null ? "" : downloader.getUsername());
response.success(node);
return response.toJSONString(); return response.toJSONString();
} }
@@ -268,7 +326,7 @@ public class GalleryManageService {
Gallery gallery = galleryMapper.selectGalleryByName("%" + name + "%"); Gallery gallery = galleryMapper.selectGalleryByName("%" + name + "%");
if (gallery != null) if (gallery != null)
response.success(new ObjectMapper().valueToTree(gallery).toString()); response.success(objectMapper.valueToTree(gallery));
else else
response.failure("没有找到该名字的图片"); response.failure("没有找到该名字的图片");
@@ -286,7 +344,7 @@ public class GalleryManageService {
Gallery[] galleries = galleryMapper.selectGalleryByDownloader(userMapper.selectUserByAuthCode(AuthCode).getId()); Gallery[] galleries = galleryMapper.selectGalleryByDownloader(userMapper.selectUserByAuthCode(AuthCode).getId());
if (galleries.length > 0) if (galleries.length > 0)
response.success(new ObjectMapper().valueToTree(galleries).toString()); response.success(objectMapper.valueToTree(galleries));
else else
response.failure("您未下载图片"); response.failure("您未下载图片");
@@ -324,8 +382,46 @@ public class GalleryManageService {
return response.toJSONString(); return response.toJSONString();
} }
// 通过授权后才落库并通知节点,避免被拒请求仍删除节点文件。 // 通过授权后才产生外部副作用。协议 v2 的删除顺序:
// 先落「待删除」意图 → 节点确认(或文件已不存在)→ 才在同一事务删任务行并留墓碑。
// 失败时任务行与意图都保留,可沿原记录重试并复用同一个删除 operationId。
log.info("删除图片{}", gallery.getName()); log.info("删除图片{}", gallery.getName());
if (lifecycleService != null) {
// 被拒绝(例如任务仍在提交中)时必须回业务失败:
// 绝不能退避到裸删除,否则会与并发的创建 POST 同时执行并写回已删/重建的 gid。
GalleryTaskLifecycleService.DeleteIntent intent = lifecycleService.beginDelete(gallery.getGid());
if (!intent.accepted()) {
response.failure(intent.message());
log.info("拒绝删除 gid={}:{}", gid, intent.message());
return response.toJSONString();
}
byte result = remoteService.sendDeleteCommand(intent.deleteOperationId(),
gallery.getGid(), gallery.getName());
switch (result) {
case 0 -> {
lifecycleService.completeDelete(gallery.getGid(), intent.deleteOperationId());
response.success();
}
// 文件已不存在等于目标达成:同样完成本地清理,避免留下永远删不掉的任务行。
case ErrorCode.FILE_NOT_FOUND -> {
lifecycleService.completeDelete(gallery.getGid(), intent.deleteOperationId());
response.success();
}
case ErrorCode.IO_ERROR -> {
lifecycleService.failDelete(gallery.getGid());
response.failure("图片:" + gallery.getName() + "删除失败,IO错误");
}
// 节点无响应/超时:保留记录与待删除意图,可重试。
default -> {
lifecycleService.failDelete(gallery.getGid());
response.failure("图片:" + gallery.getName() + "删除失败,节点无响应");
}
}
if (!response.isSuccess())
log.info(response.getData());
return response.toJSONString();
}
galleryMapper.deleteGalleryByGid(gallery.getGid()); galleryMapper.deleteGalleryByGid(gallery.getGid());
switch (remoteService.deleteGallery(gallery)) { switch (remoteService.deleteGallery(gallery)) {
case ErrorCode.IO_ERROR -> response.failure("图片:" + gallery.getName() + "删除失败,IO错误"); case ErrorCode.IO_ERROR -> response.failure("图片:" + gallery.getName() + "删除失败,IO错误");
@@ -365,36 +461,26 @@ public class GalleryManageService {
lastResetAmountTime == null || lastResetAmountTime.getValue() == null lastResetAmountTime == null || lastResetAmountTime.getValue() == null
? "" : lastResetAmountTime.getValue()); ? "" : lastResetAmountTime.getValue());
response.success(new ObjectMapper().valueToTree(data).toString()); response.success(objectMapper.valueToTree(data));
return response.toJSONString(); return response.toJSONString();
} }
public String cacheImagesKey(String url) { public String cacheImagesKey(String url) {
Response response = Response.generateResponse();
String gid = String.valueOf(GalleryUtil.parseGid(url));
GidToKey gidToKey = imageCacheMapper.selectKeyByGid(gid);
//已缓存过,直接返回
if(gidToKey != null) {
return response.success(objectMapper.valueToTree(gidToKey)).toJSONString();
}
try { try {
gidToKey = new GidToKey(); GidToKey index = galleryIndexService.ensureIndex(url);
gidToKey.setGid(gid); return Response.generateResponse().success(objectMapper.valueToTree(index)).toJSONString();
gidToKey.setKey(url.split("/")[5].strip()); } catch (com.lion.lionwebsite.Exception.GalleryIndexException error) {
ArrayList<ImageKeyCache> imageKeyCaches = GalleryUtil.parseImageKeys(url); String message = switch (error.getReason()) {
if(imageKeyCaches == null) case INVALID_LINK -> "链接错误";
return response.failure("该图片已下架或已被删除").toJSONString(); case UNAVAILABLE -> "该图片已下架或已被删除";
gidToKey.setPages(imageKeyCaches.size()); case MALFORMED, TRANSIENT -> "网络波动或其他异常";
imageCacheMapper.insertGidToKey(gidToKey); };
for (ImageKeyCache imageKeyCache : imageKeyCaches) log.warn("缓存图片索引失败 reason={}", error.getReason());
imageCacheMapper.insertImageKeyCache(imageKeyCache); return Response._failure(message);
response.success(objectMapper.valueToTree(gidToKey)); } catch (RuntimeException error) {
}catch (IOException e){ log.warn("缓存图片索引失败 errorType={}", error.getClass().getSimpleName());
log.error(e.getMessage()); return Response._failure("网络波动或其他异常");
response.failure("网络波动或其他异常");
} }
return response.toJSONString();
} }
public Callable<?> getCachedImage(String gid, Integer page, HttpServletRequest request, HttpServletResponse response) { public Callable<?> getCachedImage(String gid, Integer page, HttpServletRequest request, HttpServletResponse response) {
@@ -442,10 +528,24 @@ public class GalleryManageService {
Gallery[] galleries = galleryMapper.selectUnDoneGalleries(); Gallery[] galleries = galleryMapper.selectUnDoneGalleries();
if(galleries != null && galleries.length != 0) { if(galleries != null && galleries.length != 0) {
log.info("发送未下载完成图片至节点,{}本", galleries.length); // 只重投「确实已提交上游」的任务:提交未知/预占中/待删除的任务重投会变成
for (Gallery gallery : galleries) // 未计量副作用(节点侧显示下载完成但没有任何额度记录)。
int sent = 0;
int skipped = 0;
for (Gallery gallery : galleries) {
if (lifecycleService != null
&& !lifecycleService.resolveDispatch(gallery.getGid()).dispatchable()) {
skipped++;
continue;
}
remoteService.addGalleryToQueue(gallery); remoteService.addGalleryToQueue(gallery);
response.success(String.format("发送未下载完成图片至节点,%s本", galleries.length)); sent++;
}
log.info("发送未下载完成图片至节点,{}本(跳过需要对账的{}本)", sent, skipped);
if (sent == 0)
response.failure("没有可安全重投的任务,需先人工对账提交未知的任务");
else
response.success(String.format("发送未下载完成图片至节点,%s本", sent));
}else{ }else{
response.failure("当前没有未下载完成的图片"); response.failure("当前没有未下载完成的图片");
} }
@@ -461,6 +561,13 @@ public class GalleryManageService {
return Response._success("下载完成"); return Response._success("下载完成");
if(remoteService.isDead()) if(remoteService.isDead())
return Response._failure("节点不在线,无法重试"); return Response._failure("节点不在线,无法重试");
// 提交结果未知(UNKNOWN)或正在删除的任务不能重试:上游不支持幂等,重发会让
// 「可能已受理但未计量」的任务在节点侧变成下载完成却 0 额度。这类任务需人工对账。
if (lifecycleService != null) {
var decision = lifecycleService.resolveDispatch(gid);
if (decision.kind() == GalleryTaskLifecycleService.DispatchDecision.Kind.REFUSE)
return Response._failure("该任务提交结果未知或正在删除,请先人工对账后再重试");
}
RemoteService.RetryResult retryResult = remoteService.retryGallery(gallery); RemoteService.RetryResult retryResult = remoteService.retryGallery(gallery);
if(retryResult.success()) if(retryResult.success())
@@ -0,0 +1,783 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.*;
import com.lion.lionwebsite.Domain.*;
import com.lion.lionwebsite.Exception.ResolutionNotMatchException;
import com.lion.lionwebsite.Message.TaskCommandMessage;
import com.lion.lionwebsite.Message.TaskReportMessage;
import com.lion.lionwebsite.Util.PreparedGallery;
import com.lion.lionwebsite.Util.Response;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
import org.springframework.transaction.support.TransactionTemplate;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
/**
* 下载任务的持久化生命周期(协议 v2)。
*
* <p>核心约束:
* <ol>
* <li><b>先保存意图,再产生外部副作用</b>:同一 gid 的唯一 CREATE 意图先落库(占位任务行 + RESERVED 操作记录),
* 通过 gid 的主键唯一约束把并发创建收敛成「一个赢家」;</li>
* <li><b>SUBMITTING 之后绝不自动重发</b>:上游不支持幂等,提交结果不确定时只能落 UNKNOWN,等显式对账/解决;</li>
* <li><b>提交结果与额度用一个短事务写入</b>:Gallery 与 weekUsedAmount 要么一起成功,要么一起回滚,
* 事务提交之后才把 DOWNLOAD 命令下发给节点;</li>
* <li><b>删除失败保留记录</b>:先落待删除意图,节点确认(或文件已不存在)后才在同一事务里删任务行并留墓碑;
* 失败则保留 Gallery 与 DELETE_PENDING 意图,可沿原记录重试且复用同一删除 operationId。</li>
* </ol>
*
* <p>所有网络调用都发生在事务之外;事务回调内部不做 I/O,也不吞异常(吞掉会错误提交)。
*/
@Service
@Slf4j
public class GalleryTaskLifecycleService implements TaskReportHandler {
final GalleryMapper galleryMapper;
final CollectMapper collectMapper;
final CustomConfigurationMapper configurationMapper;
final UserMapper userMapper;
final TaskOperationRepository taskOperationRepository;
final UpstreamGateway upstreamGateway;
final RemoteService remoteService;
/**
* 短全局创建预占锁。
*
* <p>只有「登记唯一 CREATE 意图」这一小段数据库写入在锁内,网络调用一律在锁外。
* 单进程内它把并发同 gid 创建直接收敛成一个赢家;跨进程的正确性仍由
* {@code task_operation}/{@code gallery} 的主键唯一约束兜底。
*/
private final Object reservationLock = new Object();
/**
* 测试钩子:进入上游只读解析之前执行。
* 用来让并发删除发生在预占之后、SUBMITTING 之前。生产环境是空实现。
*/
volatile Runnable beforePrepare = () -> { };
/**
* 测试钩子:SUBMITTING 已持久化之后、真正 POST 上游之前执行。
* 用来制造「提交瞬间任务被删除」的 late generation 场景。生产环境是空实现。
*/
volatile Runnable afterMarkSubmitting = () -> { };
/** 主库事务模板;测试可注入内存 sqlite 的事务模板。 */
TransactionTemplate transactions;
public GalleryTaskLifecycleService(GalleryMapper galleryMapper,
CollectMapper collectMapper,
CustomConfigurationMapper configurationMapper,
UserMapper userMapper,
TaskOperationRepository taskOperationRepository,
UpstreamGateway upstreamGateway,
RemoteService remoteService,
@Qualifier("mainTransactionTemplate") TransactionTemplate transactions) {
this.galleryMapper = galleryMapper;
this.collectMapper = collectMapper;
this.configurationMapper = configurationMapper;
this.userMapper = userMapper;
this.taskOperationRepository = taskOperationRepository;
this.upstreamGateway = upstreamGateway;
this.remoteService = remoteService;
this.transactions = transactions;
}
/** 测试用:替换事务模板或在没有 Spring 事务时退化为直连执行。 */
void setTransactions(TransactionTemplate transactions) {
this.transactions = transactions;
}
/**
* 一次「登记创建意图」的结果。
*
* <p>只有赢家能继续做外部副作用;输家(已存在有效意图 / 数据库竞争失败)
* 一律转成业务失败,绝不再 POST 上游。
*/
private enum ReserveOutcome {
/** 已成功登记唯一 CREATE 意图,可以做只读解析与提交。 */
RESERVED,
/** 同一 gid 已存在有效意图(或任务行)。 */
DUPLICATE,
/** 登记时的数据库竞争/失败:没有产生任何副作用,必须转业务失败。 */
CONTENDED
}
/** 一次「登记创建意图」的返回:结果 + 赢家持有的操作记录。 */
private record Reservation(ReserveOutcome outcome, TaskOperation operation) {
}
/**
* 单次下发决策。
*
* <p>把「能不能把这个任务交给节点」从「下载是否已提交」里拆出来:
* 未确认上游提交结果(UNKNOWN / 预占 / 待删除 / 删除墓碑)绝不能下发,
* 否则未计量的人工对账任务可能在节点侧变成「下载完成却 0 额度」。
*/
public static final class DispatchDecision {
public enum Kind {
/** 当前 operation 已是 SUBMITTED:可以按稳定 operationId 下发。 */
READY,
/** 迁移前的旧任务行:没有操作记录但历史上确实提交过下载。 */
LEGACY,
/** 不可下发:状态不明或被删除,下发会产生未计量副作用。 */
REFUSE
}
private final Kind kind;
private final String operationId;
private DispatchDecision(Kind kind, String operationId) {
this.kind = kind;
this.operationId = operationId;
}
public Kind kind() {
return kind;
}
public String operationId() {
return operationId;
}
/** 只有 READY/LEGACY 允许真正下发到节点。 */
public boolean dispatchable() {
return kind != Kind.REFUSE && operationId != null;
}
/** 拒绝下发(供调用方与测试构造)。 */
public static DispatchDecision refuse() {
return REFUSE;
}
/** 可直接下发(供测试构造)。 */
public static DispatchDecision ready(String operationId) {
return new DispatchDecision(Kind.READY, operationId);
}
}
private static final DispatchDecision REFUSE = new DispatchDecision(DispatchDecision.Kind.REFUSE, null);
/**
* 一次删除登记的返回。
*
* <p>用显式结果替代「null 即退避到裸删除」的旧语义:
* 被拒绝时调用方必须回业务失败,不能绕过待删除意图直接删节点文件。
*/
public record DeleteIntent(boolean accepted, String deleteOperationId, String message) {
static DeleteIntent refused(String message) {
return new DeleteIntent(false, null, message);
}
static DeleteIntent accepted(String deleteOperationId) {
return new DeleteIntent(true, deleteOperationId, null);
}
}
/** 在事务里执行;没有配置事务模板时候直接执行(仅便于构造 mock 测试)。 */
private <T> T inTransaction(java.util.function.Supplier<T> body) {
TransactionTemplate template = transactions;
if (template == null)
return body.get();
return template.execute(status -> {
try {
return body.get();
} catch (RuntimeException e) {
// 回调里必须把异常继续抛出,否则 Spring 会把只读了一半的事务当成正常提交。
status.setRollbackOnly();
throw e;
}
});
}
/**
* 创建任务的持久化前置:在同一个事务里登记唯一 CREATE 意图。
*
* <p>同时写入可见的占位任务行(状态「提交中」),这样提交期间任务列表就能解析该 gid,
* 不会出现「上游已受理但本地查不到」的空窗。
*
* @return 已登记意图(赢家)返回它;同一 gid 已有有效意图返回 {@code null}(输家)。
*/
private Reservation reserveCreateIntent(int gid, String placeholderName, int downloaderId) {
// 锁内只做数据库写入:把并发同 gid 创建串行化,网络调用留在锁外。
synchronized (reservationLock) {
try {
TaskOperation operation = inTransaction(() -> {
TaskOperation existingOperation = taskOperationRepository.selectByGid(gid);
Gallery existingGallery = galleryMapper.selectGalleryByGid(gid);
if (existingOperation != null) {
// 只有「删除已完成」的墓碑才允许用全新 operationId 重新创建(显式重建)。
if (!TaskOperation.STATE_DELETED.equals(existingOperation.getState()))
return null;
if (existingGallery != null)
return null;
String operationId = TaskProtocol.newDownloadOperationId();
taskOperationRepository.reactivate(gid, operationId, TaskOperation.STATE_RESERVED, now());
insertPlaceholder(gid, placeholderName, downloaderId);
return taskOperationRepository.selectByGid(gid);
}
if (existingGallery != null)
return null;
TaskOperation created = new TaskOperation();
created.setGid(gid);
created.setOperationId(TaskProtocol.newDownloadOperationId());
created.setState(TaskOperation.STATE_RESERVED);
created.setUpdateTime(now());
taskOperationRepository.insert(created);
insertPlaceholder(gid, placeholderName, downloaderId);
return created;
});
return operation == null
? new Reservation(ReserveOutcome.DUPLICATE, null)
: new Reservation(ReserveOutcome.RESERVED, operation);
} catch (RuntimeException e) {
// 数据库竞争/失败发生在任何外部副作用之前,绝不能穿透成 500,也不能继续提交。
log.warn("登记创建意图失败,按业务失败处理 gid={} error={}", gid, e.getClass().getSimpleName());
return new Reservation(ReserveOutcome.CONTENDED, null);
}
}
}
/** 写入占位任务行:让提交期间的任务在列表里可见。 */
private void insertPlaceholder(int gid, String name, int downloaderId) {
Gallery placeholder = new Gallery();
placeholder.setGid(gid);
placeholder.setName(name);
placeholder.setStatus(TaskStatus.PENDING_SUBMIT);
placeholder.setDownloader(downloaderId);
placeholder.setCreateTime(System.currentTimeMillis() / 1000);
placeholder.setProceeding(0);
galleryMapper.insertGallery(placeholder);
}
/**
* 标记为正在提交,并用 CAS 确认这一代意图仍然有效。
*
* <p>只有当前状态与 operationId 都匹配(RESERVED → SUBMITTING)才返回 true;
* 期间被删除或重建返回 false,调用方必须停下,绝不能再 POST 上游。
*
* @return true 表示已持久化 SUBMITTING,可以安全产生上游副作用
*/
boolean markSubmitting(String operationId, int gid) {
return inTransaction(() -> taskOperationRepository.compareAndSetCreateState(gid, operationId,
TaskOperation.STATE_RESERVED, TaskOperation.STATE_SUBMITTING, now()) == 1);
}
/**
* 提交结果未知(超时/断连/中断)。保留意图,等待人工对账,不自动重发。
*
* <p>TaskOperation 与 Gallery 状态在同一个短事务里更新:
* 操作落 UNKNOWN 而任务行仍显示「提交中」会让运营侧误以为还能自动重试,
* 因此两边必须一致地表达「不确定,需要人工对账」。
*/
void markUnknown(String operationId, int gid) {
inTransaction(() -> {
int advanced = taskOperationRepository.compareAndSetCreateState(gid, operationId,
TaskOperation.STATE_SUBMITTING, TaskOperation.STATE_UNKNOWN, now());
// 只有本次这一代确实从 SUBMITTING 推进到 UNKNOWN 时才同步任务行:
// 若期间已被删除/重建(CAS 返回 0),绝不能再改那条记录。
if (advanced != 1)
return null;
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
if (gallery != null && !TaskStatus.UNKNOWN.equals(gallery.getStatus()))
galleryMapper.updateGalleryStatus(gid, TaskStatus.UNKNOWN);
return null;
});
}
/**
* 提交成功后的「一次性短事务」:写入 Gallery 行、累计周用量、把操作推进为 SUBMITTED。
* 三者同一个事务,任何一个失败都一起回滚,避免出现「任务存在但用量未计」这类不一致。
*/
/**
* 提交成功后的落库(带世代校验)。
*
* @param expectedOperationId 本次提交所归属的 operationId;只有它仍是当前有效意图时才能落库,
* 否则(被删除/重建)必须抛错,避免复活已删 gid 或污染新世代
*/
int persistSubmitted(Gallery gallery, int downloaderId, String expectedOperationId) {
return inTransaction(() -> {
TaskOperation operation = taskOperationRepository.selectByGid(gallery.getGid());
// 严格匹配本次世代:任务被删除、被重建或换了 operationId 时都不能落库。
if (operation == null || TaskOperation.STATE_DELETED.equals(operation.getState())
|| TaskOperation.STATE_DELETE_PENDING.equals(operation.getState()))
throw new IllegalStateException("任务已删除或没有有效创建意图,禁止落库复活 gid=" + gallery.getGid());
if (expectedOperationId == null || !expectedOperationId.equals(operation.getOperationId()))
throw new IllegalStateException("创建意图世代不匹配,禁止落库 gid=" + gallery.getGid());
// 幂等:同一意图重复落库(例如上游已提交后重跑)不得把用量累计两次。
boolean alreadyCounted = TaskOperation.STATE_SUBMITTED.equals(operation.getState());
gallery.setDownloader(downloaderId);
gallery.set_download(true);
if (!TaskStatus.SUBMITTED.equals(gallery.getStatus()))
gallery.setStatus(TaskStatus.SUBMITTED);
Gallery existing = galleryMapper.selectGalleryByGid(gallery.getGid());
if (existing == null)
galleryMapper.insertGallery(gallery);
else
galleryMapper.updateGallery(gallery);
if (!alreadyCounted)
configurationMapper.incrementConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, gallery.getFileSize());
taskOperationRepository.updateState(gallery.getGid(), TaskOperation.STATE_SUBMITTED, now());
return 1;
});
}
/**
* 预占式创建:登记 gid 唯一意图(提交任务行占位)→ 只读解析上游 → 标 SUBMITTING → 真正提交。
*
* <p>上游提交发生在事务之外,且只在 SUBMITTING 已持久化之后执行一次。网络异常一律落 UNKNOWN。
*/
public Response createTask(int gid, String placeholderName, String link, String targetResolution, int downloaderId) {
Response response = Response.generateResponse();
Reservation reservation = reserveCreateIntent(gid, placeholderName, downloaderId);
if (reservation.outcome() == ReserveOutcome.DUPLICATE) {
response.failure("任务队列已有此任务,请点击查找任务");
return response;
}
if (reservation.outcome() == ReserveOutcome.CONTENDED) {
// 预占阶段的数据库竞争不产生任何副作用;转业务失败,绝不下发上游、绝不穿透成 500。
response.failure("任务提交失败,请稍后重试");
return response;
}
// 解析上游是只读步骤;真正的 POST 只在 SUBMITTING 持久化之后发生。
// 生命周期约束:RESERVED 阶段被中断是安全的(可释放重来),SUBMITTING 之后一律按 UNKNOWN 处理。
return submitPreparedIntent(gid, reservation.operation(), link, targetResolution, downloaderId);
}
private Response submitPreparedIntent(int gid, TaskOperation operation, String link, String targetResolution,
int downloaderId) {
Response response = Response.generateResponse();
beforePrepare.run();
PreparedGallery prepared;
try {
prepared = upstreamGateway.prepare(link);
} catch (IOException e) {
log.warn("解析任务链接失败 gid={}", gid, e);
releaseReservation(gid, operation.getOperationId());
response.failure("IO错误,可能是网络波动");
return response;
} catch (RuntimeException e) {
log.warn("解析任务链接异常 gid={}", gid, e);
releaseReservation(gid, operation.getOperationId());
response.failure("提交任务失败,未知原因,请检查链接是否正常");
return response;
}
if (prepared == null || prepared.gallery() == null) {
releaseReservation(gid, operation.getOperationId());
response.failure("提交任务失败,未知原因,请检查链接是否正常");
return response;
}
Gallery gallery = prepared.gallery();
gallery.setGid(gid);
// CAS 确认这一代意图仍然有效(未被并发删除/重建),并把状态持久化为 SUBMITTING。
// 只有成功之后才允许真正 POST;失败说明任务已不在当前生命周期里,直接放弃。
if (!markSubmitting(operation.getOperationId(), gid)) {
log.warn("创建意图已失效(可能被并发删除),放弃提交 gid={} operationId={}",
gid, operation.getOperationId());
response.failure("任务已被删除或状态已变化,请刷新任务列表");
return response;
}
afterMarkSubmitting.run();
Gallery submitted;
try {
submitted = upstreamGateway.submit(prepared, targetResolution);
} catch (ResolutionNotMatchException e) {
releaseReservation(gid, operation.getOperationId());
response.failure("提交失败,分辨率不存在");
return response;
} catch (IOException | RuntimeException e) {
// 超时或断连时上游可能已经受理,无法确认:只能记 UNKNOWN,绝不自动重发。
log.warn("上游提交结果未知 gid={} operationId={}", gid, operation.getOperationId(), e);
markUnknown(operation.getOperationId(), gid);
response.failure("上游提交结果未知,请稍后在任务列表中确认,不要重复提交");
return response;
}
if (submitted == null || !TaskStatus.SUBMITTED.equals(submitted.getStatus())) {
markUnknown(operation.getOperationId(), gid);
response.failure("上游提交结果未知,请稍后在任务列表中确认,不要重复提交");
return response;
}
submitted.setGid(gid);
try {
persistSubmitted(submitted, downloaderId, operation.getOperationId());
} catch (RuntimeException e) {
// 上游已受理但本地落库失败:保留 SUBMITTING 意图并标 UNKNOWN,任务行仍在列表里可对账。
log.error("提交成功后落库失败 gid={} operationId={}", gid, operation.getOperationId(), e);
markUnknown(operation.getOperationId(), gid);
response.failure("任务已提交,但本地保存失败,请在任务列表中确认");
return response;
}
// 只有事务提交之后才向节点下发命令;重复下发由节点按 operationId 去重。
byte dispatchResult = dispatchDownload(gid, submitted.getName());
Gallery current = galleryMapper.selectGalleryByGid(gid);
Gallery payload = current == null ? submitted : current;
if (dispatchResult == 0)
response.success(objectMapper.valueToTree(payload));
else
response.failure("任务已保存,但节点未确认接收;请刷新任务列表后重试");
return response;
}
/**
* 提交成功后的落库 + 下发。落库(含用量)在一个短事务里完成,事务提交后才向节点发命令。
*
* <p>先解析当前世代再落库:世代不存在(未提交过 / 已删除)时不落库也不下发。
*
* @return 已持久化的 Gallery;数据库失败或世代无效时抛出异常,调用方回业务失败且不下发节点。
*/
public Gallery persistAndDispatch(Gallery gallery, int downloaderId) {
DispatchDecision decision = resolveDispatch(gallery.getGid());
if (!decision.dispatchable())
throw new IllegalStateException("任务没有可下发的有效世代 gid=" + gallery.getGid());
persistSubmitted(gallery, downloaderId, decision.operationId());
dispatchDownload(gallery.getGid(), gallery.getName());
return gallery;
}
/** 向节点下发 DOWNLOAD 命令(复用当前有效 operationId),返回节点结果码。 */
public byte dispatchDownload(int gid, String name) {
DispatchDecision decision = resolveDispatch(gid);
if (!decision.dispatchable())
return -1;
return remoteService.sendTaskCommand(TaskCommandMessage.ACTION_DOWNLOAD, decision.operationId(), gid, name);
}
/**
* 判定一个 gid 能否下发给节点,并为迁移前的旧任务补齐稳定 operationId。
*
* <p>规则:
* <ul>
* <li>v2 任务只有 {@code SUBMITTED} 可下发;{@code UNKNOWN} 必须人工对账,
* 否则未计量的未知任务会在节点侧变成「下载完成却 0 额度」;</li>
* <li>预占中的 {@code RESERVED}/{@code SUBMITTING}、待删除与删除墓碑都不下发;</li>
* <li>没有任何操作记录、但任务行确实存在且状态已是「已提交」及之后的历史任务,
* 补建一个稳定 operationId 后再下发(这是唯一的 legacy 兼容路径)。</li>
* </ul>
*/
public DispatchDecision resolveDispatch(int gid) {
TaskOperation operation = taskOperationRepository.selectByGid(gid);
if (operation != null && TaskOperation.STATE_SUBMITTED.equals(operation.getState())
&& operation.getOperationId() != null)
return new DispatchDecision(DispatchDecision.Kind.READY, operation.getOperationId());
// 没有操作记录,或只有「旧世代」记录(缺 operationId):必须先确认任务行确实已提交过,
// 再补一个稳定 operationId;UNKNOWN/预占/待删除一律拒绝。
if (operation != null && !TaskOperation.STATE_SUBMITTED.equals(operation.getState()))
return REFUSE;
if (!isLegacySubmitted(gid))
return REFUSE;
String freshId = ensureStableOperationId(gid);
return freshId == null ? REFUSE : new DispatchDecision(DispatchDecision.Kind.LEGACY, freshId);
}
/** 迁移前的旧任务行:没有操作记录,但状态表明上游确实已受理过下载,可安全重投。 */
private boolean isLegacySubmitted(int gid) {
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
if (gallery == null || gallery.getStatus() == null)
return false;
return switch (gallery.getStatus()) {
case TaskStatus.SUBMITTED, TaskStatus.DOWNLOADING, TaskStatus.WAITING_COMPRESS,
TaskStatus.COMPRESSING -> true;
// 提交未知 / 提交中 / 待删除 / 删除失败 一律不算已提交,不能下发。
default -> false;
};
}
/**
* 为一个已经真正提交过的旧任务持久化稳定 operationId。
*
* <p>必须是短事务写入、先落库后返回,之后重投/重连才复用同一个 ID;
* 用 {@code insert or ignore} + 回读,避免与并发创建互相覆盖。
*
* @return 稳定 operationId;无法安全建立时返回 null(调用方不下发)
*/
String ensureStableOperationId(int gid) {
// 与创建预占共用同一把短锁(锁内只有数据库写),并发为同一 gid 补 ID 时只能有一个赢家。
synchronized (reservationLock) {
String candidate = TaskProtocol.newDownloadOperationId();
return inTransaction(() -> {
if (taskOperationRepository.selectByGid(gid) == null)
taskOperationRepository.insertIfAbsent(gid, candidate, TaskOperation.STATE_SUBMITTED, now());
else
taskOperationRepository.assignOperationIdIfMissing(gid, candidate, now());
TaskOperation current = taskOperationRepository.selectByGid(gid);
if (current == null || current.getOperationId() == null)
return null;
// 只有确实处于 SUBMITTED 的世代才可下发;并发写入的其他状态一律拒绝。
return TaskOperation.STATE_SUBMITTED.equals(current.getState()) ? current.getOperationId() : null;
});
}
}
/**
* 取消预占(解析失败、分辨率不存在等:还没有产生任何上游副作用)。
*
* <p>只删除与本次创建 intent 匹配的 RESERVED 记录:若期间该 gid 已被删除并重建
* (新的 operationId / 墓碑),这里绝不能把新世代记录抹掉。占位行同理只删「提交中」。
*/
private void releaseReservation(int gid, String operationId) {
inTransaction(() -> {
// 只删与本代 operationId 匹配且仍停在 RESERVED/SUBMITTING 的记录。
int removed = taskOperationRepository.deleteReservationIfSameGeneration(gid, operationId);
// 只有确实删掉了本代预占记录,才能删对应的占位行:
// 若期间已出现新的 RESERVED 代,删除返回 0,此时绝不能误删新占位。
if (removed == 1) {
Gallery existing = galleryMapper.selectGalleryByGid(gid);
if (existing != null && TaskStatus.PENDING_SUBMIT.equals(existing.getStatus()))
galleryMapper.deleteGalleryByGid(gid);
}
return null;
});
}
/**
* 删除的持久化部分。
*
* <p>正在创建意图(RESERVED/SUBMITTING)的任务不能删除:此刻 DELETE 与创建 POST 会同时发生,
* 随后创建落库还会把已删/重建的 gid 写回。这里直接拒绝,由用户在提交结果明确后再操作。
*
* @return 显式结果:被拒时调用方必须回业务失败,不能退避到裸删除
*/
public DeleteIntent beginDelete(int gid) {
return inTransaction(() -> {
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
if (gallery == null)
return DeleteIntent.refused("删除失败,该图片不存在");
TaskOperation operation = taskOperationRepository.selectByGid(gid);
if (operation != null && (TaskOperation.STATE_RESERVED.equals(operation.getState())
|| TaskOperation.STATE_SUBMITTING.equals(operation.getState())))
return DeleteIntent.refused("任务正在提交中,请等待提交结果明确后再删除");
if (operation == null) {
// 历史任务行没有操作记录:直接落一条 DELETE_PENDING 意图,避免中间态缺字段。
TaskOperation fresh = new TaskOperation();
fresh.setGid(gid);
fresh.setOperationId(TaskProtocol.newDownloadOperationId());
fresh.setDeleteOperationId(TaskProtocol.newDeleteOperationId());
fresh.setState(TaskOperation.STATE_DELETE_PENDING);
fresh.setUpdateTime(now());
taskOperationRepository.insert(fresh);
galleryMapper.updateGalleryStatus(gid, TaskStatus.DELETE_PENDING);
return DeleteIntent.accepted(fresh.getDeleteOperationId());
}
if (operation.getDeleteOperationId() == null) {
operation.setDeleteOperationId(TaskProtocol.newDeleteOperationId());
operation.setUpdateTime(now());
}
taskOperationRepository.updateDeleteIntent(gid, operation.getDeleteOperationId(),
TaskOperation.STATE_DELETE_PENDING, now());
galleryMapper.updateGalleryStatus(gid, TaskStatus.DELETE_PENDING);
return DeleteIntent.accepted(operation.getDeleteOperationId());
});
}
/**
* 删除的完成部分:节点已确认(或文件已不存在)后,在一个短事务里删任务行、收藏并留墓碑。
*
* <p>事务提交前若失败,任务行与 DELETE_PENDING 意图都保留,可以继续重试。
* 删除意图 ID 必须匹配(防止并发删除/重建时抹掉别的一代),随后只把 DELETE_PENDING
* 世代推进为 DELETED;墓碑(DELETED 且无删除意图)不允许在这里被重新激活。
*
* @return true 表示本次删除意图已完成并留下墓碑
*/
public boolean completeDelete(int gid, String deleteOperationId) {
return inTransaction(() -> {
TaskOperation operation = taskOperationRepository.selectByGid(gid);
if (operation == null || operation.getDeleteOperationId() == null
|| !operation.getDeleteOperationId().equals(deleteOperationId))
return false;
// 先清收藏再删任务行:收藏一侧在部分历史库上带有指向 gallery 的外键。
collectMapper.deleteByGid(gid);
galleryMapper.deleteGalleryByGid(gid);
int updated = taskOperationRepository.completeDeleteIfSameGeneration(gid, deleteOperationId,
TaskOperation.STATE_DELETED, now());
return updated == 1;
});
}
/** 删除失败:保留任务行并记一条可重试的失败状态。 */
public void failDelete(int gid) {
inTransaction(() -> {
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
if (gallery == null)
return null;
gallery.setStatus(TaskStatus.DELETE_FAILED);
galleryMapper.updateGallery(gallery);
return null;
});
}
/**
* 启动恢复:把上次中断在 SUBMITTING 的意图判为 UNKNOWN,并安全释放还停在 RESERVED 的预占。
*
* <p>这段恢复必须发生在任何自动重投之前:
* <ul>
* <li>{@code SUBMITTING}:上游可能已经受理但本地无法确认,只能落 UNKNOWN(操作与任务行同时
* 更新为「提交未知」),交人工对账,绝不自动再 POST;</li>
* <li>{@code RESERVED}:还没有产生任何上游副作用,重启后可以直接释放预占
* (删掉匹配的 RESERVED 记录与「提交中」占位行),让用户重新提交。</li>
* </ul>
*
* @return 本次恢复(判未知 + 释放预占)处理的任务数
*/
public int recoverInterruptedSubmissions() {
return inTransaction(() -> {
int recovered = 0;
for (TaskOperation operation : taskOperationRepository.selectByState(TaskOperation.STATE_SUBMITTING)) {
int advanced = taskOperationRepository.compareAndSetCreateState(operation.getGid(), operation.getOperationId(),
TaskOperation.STATE_SUBMITTING, TaskOperation.STATE_UNKNOWN, now());
if (advanced != 1)
continue;
Gallery gallery = galleryMapper.selectGalleryByGid(operation.getGid());
if (gallery != null && !TaskStatus.UNKNOWN.equals(gallery.getStatus()))
galleryMapper.updateGalleryStatus(operation.getGid(), TaskStatus.UNKNOWN);
recovered++;
}
for (TaskOperation operation : taskOperationRepository.selectByState(TaskOperation.STATE_RESERVED)) {
// 还没 POST:释放匹配的预占记录与占位行,允许重启后重新提交。
// 只有确实删掉了这条本代 RESERVED 记录,才删其占位行;否则不动(可能已换代)。
int removed = taskOperationRepository.deleteReservationIfSameGeneration(operation.getGid(),
operation.getOperationId());
if (removed == 1) {
Gallery gallery = galleryMapper.selectGalleryByGid(operation.getGid());
if (gallery != null && TaskStatus.PENDING_SUBMIT.equals(gallery.getStatus()))
galleryMapper.deleteGalleryByGid(operation.getGid());
}
recovered++;
}
return recovered;
});
}
/** 已完成状态上报的处理:丢弃旧 revision、旧 operation、待删除任务与重复上报。 */
@Override
public CommittedReport apply(TaskReportMessage report) {
if (report == null || report.getReportId() == null || report.getGalleryTasks() == null)
return null;
Applied applied;
try {
applied = inTransaction(() -> applyInTransaction(report));
} catch (RuntimeException e) {
// 数据库失败:不返回可回执结果,节点会保留快照并稍后重投;失败写入绝不清任务。
log.warn("应用节点上报失败,未回执 reportId={}", report.getReportId(), e);
return null;
}
if (applied == null)
return null;
return new CommittedReport(report.getReportId(), applied.newlyCompleted(), applied.acceptedTasks());
}
/** 事务内应用上报的结果:新完成的任务与真正被接受的进度更新。 */
private record Applied(List<Gallery> newlyCompleted, List<GalleryTask> acceptedTasks) {
}
/**
* 在一个事务里应用上报。
*
* @return 非 null 表示已经提交(可以回 ACK,重复上报也会返回结果以停止重放)。
*/
private Applied applyInTransaction(TaskReportMessage report) {
long revision = report.getRevision();
List<Gallery> newlyCompleted = new ArrayList<>();
List<GalleryTask> acceptedTasks = new ArrayList<>();
for (GalleryTask task : report.getGalleryTasks()) {
if (task == null)
continue;
TaskOperation operation = taskOperationRepository.selectByGid(task.getGid());
Gallery gallery = galleryMapper.selectGalleryByGid(task.getGid());
if (gallery == null) {
// 未知 gid(任务已删除):忽略但 ACK,让节点停止重放。
log.debug("忽略节点上报的未知任务 gid={}", task.getGid());
continue;
}
String expectedOperationId = report.getOperationIds() == null
? null : report.getOperationIds().get(task.getGid());
if (operation == null) {
// 旧任务行(迁移前创建)没有操作记录:按显式生成策略补建并采纳上报里的 operationId。
operation = adoptGeneration(task.getGid(), expectedOperationId);
} else if (TaskOperation.STATE_DELETED.equals(operation.getState())) {
// 墓碑必须永久忽略旧上报:删除已完成的任务只能由显式 CREATE 重新建立新世代。
// (gallery 已被删除时上面的 null 分支已经提前忽略,这里防御性兜底。)
log.debug("忽略已删除 gid 的迟到上报 gid={}", task.getGid());
continue;
} else if (TaskOperation.STATE_DELETE_PENDING.equals(operation.getState())) {
log.info("忽略待删除任务的迟到上报 gid={}", task.getGid());
continue;
} else if (operation.getLastRevision() > 0 && revision <= operation.getLastRevision()) {
log.debug("丢弃过期上报 gid={} revision={} lastRevision={}",
task.getGid(), revision, operation.getLastRevision());
continue;
} else if (operation.getOperationId() != null) {
// v2 任务必须有归属:缺 operationId 或与当前世代不符都不能改当前任务,
// 否则「删除后重建」的新世代会被旧上报污染。
if (expectedOperationId == null || !operation.getOperationId().equals(expectedOperationId)) {
log.info("丢弃无法归属/旧世代的迟到上报 gid={} 上报={} 当前={}",
task.getGid(), expectedOperationId, operation.getOperationId());
continue;
}
}
String status = TaskStatus.fromReport(task.getStatus());
if (status == null) {
log.warn("未知上报状态码 gid={} status={}", task.getGid(), task.getStatus());
continue;
}
gallery.setProceeding(task.getProceeding());
if (task.getName() != null)
gallery.setName(task.getName());
boolean justCompleted = TaskStatus.COMPLETE.equals(status)
&& !TaskStatus.COMPLETE.equals(gallery.getStatus());
gallery.setStatus(status);
galleryMapper.updateGallery(gallery);
taskOperationRepository.updateReportReceipt(task.getGid(), revision, report.getReportId(), now());
acceptedTasks.add(task);
if (justCompleted)
newlyCompleted.add(gallery);
}
return new Applied(newlyCompleted, acceptedTasks);
}
/**
* 为迁移前创建、尚无操作记录的历史任务建立一次「世代」:优先采纳上报里的 operationId;
* 旧节点不带 operationId 时保留 null,让后续同世代的进度上报仍能被接受。
*
* <p>不在这里凭空生成随机 ID:那会让第一份进度上报之后的所有旧上报都因「无法归属」被丢弃。
* 真正的稳定 operationId 会在首次自动下发时用 {@code ensureStableOperationId} 落库,
* 之后该世代即可被严格匹配。不会覆盖任何已有任务状态。
*/
private TaskOperation adoptGeneration(int gid, String reportedOperationId) {
TaskOperation operation = new TaskOperation();
operation.setGid(gid);
operation.setOperationId(reportedOperationId);
operation.setState(TaskOperation.STATE_SUBMITTED);
operation.setUpdateTime(now());
taskOperationRepository.insert(operation);
return operation;
}
private static long now() {
return System.currentTimeMillis() / 1000;
}
/** 供装配层解析当前有效 operationId(避免装配代码访问包私有字段)。 */
public String currentOperationId(int gid) {
TaskOperation operation = taskOperationRepository.selectByGid(gid);
return operation == null ? null : operation.getOperationId();
}
}
@@ -2,9 +2,7 @@ package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper; import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
import com.lion.lionwebsite.Dao.normal.GalleryMapper; import com.lion.lionwebsite.Dao.normal.GalleryMapper;
import com.lion.lionwebsite.Dao.normal.ShareFileMapper;
import com.lion.lionwebsite.Domain.CustomConfiguration; import com.lion.lionwebsite.Domain.CustomConfiguration;
import com.lion.lionwebsite.Domain.ShareFile;
import com.lion.lionwebsite.Util.CustomUtil; import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.GalleryUtil; import com.lion.lionwebsite.Util.GalleryUtil;
import lombok.Data; import lombok.Data;
@@ -23,6 +21,7 @@ import java.net.URLDecoder;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.nio.file.*; import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes; import java.nio.file.attribute.BasicFileAttributes;
import java.nio.file.attribute.FileTime;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatter;
import java.util.*; import java.util.*;
@@ -45,15 +44,13 @@ public class LocalService{
final CustomConfigurationMapper configurationMapper; final CustomConfigurationMapper configurationMapper;
final ShareFileMapper shareFileMapper;
final GalleryMapper galleryMapper; final GalleryMapper galleryMapper;
final PushService pushService; final PushService pushService;
final RemoteService remoteService; final RemoteService remoteService;
final SubscriptionRefreshService subscriptionRefreshService; final SubscriptionRefreshScheduler subscriptionRefreshScheduler;
/** /**
* 检查连接是否有效,如果无效自动重连 * 检查连接是否有效,如果无效自动重连
@@ -104,115 +101,23 @@ public class LocalService{
} }
/** /**
* 定时更新订阅 * 手动「更新订阅」:立即刷新全部启用账号,供管理页按钮调用。
* @throws IOException 下载以及保存异常 *
* <p>定时刷新已移交 {@link SubscriptionRefreshScheduler}。原先这里还有一个
* {@code @Scheduled(fixedRate = 86400000)} 的定时入口,它会一次性刷新全部账号
* (十几个账号在十几秒内打满),且因为没有 initialDelay,每次重启都会立刻重刷一遍,
* 使上游看到的请求密度取决于部署频率。该入口已移除。
*
* <p>不再保留 {@code isManual} 参数:定时路径已不存在,留着它只会让人以为
* 「传 false 就是不刷新」,而实际语义是「什么都不做却报成功」。
*/ */
@Scheduled(fixedRate = 86400000) public boolean updateSub() {
public void updateSubScheduler() throws IOException { boolean success = subscriptionRefreshScheduler.refreshAllNow();
updateSub(false);
}
/**
* 更新订阅链接的实际方法
*/
public boolean updateSub(boolean isManual) throws IOException {
// 手动和定时入口均刷新全部启用子账号;全部成功后更新原有“上次更新时间”。
boolean success = subscriptionRefreshService.refreshAll();
remoteService.requestSubscriptionSync(); remoteService.requestSubscriptionSync();
if (success) if (success)
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME, configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME,
CustomUtil.dateTimeFormatter().format(LocalDateTime.now())); CustomUtil.dateTimeFormatter().format(LocalDateTime.now()));
return success; return success;
/*
DateTimeFormatter dateTimeFormatter = CustomUtil.dateTimeFormatter();
CustomConfiguration customConfiguration = configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME);
//如果不是手动,则判断更新间隔是否满足,不满足则取消更新
if (!isManual) {
LocalDateTime lastUpdate = LocalDateTime.parse(customConfiguration.getValue(), CustomUtil.dateTimeFormatter());
LocalDateTime now = LocalDateTime.now();
now = now.plusHours(-3);
if(now.isBefore(lastUpdate))
return false;
}
File DouNaiClashFile = new File("sub/DouNaiClash.txt");
File DouNaiV2rayFile = new File("sub/DouNaiV2ray.txt");
File directory = new File("sub");
if(!directory.isDirectory())
Files.createDirectory(Paths.get("sub"));
List<String> DouNaiClash_profile;
//下载豆奶v2ray订阅
try(FileWriter writer = new FileWriter(DouNaiV2rayFile)) {
String DouNaiV2rayRaw = Get(DouNaiV2ray).getFirst();
String[] v2rayPlain = new String(Base64.getDecoder().decode(DouNaiV2rayRaw)).split("\n");
StringBuilder stringBuilder = new StringBuilder();
Pattern pattern = Pattern.compile("-?\\d+(\\.\\d+)?");
//过滤高倍率节点
for(String node: v2rayPlain){
String name = URLDecoder.decode(node.split("#")[1], StandardCharsets.UTF_8);
if(name.contains("流量")){
Matcher matcher = pattern.matcher(name.substring(name.indexOf("(") + 1, name.indexOf(")")));
if (matcher.find()) {
// 将匹配到的数字添加到列表中
float ratio = Float.parseFloat(matcher.group());
if(ratio <= 2) {
stringBuilder.append(node).append("\n");
continue;
}
}
stringBuilder.append(node).append("\n");
}
else{
stringBuilder.append(node).append("\n");
}
}
writer.write(new String(Base64.getEncoder().encode(stringBuilder.toString().getBytes(StandardCharsets.UTF_8))));
log.info("load DouNai v2ray complete");
}catch (IOException e){
log.error("load DouNai v2ray failure", e);
}
//下载豆奶clash订阅
try(FileWriter writer = new FileWriter(DouNaiClashFile)) {
DouNaiClash_profile = Get(DouNaiClash);
//过滤高倍率节点
ArrayList<String> clashProcessed = new ArrayList<>();
boolean isProxies = false;
boolean skip = false;
for(String line: DouNaiClash_profile){
if(line.equals("proxies:"))
isProxies = true;
else if(line.equals("proxy-groups:") && isProxies)
isProxies = false;
if(isProxies) {
if (line.contains("name"))
skip = line.contains("流量");
if (!skip)
clashProcessed.add(line);
}
else
if (!line.contains("流量"))
clashProcessed.add(line);
}
for(String line: clashProcessed)
writer.write(line + "\n");
log.info("load DouNai clash complete");
}catch (IOException e){
log.error("load DouNai clash failure", e);
}
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME, dateTimeFormatter.format(LocalDateTime.now()));
return true;
*/
} }
/** /**
@@ -242,23 +147,6 @@ public class LocalService{
return temp; return temp;
} }
/**
* 每天四点,清理过期分享码
*/
@Scheduled(cron = "0 0 4 * * *")
public void checkShareCode(){
ShareFile[] shareFiles = shareFileMapper.selectAllShareFile();
Calendar now;
Calendar expireTime;
for(ShareFile shareFile: shareFiles){
now = Calendar.getInstance();
expireTime = Calendar.getInstance();
expireTime.setTime(shareFile.getExpireTime());
if(now.after(expireTime))
shareFileMapper.deleteShareFile(shareFile.getShareCode());
}
}
/** /**
* 每周,查看缩略图数量,按照访问时间排序,删除超出1w的部分 * 每周,查看缩略图数量,按照访问时间排序,删除超出1w的部分
*/ */
@@ -266,13 +154,15 @@ public class LocalService{
public void clearThumbnailCache(){ public void clearThumbnailCache(){
String cachePath = "/storage/hentaiCache/"; String cachePath = "/storage/hentaiCache/";
File directory = new File(cachePath); File directory = new File(cachePath);
List<Path> files = new ArrayList<>(); // 排序需要文件的最后访问时间。原先在比较器里每次重新读属性(n log n 次系统调用),
// 且属性读取失败会让整个排序抛 RuntimeException;这里在遍历时一次性带出属性。
List<Map.Entry<Path, FileTime>> files = new ArrayList<>();
try { try {
Files.walkFileTree(directory.toPath(), new SimpleFileVisitor<>() { Files.walkFileTree(directory.toPath(), new SimpleFileVisitor<>() {
@Override @Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) { public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
if (attrs.isRegularFile()) { if (attrs.isRegularFile()) {
files.add(file); files.add(Map.entry(file, attrs.lastAccessTime()));
} }
return CONTINUE; return CONTINUE;
} }
@@ -281,22 +171,15 @@ public class LocalService{
throw new RuntimeException(e); throw new RuntimeException(e);
} }
files.sort((f1, f2) -> { files.sort(Map.Entry.comparingByValue());
try {
return Files.readAttributes(f1, BasicFileAttributes.class).lastAccessTime()
.compareTo(Files.readAttributes(f2, BasicFileAttributes.class).lastAccessTime());
} catch (IOException e) {
throw new RuntimeException(e);
}
});
if (files.size() > 10000) { if (files.size() > 10000) {
List<Path> toDelete = files.subList(0, files.size() - 10000); List<Map.Entry<Path, FileTime>> toDelete = files.subList(0, files.size() - 10000);
for (Path file : toDelete) { for (Map.Entry<Path, FileTime> entry : toDelete) {
try { try {
Files.delete(file); Files.delete(entry.getKey());
}catch (IOException e){ }catch (IOException e){
log.warn("删除缩略图缓存文件失败: {}", file, e); log.warn("删除缩略图缓存文件失败: {}", entry.getKey(), e);
} }
} }
log.info("Deleted {} files", toDelete.size()); log.info("Deleted {} files", toDelete.size());
@@ -1,43 +1,18 @@
package com.lion.lionwebsite.Service; package com.lion.lionwebsite.Service;
import cn.hutool.core.io.FileUtil;
import cn.hutool.core.util.RandomUtil;
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper; import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
import com.lion.lionwebsite.Dao.normal.ShareFileMapper;
import com.lion.lionwebsite.Dao.normal.UserMapper;
import com.lion.lionwebsite.Domain.CustomConfiguration; import com.lion.lionwebsite.Domain.CustomConfiguration;
import com.lion.lionwebsite.Domain.ShareFile;
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.FileDownload;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import tools.jackson.databind.ObjectMapper;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import lombok.Data; import lombok.Data;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import java.io.BufferedOutputStream; import java.util.HashMap;
import java.io.File; import java.util.Map;
import java.io.IOException;
import java.io.InputStream; import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
import java.io.OutputStream;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@Service @Service
@@ -46,219 +21,8 @@ import java.util.concurrent.Executors;
public class PersonalService{ public class PersonalService{
final CustomConfigurationMapper configurationMapper; final CustomConfigurationMapper configurationMapper;
final UserMapper userMapper;
final ShareFileMapper shareFileMapper;
final TaskHandlerInterceptor taskHandlerInterceptor;
String StoragePath = "/storage/";
DateTimeFormatter dateTimeFormatter = CustomUtil.dateTimeFormatter();
ExecutorService compressThreadPool = Executors.newFixedThreadPool(1);
final PushService pushService; final PushService pushService;
/**
* 获取文件列表,同时带上分享码以及过期时间
* @param path 路径
* @return 文件列表
*/
public String getFiles(String path) {
Response response = Response.generateResponse();
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
File root_file = new File(StoragePath + path);
Calendar now = Calendar.getInstance();
Calendar expireTime = Calendar.getInstance();
//如果目标路径是文件夹
if(root_file.isDirectory()) {
ArrayList<Map<String, String>> fileMaps = new ArrayList<>();
File[] originalFiles = root_file.listFiles();
//如果文件夹里的文件不为空
if (originalFiles != null) {
originalFiles = Arrays.stream(originalFiles).sorted(Comparator.comparing(File::getName)).toArray(File[]::new);
ArrayList<ShareFile> shareFiles = shareFileMapper.selectShareFilesByFilePath(root_file.getAbsolutePath());
//遍历文件,放入文件信息以及查询对应的分享码
for (File file : originalFiles) {
Map<String, String> fileMap = new LinkedHashMap<>();
fileMap.put("name", file.getName());
fileMap.put("path", file.getAbsolutePath());
if (file.isDirectory()) {
fileMap.put("type", "FOLDER");
} else if (file.isFile()) {
fileMap.put("type", "FILE"); //处理文件大小单位
String fileSize = CustomUtil.fileSizeToString(file.length());
fileMap.put("size", fileSize);
Iterator<ShareFile> iterator = shareFiles.iterator();
while(iterator.hasNext()){
ShareFile shareFile = iterator.next();
if(shareFile.getFilePath().equals(file.getAbsolutePath())){
expireTime.setTime(shareFile.getExpireTime());
if(now.after(expireTime)){
shareFileMapper.deleteShareFile(shareFile.getShareCode());
}
else {
fileMap.put("shareCode", shareFile.getShareCode());
fileMap.put("expireTime", dateTimeFormatter.format(shareFile.getExpireTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
}
iterator.remove();
break;
}
}
}
fileMaps.add(fileMap);
}
response.success(new ObjectMapper().valueToTree(fileMaps).toString());
}
else
response.failure("文件夹为空");
}
return response.toJSONString();
}
/**
* 下载文件
* @param request 请求对象
* @param response 响应对象
* @param path 目标路径
*/
public void download(HttpServletRequest request, HttpServletResponse response, String path){
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
File file = new File(path);
if(file.exists())
FileDownload.export(request, response, path);
else
try{
response.getWriter().print("404 NOT FOUND");
}catch (IOException e){
log.warn("输出404失败", e);
}
}
/**
* 上传文件
* @param path 目标路径
* @param fileName 文件名称
* @param file 文件对象
* @return 上传结果
*/
public String uploadFile(String path, String fileName, MultipartFile file) {
Response response = Response.generateResponse();
log.info("上传文件:{}, 目标路径:{}", fileName, path);
if(path == null || fileName == null || file == null){
response.failure("参数不完整");
return response.toJSONString();
}
File directory = new File(StoragePath + path);
if(directory.isDirectory()){
File targetFile = new File(StoragePath + path, fileName);
if(targetFile.exists())
response.failure("目标文件已存在");
else
try {
file.transferTo(Path.of(StoragePath + path, fileName));
response.success("上传成功");
} catch (IOException e) {
response.failure("上传失败");
log.error("上传失败: {}", fileName, e);
}
}
else
response.failure("该路径不存在或者不是文件夹");
return response.toJSONString();
}
/**
* 创建分享码
* @param path 目标路径
* @param expireHour 过期时间
* @return 如果成功则是分享码以及过期时间,失败则是失败原因
*/
public String shareFile(String path, Integer expireHour) {
Response response = Response.generateResponse();
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
Map<String, String> jsonObject = new HashMap<>();
File file = new File(path);
if(file.isFile()){ //如果是文件,则生成分享码,调用此接口时不需要考虑分享已分享文件以及分享已过期文件
String ShareCode;
ShareCode = RandomUtil.randomString(8);
Calendar expireTime = Calendar.getInstance();
expireTime.add(Calendar.HOUR, expireHour);
shareFileMapper.insertShareFile(ShareCode, path, expireTime.getTime());
jsonObject.put("shareCode", ShareCode);
jsonObject.put("expireTime", dateTimeFormatter.format(expireTime.getTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
response.success(new ObjectMapper().valueToTree(jsonObject).toString());
}
else
response.failure("此路径为文件夹或不存在");
return response.toJSONString();
}
/**
* 延长分享时间
* @param path 目标文件路径
* @param extendHour 延长小时数
* @return 延长结果
*/
public String extendShareTime(String path, Integer extendHour) {
Response response = Response.generateResponse();
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
Map<String, String> data = new LinkedHashMap<>();
ShareFile shareFile = shareFileMapper.selectShareFileByFilePath(path);
if(shareFile != null){
Calendar calendar = Calendar.getInstance();
calendar.setTime(shareFile.getExpireTime());
calendar.add(Calendar.HOUR, extendHour);
shareFile.setExpireTime(calendar.getTime());
shareFileMapper.updateShareFile(shareFile);
data.put("expireTime", dateTimeFormatter.format(shareFile.getExpireTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
data.put("path", shareFile.getFilePath());
response.success(new ObjectMapper().valueToTree(data).toString());
}
else{
response.failure("该文件未被分享");
}
return response.toJSONString();
}
/**
* 取消分享
* @param path 目标路径
* @return 取消结果
*/
public String cancelShare(String path) {
Response response = Response.generateResponse();
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
ShareFile shareFile = shareFileMapper.selectShareFileByFilePath(path);
if(shareFile != null){
shareFileMapper.deleteShareFile(shareFile.getShareCode());
response.success("取消分享成功");
}
else{
response.failure("该文件未被分享");
}
return response.toJSONString();
}
/** /**
* 获取订阅文件上次更新时间 * 获取订阅文件上次更新时间
* @return 订阅文件上次更新时间 * @return 订阅文件上次更新时间
@@ -282,87 +46,7 @@ public class PersonalService{
jsonObject.put("ip", ip); jsonObject.put("ip", ip);
jsonObject.put("lastUpdateTime", updateTime); jsonObject.put("lastUpdateTime", updateTime);
response.success(new ObjectMapper().valueToTree(jsonObject).toString().replace("\"", " ").replace("\\", " ")); response.success(objectMapper.valueToTree(jsonObject).toString().replace("\"", " ").replace("\\", " "));
return response.toJSONString();
}
/**
* 打包文件
* @param path 目标路径
* @return 响应提交成功,因为该方法为异步执行,未完成时后辍为undone
*/
public String compress(String path) {
Response response = Response.generateResponse();
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
File file = new File(path);
String finalPath = path;
if(!file.isDirectory() || !file.exists()){
response.failure("选中的路径不是文件夹");
return response.toJSONString();
}
compressThreadPool.submit(() -> {
Path temporary = Paths.get(finalPath + ".tar***undone");
try {
writeTar(Paths.get(finalPath), temporary);
// writeTar closes the archive (including its trailer) before publication.
try {
Files.move(temporary, Paths.get(finalPath + ".tar"), StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
} catch (AtomicMoveNotSupportedException e) {
Files.move(temporary, Paths.get(finalPath + ".tar"), StandardCopyOption.REPLACE_EXISTING);
}
log.info("打包成功: {}.tar", finalPath);
} catch (IOException e) {
log.error("打包失败", e);
} finally {
try { Files.deleteIfExists(temporary); }
catch (IOException e) { log.warn("清理打包临时文件失败", e); }
}
});
response.success("加入队列成功");
return response.toJSONString();
}
static void writeTar(Path directory, Path output) throws IOException {
try (OutputStream stream = new BufferedOutputStream(Files.newOutputStream(output));
TarArchiveOutputStream archive = new TarArchiveOutputStream(stream)) {
archive.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
Files.walkFileTree(directory, new SimpleFileVisitor<>() {
@Override public FileVisitResult preVisitDirectory(Path path, BasicFileAttributes attrs) throws IOException {
if (!path.equals(directory)) {
archive.putArchiveEntry(new TarArchiveEntry(path.toFile(), directory.relativize(path).toString()));
archive.closeArchiveEntry();
}
return FileVisitResult.CONTINUE;
}
@Override public FileVisitResult visitFile(Path path, BasicFileAttributes attrs) throws IOException {
archive.putArchiveEntry(new TarArchiveEntry(path.toFile(), directory.relativize(path).toString()));
try (InputStream input = Files.newInputStream(path)) {
IOUtils.copy(input, archive);
}
archive.closeArchiveEntry();
return FileVisitResult.CONTINUE;
}
});
}
}
/**
* 删除文件
* @param path 目标路径
* @return 删除结果
*/
public String deleteFile(String path) {
Response response = Response.generateResponse();
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
File file = new File(path);
if(FileUtil.del(file))
response.success("删除成功");
else
response.failure("删除失败");
return response.toJSONString(); return response.toJSONString();
} }
@@ -2,35 +2,23 @@ package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper; import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
import com.lion.lionwebsite.Dao.normal.ShareFileMapper;
import com.lion.lionwebsite.Dao.normal.UserMapper; import com.lion.lionwebsite.Dao.normal.UserMapper;
import com.lion.lionwebsite.Domain.CustomConfiguration; import com.lion.lionwebsite.Domain.CustomConfiguration;
import com.lion.lionwebsite.Domain.ShareFile;
import com.lion.lionwebsite.Domain.User; import com.lion.lionwebsite.Domain.User;
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor; import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
import com.lion.lionwebsite.Util.CustomUtil; import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.FileDownload;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.Calendar;
@Service @Service
@RequiredArgsConstructor @RequiredArgsConstructor
public class PublicService { public class PublicService {
final CustomConfigurationMapper configurationMapper; final CustomConfigurationMapper configurationMapper;
final ShareFileMapper shareFileMapper;
final UserMapper userMapper; final UserMapper userMapper;
final TaskHandlerInterceptor taskHandlerInterceptor; final TaskHandlerInterceptor taskHandlerInterceptor;
@@ -44,41 +32,6 @@ public class PublicService {
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME, CustomUtil.now()); configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME, CustomUtil.now());
} }
/**
* 通过分享码获取文件
* @param httpRequest 请求对象
* @param httpResponse 响应对象
* @param ShareCode 分享码
* @throws IOException 响应时的异常
*/
public boolean GetFile(HttpServletRequest httpRequest, HttpServletResponse httpResponse, String ShareCode) throws IOException {
Response response = Response.generateResponse();
//参数为空的情况
if(ShareCode == null) {
response.failure("ShareCode invalid");
} else {
ShareFile shareFile = shareFileMapper.selectShareFileByShareCode(ShareCode);
Calendar ExpireTime = Calendar.getInstance();
Calendar now = Calendar.getInstance();
if (shareFile != null) {
ExpireTime.setTime(shareFile.getExpireTime());
if (ExpireTime.after(now) && new File(shareFile.getFilePath()).isFile()) {
FileDownload.export(httpRequest, httpResponse, shareFile.getFilePath());
return true;
} else {
shareFileMapper.deleteShareFile(shareFile.getShareCode());
response.failure("ShareCode is expired or File is not exist");
}
} else
response.failure("ShareCode is not exist or expired");
}
httpResponse.getOutputStream().write(response.toJSONString().getBytes(StandardCharsets.UTF_8));
return false;
}
/** /**
* 修改授权码,如果能够执行此方法,则授权码一定存在 * 修改授权码,如果能够执行此方法,则授权码一定存在
* @param AuthCode 原来的授权码 * @param AuthCode 原来的授权码
@@ -6,7 +6,6 @@ import com.lion.lionwebsite.Util.GalleryUtil;
import com.lion.lionwebsite.Util.ImageFileCache; import com.lion.lionwebsite.Util.ImageFileCache;
import java.nio.file.Path; import java.nio.file.Path;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import tools.jackson.databind.ObjectMapper;
import jakarta.servlet.ServletOutputStream; import jakarta.servlet.ServletOutputStream;
import jakarta.servlet.http.HttpServletRequest; import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse; import jakarta.servlet.http.HttpServletResponse;
@@ -20,9 +19,12 @@ import org.springframework.stereotype.Service;
import java.io.*; import java.io.*;
import java.net.URI; import java.net.URI;
import java.net.URISyntaxException; import java.net.URISyntaxException;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.*; import java.util.*;
import static com.lion.lionwebsite.Util.CustomUtil.fourZeroFour; import static com.lion.lionwebsite.Util.CustomUtil.fourZeroFour;
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
@Service @Service
@Slf4j @Slf4j
@@ -33,12 +35,15 @@ public class QueryService {
public String query(String keyword, String prev, String next) { public String query(String keyword, String prev, String next) {
Response response = Response.generateResponse(); Response response = Response.generateResponse();
String result; String result;
String param = "?f_search=" + keyword.replace(" ", "+") + "&f_sft=on&f_sfu=on&f_sfl=on"; // 关键词来自用户输入,必须编码后再拼进查询串:未编码的 &、#、中文、空格
// 会被上游当成额外的查询参数或截断,搜索行为不可预期。
String param = "?f_search=" + URLEncoder.encode(keyword == null ? "" : keyword, StandardCharsets.UTF_8)
+ "&f_sft=on&f_sfu=on&f_sfl=on";
if(prev != null) if(prev != null)
param += "&prev=" + prev; param += "&prev=" + URLEncoder.encode(prev, StandardCharsets.UTF_8);
else if(next != null) else if(next != null)
param += "&next=" + next; param += "&next=" + URLEncoder.encode(next, StandardCharsets.UTF_8);
try{ try{
result = GalleryUtil.requests("https://exhentai.org/" + param, "get", null, null); result = GalleryUtil.requests("https://exhentai.org/" + param, "get", null, null);
@@ -49,35 +54,16 @@ public class QueryService {
} }
Document parse = Jsoup.parse(result); Document parse = Jsoup.parse(result);
Elements elements = parse.select("body > div.ido > div:nth-child(2) > table > tbody > tr"); java.util.List<GalleryForQuery> galleries;
ArrayList<GalleryForQuery> galleries = new ArrayList<>();
ListIterator<Element> elementListIterator = elements.listIterator();
try { try {
elementListIterator.next(); //skip firstlink galleries = com.lion.lionwebsite.Util.GallerySearchParser.parse(parse);
}catch (NullPointerException | NoSuchElementException e){ } catch (IOException | RuntimeException error) {
response.failure("没有搜索到结果"); log.warn("上游搜索页面格式不支持: {}", error.getClass().getSimpleName());
return response.toJSONString(); return response.failure("上游搜索页面格式不支持").toJSONString();
} }
if (galleries.isEmpty()) return response.failure("没有搜索到结果").toJSONString();
while (elementListIterator.hasNext()){ response.success(objectMapper.valueToTree(galleries));
Element element = elementListIterator.next();
GalleryForQuery gallery = new GalleryForQuery();
gallery.setType(element.child(0).text()); //type
String src = element.child(1).select("img").attr("src");
if (src.startsWith("data"))
src = element.child(1).select("img").attr("data-src");
src = src.replace("https://s.exhentai.org", "");
gallery.setThumbnailUrl(src); //thumbnailSrc
gallery.setUploadTime(element.child(1).select("div > div [onclick]").get(1).text()); //uploadTime
gallery.setLink(element.child(2).child(0).attr("href")); //link
gallery.setGid(GalleryUtil.parseGid(gallery.getLink()) + ""); //gid
gallery.setName(element.child(2).child(0).child(0).text()); //name
gallery.setPage(Integer.parseInt(element.child(3).child(1).text().split(" ")[0])); //page
galleries.add(gallery);
}
response.success(new ObjectMapper().valueToTree(galleries).toString());
Elements nextLink = parse.select("#unext"); Elements nextLink = parse.select("#unext");
if(nextLink.hasAttr("href")) if(nextLink.hasAttr("href"))
response.set("next", nextLink.attr("href")); response.set("next", nextLink.attr("href"));
@@ -18,7 +18,6 @@ import io.netty.util.concurrent.Promise;
import lombok.Data; import lombok.Data;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import jakarta.annotation.PostConstruct; import jakarta.annotation.PostConstruct;
@@ -38,6 +37,7 @@ import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException; import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.List;
@Service @Service
@Data @Data
@@ -69,8 +69,6 @@ public class RemoteService {
volatile boolean stopping; volatile boolean stopping;
volatile ServerSocket monitorSocket; volatile ServerSocket monitorSocket;
ExecutorService downloadThread = Executors.newCachedThreadPool();
Thread monitor; Thread monitor;
AtomicInteger atomicInteger = new AtomicInteger(0); AtomicInteger atomicInteger = new AtomicInteger(0);
@@ -79,12 +77,162 @@ public class RemoteService {
final SubscriptionStandbySnapshotService subscriptionStandbySnapshotService; final SubscriptionStandbySnapshotService subscriptionStandbySnapshotService;
/**
* 协议 v2 上报处理器。用 setter 注入而不是构造参数,避免 RemoteService 与
* GalleryTaskLifecycleService 之间的依赖环(后者需要 RemoteService 下发命令)。
*/
volatile TaskReportHandler taskReportHandler;
/**
* gid → 当前有效的下载 operationId。由生命周期服务在装配期注册(保持依赖方向单向)。
* 未注册或返回 null 时,重投/重试退化为旧的无 operationId 命令。
*/
volatile java.util.function.IntFunction<String> downloadOperationIdResolver;
/**
* gid → 下发判定(是否允许下发 + 用哪个稳定 operationId)。
*
* <p>替代「能解析出 operationId 就下发」的旧语义:UNKNOWN / 预占中 / 待删除的任务
* 必须被明确拒绝,否则未计量的人工对账任务可能在节点侧变成「下载完成却 0 额度」。
*/
volatile java.util.function.IntFunction<GalleryTaskLifecycleService.DispatchDecision> dispatchResolver;
/**
* 生命周期装配是否已经完成(handler/resolver 已绑定 + 启动恢复已执行)。
*
* <p>在它为 true 之前,绝不自动重投未完成任务:启动早期还不知道哪些任务是
* SUBMITTED、哪些是 UNKNOWN,盲目下发会制造未计量副作用。
*/
volatile boolean lifecycleReady;
/**
* 启动恢复是否安全完成。恢复失败时拒绝自动重投,避免在状态不明的库上继续派发。
*/
volatile boolean recoverySafe;
/**
* 节点是否声明了 v2 可靠任务能力。只有协商成功之后才发送类型 11..13,
* 否则继续走旧的无确认上报,保持向后兼容。
*/
volatile boolean v2Negotiated;
/** 本次连接的 Hello 协商结果;收到节点 Hello(或超时)后完成。 */
volatile CompletableFuture<Boolean> helloNegotiation;
/** 明确「未下发」的结果码:与 -1(发送失败/超时)区分开。 */
public static final byte REFUSED = -3;
/** 事务提交之后的通知/推送执行器:外部延迟(Telegram、慢浏览器)不得阻塞状态提交。 */
final ExecutorService notificationExecutor = Executors.newSingleThreadExecutor(r -> {
Thread thread = new Thread(r, "gallery-notify");
thread.setDaemon(true);
return thread;
});
/**
* 协议握手的专用线程。hello 等待响应是阻塞调用,绝不能放在 Netty 事件循环上,
* 否则响应本身要由同一个事件循环投递,会直接死锁。
*/
final ExecutorService handshakeExecutor = Executors.newSingleThreadExecutor(r -> {
Thread thread = new Thread(r, "node-protocol-handshake");
thread.setDaemon(true);
return thread;
});
final ExecutorService subscriptionSyncExecutor = Executors.newSingleThreadExecutor(r -> { final ExecutorService subscriptionSyncExecutor = Executors.newSingleThreadExecutor(r -> {
Thread thread = new Thread(r, "subscription-standby-sync"); Thread thread = new Thread(r, "subscription-standby-sync");
thread.setDaemon(true); thread.setDaemon(true);
return thread; return thread;
}); });
/** 测试/装配用:绑定上报处理器。 */
public void setTaskReportHandler(TaskReportHandler taskReportHandler) {
this.taskReportHandler = taskReportHandler;
}
public void setDownloadOperationIdResolver(java.util.function.IntFunction<String> resolver) {
this.downloadOperationIdResolver = resolver;
}
public void setDispatchResolver(java.util.function.IntFunction<GalleryTaskLifecycleService.DispatchDecision> resolver) {
this.dispatchResolver = resolver;
}
/**
* 由装配层在「绑定 handler/resolver + 完成启动恢复」之后调用:标记可以安全派发并建立连接。
*
* <p>初始连接不再放在 {@code @PostConstruct}:那时 handler/resolver 都还没绑定,
* 连接成功会把尚未确认的 UNKNOWN/预占任务当作普通任务重投。
*
* @param recoverySafe 启动恢复是否成功;false 时只允许连接,不允许自动重投
*/
public void startAfterWiring(boolean recoverySafe) {
this.recoverySafe = recoverySafe;
this.lifecycleReady = true;
initChannel();
}
/** 把 gid 解析成下发判定;未注册解析器或解析失败时按「拒绝下发」处理。 */
private GalleryTaskLifecycleService.DispatchDecision resolveDispatchDecision(int gid) {
java.util.function.IntFunction<GalleryTaskLifecycleService.DispatchDecision> resolver = dispatchResolver;
if (resolver == null)
return null;
try {
return resolver.apply(gid);
} catch (Exception e) {
log.warn("解析任务下发判定失败 gid={}", gid, e);
return null;
}
}
private String resolveDownloadOperationId(int gid) {
java.util.function.IntFunction<String> resolver = downloadOperationIdResolver;
if (resolver == null)
return null;
try {
return resolver.apply(gid);
} catch (Exception e) {
log.warn("解析任务 operationId 失败 gid={}", gid, e);
return null;
}
}
/** 节点是否已完成 v2 能力协商(供健康检查/测试观察)。 */
public boolean isReliableProtocolReady() {
return v2Negotiated;
}
/**
* 处理节点上报的任务状态。原先这些 JDBC 读写直接跑在 Netty 的 IO 线程上,
* N 条上报会阻塞该 IO 线程,连带拖慢心跳与请求响应;改为单线程顺序执行,
* 既不打乱「按上报顺序覆盖状态」的语义,也避免并发写库。
*/
ExecutorService statusApplyExecutor = newStatusApplyExecutor();
private static ExecutorService newStatusApplyExecutor() {
return Executors.newSingleThreadExecutor(r -> {
Thread thread = new Thread(r, "gallery-status-apply");
thread.setDaemon(true);
return thread;
});
}
/**
* 测试钩子:等待已提交的状态处理任务全部执行完毕。
*
* <p>单线程执行器天然有序,投递一个空任务并等它跑完,即说明此前提交的
* 任务都已处理完,从而让「上报后落库」的断言保持确定性。
*/
void awaitStatusApplied() {
try {
statusApplyExecutor.submit(() -> { }).get(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (Exception e) {
log.debug("等待状态处理完成失败", e);
}
}
final AtomicBoolean subscriptionSyncQueued = new AtomicBoolean(); final AtomicBoolean subscriptionSyncQueued = new AtomicBoolean();
final AtomicBoolean subscriptionSyncRunning = new AtomicBoolean(); final AtomicBoolean subscriptionSyncRunning = new AtomicBoolean();
@@ -92,10 +240,14 @@ public class RemoteService {
@Value("${subscription.standby.sync-enabled:false}") @Value("${subscription.standby.sync-enabled:false}")
boolean subscriptionSyncEnabled; boolean subscriptionSyncEnabled;
@PostConstruct /**
void init() { * 这里刻意不再自动连接。
initChannel(); *
} * <p>原实现在 PostConstruct 里立即 initChannel(),而那一刻 TaskLifecycleWiring
* 还没绑定上报处理器、operationId 解析器,也没执行启动恢复:连接成功后会马上把
* 未完成任务(含 UNKNOWN/预占)当作普通任务重投给节点。
* 现在改为由装配层在完成绑定与恢复后调用 startAfterWiring(boolean)。
*/
public boolean initChannel(){ public boolean initChannel(){
if (stopping || !connecting.compareAndSet(false, true)) if (stopping || !connecting.compareAndSet(false, true))
@@ -147,10 +299,32 @@ public class RemoteService {
channel = channelFuture.channel(); channel = channelFuture.channel();
closeMonitorSocket(); closeMonitorSocket();
channel.writeAndFlush(new IdentityMessage("lionwebsite")); channel.writeAndFlush(new IdentityMessage("lionwebsite"));
// 协议 v2:先发旧的 Identity,再发 hello 做能力协商。
//子节点上线时,发送未完成的任务 // 协商完成之前不允许下发任何任务,也不允许自动重投未完成任务。
resetUndone(); CompletableFuture<Boolean> negotiation = queueHelloHandshake();
// 在协商线程上等待结果,再决定重投;绝不阻塞 Netty 事件循环。
handshakeExecutor.execute(() -> {
boolean reliable = awaitNegotiation(negotiation);
// 订阅快照与任务重投都要求生命周期装配已完成;装配前只保持连接。
requestSubscriptionSync(); requestSubscriptionSync();
if (!reliable) {
log.info("节点未确认 v2 能力,跳过自动重投未完成任务");
return;
}
if (!lifecycleReady) {
log.info("生命周期装配未就绪,跳过自动重投未完成任务");
return;
}
if (!recoverySafe) {
log.error("启动恢复未安全完成,拒绝自动重投未完成任务");
return;
}
try {
resetUndone();
} catch (RuntimeException e) {
log.warn("重投未完成任务失败", e);
}
});
return true; return true;
}catch (Exception e){ }catch (Exception e){
log.error("connect node failed, wait for node back online", e); log.error("connect node failed, wait for node back online", e);
@@ -223,10 +397,10 @@ public class RemoteService {
} }
} }
@Scheduled(fixedDelayString = "${subscription.standby.retry-interval-ms:60000}") // 这里刻意没有「定期重发整份快照」的定时任务:
void scheduledSubscriptionSync() { // 快照只在订阅内容变化时推送(各处 requestSubscriptionSync),节点上线时补推一次。
requestSubscriptionSync(); // 备机的过期语义是「主站失联」,由节点按「收到主站的任何消息」判定新鲜度——
} // 主站每 30 分钟的可用性检查即可持续刷新它,因此不需要为续期而周期性传输快照。
private void drainSubscriptionSyncQueue() { private void drainSubscriptionSyncQueue() {
try { try {
@@ -266,8 +440,10 @@ public class RemoteService {
if (monitor != null) monitor.interrupt(); if (monitor != null) monitor.interrupt();
failPendingRequests(); failPendingRequests();
if (channel != null) channel.close(); if (channel != null) channel.close();
statusApplyExecutor.shutdownNow();
notificationExecutor.shutdownNow();
handshakeExecutor.shutdownNow();
subscriptionSyncExecutor.shutdownNow(); subscriptionSyncExecutor.shutdownNow();
downloadThread.shutdownNow();
networkGroup.shutdownGracefully(); networkGroup.shutdownGracefully();
eventLoopGroup.shutdownGracefully(); eventLoopGroup.shutdownGracefully();
} }
@@ -285,7 +461,11 @@ public class RemoteService {
} }
public void resetUndone(){ public void resetUndone(){
if (channelFuture.channel() == null || !channelFuture.channel().isActive()) // 只有「连接在线 + v2 已协商 + 生命周期装配完成 + 启动恢复安全完成」才允许批量重投;
// 任何一项不满足都说明还不知道任务真实状态,盲目下发会制造未计量副作用。
if (channelFuture == null || channelFuture.channel() == null || !channelFuture.channel().isActive())
return;
if (!v2Negotiated || !lifecycleReady || !recoverySafe)
return; return;
Gallery[] galleries = galleryMapper.selectUnDoneGalleries(); Gallery[] galleries = galleryMapper.selectUnDoneGalleries();
@@ -298,15 +478,135 @@ public class RemoteService {
} }
public byte addGalleryToQueue(Gallery gallery){ public byte addGalleryToQueue(Gallery gallery){
GalleryTaskLifecycleService.DispatchDecision decision = resolveDispatchDecision(gallery.getGid());
if (decision != null) {
// 明确拒绝(UNKNOWN/预占/待删除等):绝不能退化为无 operationId 的旧命令。
if (!decision.dispatchable())
return REFUSED;
return sendTaskCommand(TaskCommandMessage.ACTION_DOWNLOAD, decision.operationId(),
gallery.getGid(), gallery.getName());
}
// 兼容:装配层未注册判定解析器时(如单元测试),退回只按 operationId 解析的旧行为。
String operationId = resolveDownloadOperationId(gallery.getGid());
if (operationId != null)
return sendTaskCommand(TaskCommandMessage.ACTION_DOWNLOAD, operationId, gallery.getGid(), gallery.getName());
return legacyDownloadCommand(gallery.getGid(), gallery.getName());
}
/** 旧的无 operationId 下载命令,供未启用 v2 或无法解析 operationId 时退避使用。 */
private byte legacyDownloadCommand(int gid, String name) {
GalleryTask galleryTask = new GalleryTask(); GalleryTask galleryTask = new GalleryTask();
galleryTask.setGid(gallery.getGid()); galleryTask.setGid(gid);
galleryTask.setName(gallery.getName()); galleryTask.setName(name);
DownloadPostMessage message = new DownloadPostMessage(); DownloadPostMessage message = new DownloadPostMessage();
message.setGalleryTask(galleryTask); message.setGalleryTask(galleryTask);
return sendRequest(message, 10, TimeUnit.SECONDS); return sendRequest(message, 10, TimeUnit.SECONDS);
} }
/**
* 在专用线程上执行 hello 协商,并返回一个在「收到节点 Hello」或「等待超时」后完成的 future。
*
* <p>节点的应答是 Hello 而不是 ResponseMessage,因此绝不能走 sendRequest 的响应等待器
* (那会白等 10 秒超时,排在后面的自动重投也只能 10 秒后才开始)。
* 这里直接 writeAndFlush,并让 channelRead 的 Hello 分支完成 future;
* 超时用 completeOnTimeout 处理,写失败立刻完成。
*/
private CompletableFuture<Boolean> queueHelloHandshake() {
// 10 秒内没等到节点 Hello 就按「未协商」完成,避免后续重投被无限期挂起。
CompletableFuture<Boolean> result = new CompletableFuture<Boolean>()
.completeOnTimeout(false, 10, TimeUnit.SECONDS);
helloNegotiation = result;
if (stopping) {
result.complete(false);
return result;
}
try {
handshakeExecutor.execute(() -> {
v2Negotiated = false;
try {
Channel target = channel;
if (target == null || !target.isActive()) {
result.complete(false);
return;
}
// Hello 不写入 promiseHashMap:节点回的是 Hello,没有 ResponseMessage 等待者。
target.writeAndFlush(new ProtocolHelloMessage()).addListener(future -> {
if (!future.isSuccess()) {
if (!stopping)
log.warn("协议协商发送失败: {}", future.cause() == null
? "unknown" : future.cause().getClass().getSimpleName());
result.complete(false);
}
});
} catch (RuntimeException e) {
log.warn("协议协商失败", e);
result.complete(false);
}
});
} catch (java.util.concurrent.RejectedExecutionException e) {
if (!stopping) log.warn("安排协议协商失败", e);
result.complete(false);
}
return result;
}
/** 等待协商结果;超时或中断都按「未协商」处理(宁可不下发,也不盲目重投)。 */
boolean awaitNegotiation(CompletableFuture<Boolean> negotiation) {
if (negotiation == null)
return v2Negotiated;
try {
// future 自身已在 10 秒时按「未协商」完成,这里留少量余量即可。
return negotiation.get(12, TimeUnit.SECONDS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return false;
} catch (Exception e) {
return false;
}
}
/**
* 下发一条带稳定 operationId 的任务命令;未协商 v2 时退化为旧的 DownloadPostMessage,
* 保证新旧节点组合仍可用(旧节点上无法保证可靠 ACK,这是已知的兼容边界)。
*/
public byte sendTaskCommand(String action, String operationId, int gid, String name) {
if (!v2Negotiated)
return TaskCommandMessage.ACTION_DELETE.equals(action)
? legacyDeleteCommand(gid, name)
: legacyDownloadCommand(gid, name);
TaskCommandMessage command = new TaskCommandMessage();
command.setOperationId(operationId);
command.setAction(action);
command.setGid(gid);
command.setName(name);
return sendRequest(command, 10, TimeUnit.SECONDS);
}
/** 删除命令:带独立稳定 operationId,重试复用同一个 ID。 */
public byte sendDeleteCommand(String operationId, int gid, String name) {
if (!v2Negotiated)
return legacyDeleteCommand(gid, name);
TaskCommandMessage command = new TaskCommandMessage();
command.setOperationId(operationId);
command.setAction(TaskCommandMessage.ACTION_DELETE);
command.setGid(gid);
command.setName(name);
return sendRequest(command, 10, TimeUnit.SECONDS);
}
/** 旧的无 operationId 删除命令,供未协商 v2 时退避使用。 */
private byte legacyDeleteCommand(int gid, String name) {
Gallery gallery = new Gallery();
gallery.setGid(gid);
gallery.setName(name);
return deleteGallery(gallery);
}
public RetryResult retryGallery(Gallery gallery){ public RetryResult retryGallery(Gallery gallery){
// 提交结果未知或被删除的任务不能重试:节点可能已经把未计量任务当成新下载。
GalleryTaskLifecycleService.DispatchDecision decision = resolveDispatchDecision(gallery.getGid());
if (decision != null && !decision.dispatchable())
return new RetryResult(false, "该任务提交结果未知或正在删除,需要人工对账后再重试");
CompletableFuture<String> statusFuture = new CompletableFuture<>(); CompletableFuture<String> statusFuture = new CompletableFuture<>();
retryStatusWaiters.computeIfAbsent(gallery.getGid(), ignored -> new CopyOnWriteArrayList<>()) retryStatusWaiters.computeIfAbsent(gallery.getGid(), ignored -> new CopyOnWriteArrayList<>())
.add(statusFuture); .add(statusFuture);
@@ -314,6 +614,13 @@ public class RemoteService {
byte submitResult = addGalleryToQueue(gallery); byte submitResult = addGalleryToQueue(gallery);
if(submitResult != 0 && !statusFuture.isDone()) if(submitResult != 0 && !statusFuture.isDone())
return new RetryResult(false, "节点未接受重试请求"); return new RetryResult(false, "节点未接受重试请求");
// 竞态兜底:本方法读到的 Gallery 是调用方传入的旧快照,注册等待者与节点被通知之间
// 节点可能刚完成清理(例如 ACK 之后已清队列、重复 DOWNLOAD 只回放 receipt 不再产生新上报)。
// 此时按「当前库里的真实状态」收尾:只认重新读到的完成状态,绝不把传入的旧进行中状态
// 当作节点的新确认返回。
String completedStatus = currentCompletedStatus(gallery.getGid());
if (completedStatus != null && !statusFuture.isDone())
statusFuture.complete(completedStatus);
return new RetryResult(true, statusFuture.get(10, TimeUnit.SECONDS)); return new RetryResult(true, statusFuture.get(10, TimeUnit.SECONDS));
}catch (TimeoutException e){ }catch (TimeoutException e){
return new RetryResult(false, "节点已收到重试请求,但未及时返回任务状态"); return new RetryResult(false, "节点已收到重试请求,但未及时返回任务状态");
@@ -338,6 +645,22 @@ public class RemoteService {
waiters.forEach(waiter -> waiter.complete(status)); waiters.forEach(waiter -> waiter.complete(status));
} }
/**
* 重新读库确认任务是否已真正完成;只用于「节点已接受重试」之后的竞态兜底。
*
* <p>读库失败一律返回 null:宁可继续等上报,也不能凭一次失败的读把未知状态说成完成。
*/
private String currentCompletedStatus(int gid) {
try {
Gallery current = galleryMapper.selectGalleryByGid(gid);
if (current != null && TaskStatus.COMPLETE.equals(current.getStatus()))
return current.getStatus();
} catch (RuntimeException e) {
log.warn("重试时读取任务当前状态失败 gid={}", gid, e);
}
return null;
}
public record RetryResult(boolean success, String message) {} public record RetryResult(boolean success, String message) {}
public byte deleteGallery(Gallery gallery){ public byte deleteGallery(Gallery gallery){
@@ -407,6 +730,104 @@ public class RemoteService {
//下载状态 //下载状态
if(msg instanceof DownloadStatusMessage dsm){ if(msg instanceof DownloadStatusMessage dsm){
GalleryTask[] galleryTasks = dsm.getGalleryTasks(); GalleryTask[] galleryTasks = dsm.getGalleryTasks();
// 落库与通知可能较慢(含 Telegram 推送),交给单线程顺序执行,
// 不再占用 Netty IO 线程;异常在此兜住,避免打进 IO 线程。
statusApplyExecutor.execute(() -> {
try {
applyReportedStatus(galleryTasks);
} catch (Exception e) {
log.warn("处理节点上报的任务状态失败", e);
}
});
}
else if(msg instanceof ProtocolHelloMessage hello) {
// 节点声明了 v2 可靠任务能力;此后才允许使用类型 11..13。
v2Negotiated = hello.supportsReliableTasks();
log.info("节点协议协商完成 v2={} version={}", v2Negotiated, hello.getProtocolVersion());
CompletableFuture<Boolean> negotiation = helloNegotiation;
if (negotiation != null)
negotiation.complete(v2Negotiated);
}
else if(msg instanceof TaskReportMessage report) {
// 事务提交与 ACK 都放在单线程消费队列里顺序执行,避免并发写库。
statusApplyExecutor.execute(() -> consumeV2Report(report));
}
else if(msg instanceof ResponseMessage rsm) {
Promise<AbstractMessage> promise = promiseHashMap.remove(rsm.messageId);
if(promise != null)
promise.trySuccess(rsm);
else
log.warn("收到无等待者的响应消息: messageId={}", rsm.messageId);
}
}
/** 消费协议 v2 上报:事务提交 → 唤醒重试等待者 → 回 ACK → 独立执行器做通知/推送。 */
private void consumeV2Report(TaskReportMessage report) {
try {
TaskReportHandler handler = taskReportHandler;
if (handler == null) {
log.warn("收到 v2 上报但未装配处理器,暂不回执 reportId={}", report.getReportId());
return;
}
TaskReportHandler.CommittedReport committed = handler.apply(report);
if (committed == null) {
// 提交失败:不回 ACK,节点保留快照后重投。
log.warn("v2 上报未提交,等待节点重投 reportId={}", report.getReportId());
return;
}
// 先完成等待者:apply 已确认提交,这一步绝不能因随后的 ACK 写入抛异常而丢失。
completeRetryStatusWaitersFor(committed.acceptedTasks());
// 提交成功之后才回执,让节点清理快照。
Channel target = channel;
if (target != null && target.isActive())
target.writeAndFlush(new TaskReportAckMessage(committed.reportId(), TaskReportAckMessage.RESULT_SUCCESS));
dispatchNotifications(report, committed);
} catch (Exception e) {
log.warn("处理 v2 上报失败 reportId={}", report.getReportId(), e);
}
}
/** 事务提交之后、ACK 与通知之前,按「真正被接受的更新」唤醒等待重试结果的调用方。
* 被过滤的过期/旧 operation/待删除任务不在 acceptedTasks 里,不得完成等待者。 */
private void completeRetryStatusWaitersFor(List<GalleryTask> acceptedTasks) {
if (acceptedTasks == null || acceptedTasks.isEmpty())
return;
for (GalleryTask task : acceptedTasks) {
if (task == null)
continue;
String status = TaskStatus.fromReport(task.getStatus());
if (status != null)
completeRetryStatusWaiters(task.getGid(), status);
}
}
/** 通知/推送在独立执行器里执行,外部延迟不会阻塞状态提交。 */
private void dispatchNotifications(TaskReportMessage report, TaskReportHandler.CommittedReport committed) {
// 只推送事务里真正被接受的更新(含下载中/压缩中等中间状态);
// 过期、旧世代、待删除任务已被过滤掉,不会把它们带到前端。
GalleryTask[] accepted = committed.acceptedTasks() == null
? new GalleryTask[0]
: committed.acceptedTasks().toArray(new GalleryTask[0]);
List<Gallery> newlyCompleted = committed.newlyCompleted();
try {
notificationExecutor.execute(() -> {
try {
for (Gallery gallery : newlyCompleted)
pushService.downloadComplete(gallery);
if (accepted.length > 0)
webSocketService.updateTaskProcessing(accepted);
} catch (Exception e) {
log.warn("任务完成通知/推送失败", e);
}
});
} catch (java.util.concurrent.RejectedExecutionException e) {
// 通知队列已关闭只是「推不出去」,既不能影响已提交的 ACK,也不能影响重试等待者。
log.warn("通知任务被拒绝,状态仍已提交 reportId={}", report.getReportId());
}
}
/** 把节点上报的任务状态写回库里并推送给前端,按上报顺序逐个处理。 */
private void applyReportedStatus(GalleryTask[] galleryTasks) {
for (GalleryTask galleryTask : galleryTasks) { for (GalleryTask galleryTask : galleryTasks) {
Gallery gallery = galleryMapper.selectGalleryByGid(galleryTask.getGid()); Gallery gallery = galleryMapper.selectGalleryByGid(galleryTask.getGid());
if (gallery == null) { if (gallery == null) {
@@ -438,14 +859,6 @@ public class RemoteService {
} }
webSocketService.updateTaskProcessing(galleryTasks); webSocketService.updateTaskProcessing(galleryTasks);
} }
else if(msg instanceof ResponseMessage rsm) {
Promise<AbstractMessage> promise = promiseHashMap.remove(rsm.messageId);
if(promise != null)
promise.trySuccess(rsm);
else
log.warn("收到无等待者的响应消息: messageId={}", rsm.messageId);
}
}
@Override @Override
public void channelUnregistered(ChannelHandlerContext ctx) { public void channelUnregistered(ChannelHandlerContext ctx) {
@@ -17,25 +17,53 @@ import lombok.extern.slf4j.Slf4j;
import org.jsoup.Jsoup; import org.jsoup.Jsoup;
import org.jsoup.select.Elements; import org.jsoup.select.Elements;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.transaction.support.TransactionTemplate;
import jakarta.annotation.PreDestroy;
import java.io.IOException; import java.io.IOException;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.nio.file.Files; import java.nio.file.Files;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Date; import java.util.Date;
import java.util.concurrent.Executor;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.Lock;
import java.util.function.Supplier; import java.util.function.Supplier;
@Service @Service
@Slf4j @Slf4j
@RequiredArgsConstructor @RequiredArgsConstructor
public class SubService { public class SubService {
final SubMapper subMapper; final SubMapper subMapper;
final UserMapper userMapper; final UserMapper userMapper;
final SubscriptionRefreshService refreshService; final SubscriptionRefreshService refreshService;
final RemoteService remoteService; final RemoteService remoteService;
final SubscriptionStateCoordinator stateCoordinator; final SubscriptionStateCoordinator stateCoordinator;
@Autowired
@Qualifier("mainTransactionTemplate")
TransactionTemplate transactions;
/**
* 归属地查询是同步外呼(最长十几秒),且失败只影响记录里的一列展示。
* 若放在订阅分发路径上,用户取一次订阅就得先等它完成,因此改为后台补齐。
*
* <p>非 final:测试可替换为受控执行器,避免在单测里触发真实外呼。
*/
Executor locationExecutor = newFixedLocationExecutor();
private static ExecutorService newFixedLocationExecutor() {
return new java.util.concurrent.ThreadPoolExecutor(2, 2, 0, java.util.concurrent.TimeUnit.MILLISECONDS,
new java.util.concurrent.ArrayBlockingQueue<>(128), r -> {
Thread thread = new Thread(r, "sub-location-resolver");
thread.setDaemon(true);
return thread;
}, new java.util.concurrent.ThreadPoolExecutor.AbortPolicy());
}
public String insertSubscriptionAccount(String name, String upstreamKey, boolean filterHighMultiplier, boolean enabled) { public String insertSubscriptionAccount(String name, String upstreamKey, boolean filterHighMultiplier, boolean enabled) {
Response response = Response.generateResponse(); Response response = Response.generateResponse();
SubscriptionAccount account; SubscriptionAccount account;
@@ -46,7 +74,11 @@ public class SubService {
return response.failure("名称和上游 key 不能为空").toJSONString(); return response.failure("名称和上游 key 不能为空").toJSONString();
if (subMapper.countSubscriptionAccountName(name.trim()) > 0 || subMapper.countSubscriptionAccountKey(upstreamKey.trim()) > 0) if (subMapper.countSubscriptionAccountName(name.trim()) > 0 || subMapper.countSubscriptionAccountKey(upstreamKey.trim()) > 0)
return response.failure("名称或上游 key 已存在").toJSONString(); return response.failure("名称或上游 key 已存在").toJSONString();
account = new SubscriptionAccount(null, name.trim(), upstreamKey.trim(), filterHighMultiplier, enabled, null, null, null, null, 0, null, null); account = new SubscriptionAccount();
account.setName(name.trim());
account.setUpstreamKey(upstreamKey.trim());
account.setFilterHighMultiplier(filterHighMultiplier);
account.setEnabled(enabled);
subMapper.insertSubscriptionAccount(account); subMapper.insertSubscriptionAccount(account);
} finally { } finally {
lock.unlock(); lock.unlock();
@@ -188,6 +220,16 @@ public class SubService {
} }
public void updateSub(HttpServletResponse response, HttpServletRequest request, String client, String key) { public void updateSub(HttpServletResponse response, HttpServletRequest request, String client, String key) {
Lock lock = stateCoordinator.readLock();
lock.lock();
try {
distributeSubscription(response, request, client, key);
} finally {
lock.unlock();
}
}
private void distributeSubscription(HttpServletResponse response, HttpServletRequest request, String client, String key) {
if (key == null || client == null) if (key == null || client == null)
return; return;
SubBind subBind = subMapper.selectSubBind(key); SubBind subBind = subMapper.selectSubBind(key);
@@ -198,7 +240,7 @@ public class SubService {
String ip = resolveClientIp(request); String ip = resolveClientIp(request);
String ua = request.getHeader("User-Agent"); String ua = request.getHeader("User-Agent");
if (ua == null) return; if (ua == null) return;
recordUpdate(subBind.getUser(), ip, ua); recordUpdate(subBind, ip, ua);
if (!"v2".equals(client) && !"cat".equals(client)) { if (!"v2".equals(client) && !"cat".equals(client)) {
sendStatus(response, HttpServletResponse.SC_BAD_REQUEST, "client error"); sendStatus(response, HttpServletResponse.SC_BAD_REQUEST, "client error");
return; return;
@@ -236,18 +278,50 @@ public class SubService {
|| "0:0:0:0:0:0:0:1".equals(address); || "0:0:0:0:0:0:0:1".equals(address);
} }
private void recordUpdate(String user, String ip, String ua) { private void recordUpdate(SubBind binding, String ip, String ua) {
String location; String user = binding.getUser();
// 先落一条 unknown 记录并立即返回,归属地随后在后台补齐;
// 这样取订阅的延迟不再取决于第三方定位站点的响应时间。
SubUpdateRecord record = new SubUpdateRecord(0, user, ip, ua, new Date(), "unknown");
record.setUserId(binding.getUserId());
subMapper.insertSubUpdateRecord(record);
if (subMapper.selectUpdateRecordCount(user) > 10)
subMapper.deleteSubUpdateRecordById(subMapper.selectMinUpdateRecordId(user));
// Capture the generated row ID: another request may arrive before the lookup finishes.
if (record.getId() > 0) {
try {
locationExecutor.execute(() -> enrichLocation(record.getId(), ip));
} catch (java.util.concurrent.RejectedExecutionException busy) {
log.debug("位置查询队列已满,保留 unknown recordId={}", record.getId());
}
}
}
/** 后台按记录 ID 补齐归属地;任何失败都只记日志,不影响已落库的记录。 */
private void enrichLocation(int recordId, String ip) {
try {
String location = resolveLocation(ip);
if (location == null || location.isBlank() || "unknown".equals(location))
return;
subMapper.updateSubUpdateRecordLocation(recordId, location);
} catch (Exception e) {
log.debug("补齐订阅更新记录归属地失败 recordId={}", recordId, e);
}
}
/**
* 查询 IP 归属地。第三方站点结构变化或网络异常都只降级为 unknown,
* 绝不让定位失败影响订阅分发本身。
*/
static String resolveLocation(String ip) {
try { try {
String page = GalleryUtil.requests("https://www.ip138.com/iplookup.php?ip=" + ip, "get", null, null); String page = GalleryUtil.requests("https://www.ip138.com/iplookup.php?ip=" + ip, "get", null, null);
Elements tds = Jsoup.parse(page).select("body > div > div.container > div.content > div > div:nth-child(2) > div.group-left > div > div.bd > div.table-outer > div.table-box > table > tbody > tr > td"); Elements tds = Jsoup.parse(page).select("body > div > div.container > div.content > div > div:nth-child(2) > div.group-left > div > div.bd > div.table-outer > div.table-box > table > tbody > tr > td");
location = tds.size() > 3 ? tds.get(1).text().replace("中国", "") + " " + tds.get(3).text().trim() : (tds.isEmpty() ? "unknown" : tds.get(1).text()); return tds.size() > 3 ? tds.get(1).text().replace("中国", "") + " " + tds.get(3).text().trim()
: (tds.isEmpty() ? "unknown" : tds.get(1).text());
} catch (Exception e) { } catch (Exception e) {
location = "unknown"; return "unknown";
} }
subMapper.insertSubUpdateRecord(new SubUpdateRecord(0, user, ip, ua, new Date(), location));
if (subMapper.selectUpdateRecordCount(user) > 10)
subMapper.deleteSubUpdateRecordById(subMapper.selectMinUpdateRecordId(user));
} }
public String deleteSubBind(String user) { public String deleteSubBind(String user) {
@@ -265,20 +339,26 @@ public class SubService {
Lock stateLock = stateCoordinator.writeLock(); Lock stateLock = stateCoordinator.writeLock();
stateLock.lock(); stateLock.lock();
try { try {
return action.get(); return transactions == null ? action.get() : transactions.execute(ignored -> action.get());
} finally { } finally {
stateLock.unlock(); stateLock.unlock();
} }
} }
@PreDestroy
void shutdownLocationResolver() {
if (locationExecutor instanceof ExecutorService service)
service.shutdownNow();
}
private SubscriptionAccount firstEnabledAccount() { private SubscriptionAccount firstEnabledAccount() {
return subMapper.selectAllSubscriptionAccounts().stream().filter(SubscriptionAccount::isEnabled).findFirst().orElse(null); return subMapper.selectAllSubscriptionAccounts().stream().filter(SubscriptionAccount::isEnabled).findFirst().orElse(null);
} }
private String accountJson(SubscriptionAccount account) { private tools.jackson.databind.JsonNode accountJson(SubscriptionAccount account) {
account.setV2Url(refreshService.v2Url(account)); account.setV2Url(refreshService.v2Url(account));
account.setClashUrl(refreshService.clashUrl(account)); account.setClashUrl(refreshService.clashUrl(account));
return CustomUtil.objectMapper.valueToTree(account).toString(); return CustomUtil.objectMapper.valueToTree(account);
} }
private static void sendStatus(HttpServletResponse response, int status, String message) { private static void sendStatus(HttpServletResponse response, int status, String message) {
@@ -0,0 +1,225 @@
package com.lion.lionwebsite.Service;
import java.time.Clock;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Random;
/**
* 为一批子账号排定刷新时刻。
*
* <p>目标:每个账号每 24 小时内至少刷新一次,且各账号的时刻互相错开,
* 不出现「一次全量、十几个账号在十几秒内打完」的爆发式请求。
*
* <p>排程模型是「错峰 + 固定相位」:
* <ol>
* <li><b>错峰</b>:首次排程把账号随机铺满一个窗口,两两之间至少隔开 {@code minGap};</li>
* <li><b>固定相位</b>:账号每次成功后按窗口长度(略减一个 tick)推进,
* 因此各自的相位保持不变,分散效果自我维持,不需要每天重新洗牌;</li>
* <li><b>兜底</b>:任何情况下下一次刷新都不会晚于「上次成功 + 窗口」,
* 把「24 小时内必然刷新一次」从概率保证变成确定保证。</li>
* </ol>
*
* <p>本类是纯函数式设计(注入 {@link Clock} 与带种子的 {@link Random}),
* 不触碰数据库与网络,因此可以直接做确定性单元测试。
*/
public class SubscriptionRefreshPlanner {
private final Clock clock;
private final Random random;
private final Duration window;
private final Duration minGap;
/** 一个调度周期的最大延迟;用它给窗口留余量,使实际间隔严格落在窗口内。 */
private final Duration tickInterval;
/** 刷新失败后的重试间隔。 */
private final Duration retryDelay;
/**
* @param clock 时间源,测试可注入固定时钟
* @param random 随机源,测试可注入固定种子以复现排程
* @param window 刷新窗口,即「24 小时内至少一次」中的 24 小时
* @param minGap 相邻两个账号之间的最小间隔
* @param tickInterval 调度周期,用于给窗口留出余量
* @param retryDelay 失败后的重试间隔
*/
public SubscriptionRefreshPlanner(Clock clock, Random random, Duration window, Duration minGap,
Duration tickInterval, Duration retryDelay) {
if (window.isZero() || window.isNegative())
throw new IllegalArgumentException("刷新窗口必须为正数");
if (minGap.isNegative())
throw new IllegalArgumentException("最小间隔不能为负数");
if (tickInterval.isNegative())
throw new IllegalArgumentException("调度周期不能为负数");
if (retryDelay.isZero() || retryDelay.isNegative())
throw new IllegalArgumentException("重试间隔必须为正数");
if (retryDelay.compareTo(window) > 0)
throw new IllegalArgumentException("重试间隔不能大于刷新窗口,否则无法保证窗口内重试");
this.clock = clock;
this.random = random;
this.window = window;
this.minGap = minGap;
this.tickInterval = tickInterval;
this.retryDelay = retryDelay;
}
/** 便捷构造:重试间隔默认取 1 小时。 */
public SubscriptionRefreshPlanner(Clock clock, Random random, Duration window, Duration minGap,
Duration tickInterval) {
this(clock, random, window, minGap, tickInterval, Duration.ofHours(1));
}
/**
* 账号数与最小间隔是否放得下。
*
* <p>注意这里刻意只用于「提示」而不是阻断启动:账号数是由用户在管理页决定的,
* 若因为它超限就让整个应用起不来,等于把配置细节变成一次线上故障。
* 放不下时 {@link #initialSchedule} 仍会把账号铺满整个窗口(只是间隔小于期望值),
* 结果是错峰效果变弱,而不是退回串行爆发。
*/
public boolean isCapacitySufficient(int accountCount) {
if (accountCount <= 1)
return true;
return minGap.toMillis() * (accountCount - 1) <= window.toMillis();
}
/** 容量不足时的告警文案,供调用方记录日志。 */
public String capacityMessage(int accountCount) {
long required = minGap.toMillis() * (accountCount - 1);
return String.format(
"刷新窗口放不下最小间隔:账号 %d 个、最小间隔 %d 分钟,需要 %d 分钟,但窗口只有 %d 分钟;"
+ "实际仍会铺满窗口,只是账号间间隔会小于期望值。可调小 subscription.refresh.min-gap-minutes。",
accountCount, minGap.toMinutes(),
Duration.ofMillis(required).toMinutes(), window.toMinutes());
}
/**
* 为一批「已成功刷新过」的账号铺开计划,返回顺序与 accountIds 一致。
*
* <p>已成功过的账号内容仍然可用,因此可以安心分散到整个窗口,
* 不会因为排在 20 小时后而让用户拿不到订阅。
*/
public List<Long> initialSchedule(List<Integer> accountIds) {
List<Long> result = new ArrayList<>(accountIds.size());
if (accountIds.isEmpty())
return result;
long now = clock.millis();
long windowMillis = window.toMillis();
int count = accountIds.size();
long slotWidth = windowMillis / count;
long jitterCap = Math.max(1, Math.min(slotWidth, Math.max(1, slotWidth - minGap.toMillis())));
List<Integer> slots = new ArrayList<>(count);
for (int i = 0; i < count; i++)
slots.add(i);
Collections.shuffle(slots, random);
long[] planned = new long[count];
for (int accountIndex = 0; accountIndex < count; accountIndex++) {
long offset = slots.get(accountIndex) * slotWidth + (long) (random.nextDouble() * jitterCap);
planned[accountIndex] = now + Math.min(offset, windowMillis - 1);
}
for (int i = 0; i < count; i++)
result.add(planned[i]);
return result;
}
/**
* 为尚未成功过的账号排程:尽快刷,但互相错开。
*
* <p>这类账号(新建、或上游 key 刚改过)可能根本没有可用缓存,用户立刻请求订阅
* 就会拿不到内容,因此不能像常规账号那样分散到整个窗口。这里把它们按
* {@code spacing} 依次排开,配合调度器的单轮上限逐批刷新,
* 既尽早补齐,又避免一批新账号同时打上游。
*
* @param accountIds 账号 ID,顺序即刷新优先级
* @param spacing 相邻两个账号的间隔(通常取一个 tick)
*/
public List<Long> coldStartSchedule(List<Integer> accountIds, Duration spacing) {
List<Long> result = new ArrayList<>(accountIds.size());
long now = clock.millis();
long step = Math.max(1, spacing.toMillis());
for (int i = 0; i < accountIds.size(); i++)
result.add(now + i * step);
return result;
}
/**
* 已排定的时刻是否已到期。
*
* <p>直接以排定时刻为准,这样分散效果与失败退避都得以保持。
* 只额外兜住一种异常:计划被写到超过一个窗口之后。这不可能是本调度器产生的
* 排程(成功时排在窗口减一个 tick 之后,失败时排在 4 小时以内),
* 按「计划损坏」处理并立即到期,从而保证 24 小时内必定刷新。
*
* <p>尚未排程({@code scheduled == null})视为立即到期,由调用方先行铺开计划。
*/
public boolean isDue(Long scheduled, long now) {
if (scheduled == null)
return true;
if (scheduled - now > window.toMillis())
return true;
return scheduled <= now;
}
/**
* 成功之后的下一次刷新时刻。
*
* <p>只推进「窗口 - 一个调度周期」,使 tick 即使晚触发也不会突破 24 小时;
* 从上次成功的时刻起算,保证相邻两次的间隔严格小于窗口。
*/
public long nextAfterSuccess(long successMillis) {
long advance = Math.max(1, window.minus(tickInterval).toMillis());
return successMillis + advance;
}
/**
* 失败之后的重试时刻。
*
* <p>用固定的重试间隔而不是「失败次数 × 指数退避」:刷新成功会清空 {@code last_error},
* 数据库里没有「连续失败次数」这一栏,退避指数实际上只能取到 0 或 1,
* 写出来会让人误以为有指数退避而其实没有。固定的重试间隔行为可预期,
* 且远小于窗口,不会威胁「24 小时内尝试刷新一次」。
*
* <p>唯一需要裁剪的情况:若「上次成功 + 窗口」还没到,则重试不得晚于它;
* 已经越过该点时保留重试时刻本身——重复请求并不会提高成功率,
* 反过来在每个 tick 重打上游才是真正的问题,这种情况交由陈旧告警处理。
*
* @param now 当前时刻
* @param lastSuccess 上次成功时刻;未知时传 null,不做裁剪
*/
public long nextAfterFailure(long now, Long lastSuccess) {
long candidate = now + retryDelay.toMillis();
if (lastSuccess == null)
return candidate;
long deadline = lastSuccess + window.toMillis();
return deadline > now ? Math.min(candidate, deadline) : candidate;
}
/** 是否已越过兜底线(距上次成功已达一个窗口),用于诊断与告警判定。 */
public boolean isOverdue(Long lastSuccess, long now) {
return lastSuccess != null && now - lastSuccess >= window.toMillis();
}
private long now() {
return clock.millis();
}
public Clock clock() {
return clock;
}
public Duration window() {
return window;
}
public Duration minGap() {
return minGap;
}
public Duration tickInterval() {
return tickInterval;
}
}
@@ -0,0 +1,247 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.SubMapper;
import com.lion.lionwebsite.Domain.SubscriptionAccount;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
/**
* 分散刷新各子账号的订阅。
*
* <p>取代原先「每 24 小时一次全量」的调度。旧写法有两个问题:所有账号在十几秒内
* 背靠背打完,且 {@code @Scheduled(fixedRate = 86400000)} 没有 initialDelay,
* 首次触发即立刻执行——于是每次重启都会重刷一遍全部账号,上游看到的请求密度取决于
* 部署频率,而不是每天一次。
*
* <p>现在改为高频轻量 tick,每轮只刷新已经到期的账号。「24 小时内每个账号至少
* 更新一次」由三层保证:
* <ol>
* <li>成功后排定「成功时刻 + 窗口 - 一个 tick」,相位自我维持,分散效果不会因
* 重启或重排而退化;</li>
* <li>到期时刻存在数据库里,停机期间错过计划的账号在恢复后立即到期;</li>
* <li>兜底闸:任何排定时刻都被裁剪到「上次成功 + 窗口」以内,越线即立刻刷新。</li>
* </ol>
*/
@Service
@Slf4j
public class SubscriptionRefreshScheduler {
private final SubMapper subMapper;
private final SubscriptionRefreshService refreshService;
private final SubscriptionRefreshPlanner planner;
private final PushService pushService;
private final RemoteService remoteService;
/** 单个 tick 最多刷新几个账号,避免重启后一批到期账号同时开火。 */
@Value("${subscription.refresh.max-per-tick:2}")
int maxPerTick;
/** 超过窗口的这个倍数仍未成功即告警,避免静默失败。 */
@Value("${subscription.refresh.stale-alert-multiplier:2.0}")
double staleAlertMultiplier;
/** 已告警过的账号,避免每轮重复推送;刷新成功后移除。 */
private final Set<Integer> alertedAccounts = new HashSet<>();
public SubscriptionRefreshScheduler(SubMapper subMapper,
SubscriptionRefreshService refreshService,
SubscriptionRefreshPlanner planner,
PushService pushService,
RemoteService remoteService) {
this.subMapper = subMapper;
this.refreshService = refreshService;
this.planner = planner;
this.pushService = pushService;
this.remoteService = remoteService;
}
/**
* 轻量 tick:只处理已到期的账号,且单轮有数量上限。
*
* <p>用 fixedDelay 而非 fixedRate,一轮结束再计下一轮,避免上一轮没跑完就叠加触发;
* initialDelay 让重启后的第一轮稍晚开始,先让应用完成启动。
*/
@Scheduled(fixedDelayString = "${subscription.refresh.tick-interval-ms:300000}",
initialDelayString = "${subscription.refresh.initial-delay-ms:120000}")
public void tick() {
try {
refreshDueAccounts();
} catch (Exception e) {
// 调度方法抛出异常会导致后续触发被取消,这里必须兜住。
log.error("订阅分散刷新 tick 失败", e);
}
}
/**
* 刷新当前已到期的账号。
*
* @return 本轮实际刷新的账号数
*/
public int refreshDueAccounts() {
List<SubscriptionAccount> enabled = enabledAccounts();
ensureScheduled(enabled);
long now = planner.clock().millis();
int refreshed = 0;
for (SubscriptionAccount account : enabled) {
if (refreshed >= maxPerTick)
break;
if (!isDue(account, now))
continue;
refreshOne(account, now);
refreshed++;
}
reportStale(enabled, now);
// 缓存内容变了就要让备机同步。原先全量刷新后由调用方跟一次同步,
// 改为分散刷新后必须在这里补上,否则备机要等到下一次周期同步才拿到新内容。
if (refreshed > 0)
remoteService.requestSubscriptionSync();
return refreshed;
}
/**
* 手动刷新入口:立即刷新全部启用账号。
*
* <p>刻意不受 {@code maxPerTick} 限制——管理页「更新订阅」的语义就是立刻全部刷新。
* 但成功后同样要重排各自的下一次时刻,否则刚手动刷完会让当天的计划作废,
* 该账号反而可能超过 24 小时没有下次更新。
*/
public boolean refreshAllNow() {
boolean success = true;
long now = planner.clock().millis();
for (SubscriptionAccount account : enabledAccounts()) {
if (!refreshOne(account, now))
success = false;
}
return success;
}
private List<SubscriptionAccount> enabledAccounts() {
return subMapper.selectAllSubscriptionAccounts().stream()
.filter(SubscriptionAccount::isEnabled)
.sorted(Comparator.comparing(SubscriptionAccount::getId))
.toList();
}
/** 刷新单个账号并按结果重排下一次时刻。 */
private boolean refreshOne(SubscriptionAccount account, long now) {
boolean ok = refreshService.refresh(account.getId());
if (ok) {
long next = planner.nextAfterSuccess(planner.clock().millis());
subMapper.updateNextRefreshAt(account.getId(), next);
alertedAccounts.remove(account.getId());
log.info("订阅刷新成功 accountId={} 下次刷新={}", account.getId(), next);
} else {
long next = planner.nextAfterFailure(now, account.getLastSuccessEpoch());
subMapper.updateNextRefreshAt(account.getId(), next);
log.warn("订阅刷新失败 accountId={} 下次重试={}", account.getId(), next);
}
return ok;
}
/**
* 账号是否到期。
*
* <p>以排定时刻为准;尚未排程或计划明显损坏(超出窗口)时视为立即到期,
* 由 {@link #ensureScheduled} 先铺开或纠正。
*/
private boolean isDue(SubscriptionAccount account, long now) {
return planner.isDue(account.getNextRefreshAt(), now);
}
/**
* 为尚无计划的账号补齐计划。
*
* <p>按紧迫性分三类,因为它们对「排到多晚」的容忍度不同:
* <ul>
* <li><b>从未成功过</b>(新建账号、刚改过上游 key):可能还没有可用缓存,
* 用户此时请求订阅会拿不到内容,因此尽快刷,只按 tick 间隔互相错开;</li>
* <li><b>已成功过但已超期</b>(距上次成功已达一个窗口):保证即将失效,
* 必须立即刷新,不能排到窗口内更晚的位置;</li>
* <li><b>已成功过且未超期</b>:缓存仍可分发,按窗口分散即可,不必挤在一起。</li>
* </ul>
*/
private void ensureScheduled(List<SubscriptionAccount> enabled) {
long now = planner.clock().millis();
List<SubscriptionAccount> cold = new ArrayList<>();
List<SubscriptionAccount> overdue = new ArrayList<>();
List<SubscriptionAccount> warm = new ArrayList<>();
for (SubscriptionAccount account : enabled) {
if (account.getNextRefreshAt() != null)
continue;
if (account.getLastSuccessEpoch() == null)
cold.add(account);
else if (planner.isOverdue(account.getLastSuccessEpoch(), now))
overdue.add(account);
else
warm.add(account);
}
if (cold.isEmpty() && overdue.isEmpty() && warm.isEmpty())
return;
if (!planner.isCapacitySufficient(enabled.size()))
log.warn("{}", planner.capacityMessage(enabled.size()));
// 超期账号立即到期(计划设为当前时刻),本轮就会被刷新。
for (SubscriptionAccount account : overdue)
schedule(account, now);
int index = 0;
List<Long> coldPlan = planner.coldStartSchedule(
cold.stream().map(SubscriptionAccount::getId).toList(), planner.tickInterval());
for (SubscriptionAccount account : cold)
schedule(account, coldPlan.get(index++));
List<Integer> warmIds = warm.stream().map(SubscriptionAccount::getId).toList();
List<Long> warmPlan = planner.initialSchedule(warmIds);
for (int i = 0; i < warm.size(); i++)
schedule(warm.get(i), warmPlan.get(i));
}
/**
* 写入新计划,并同步更新内存中的对象。
*
* <p>必须同时更新内存副本:否则刚被排到几小时后的账号在本轮仍带着
* {@code nextRefreshAt == null},会被 {@link #isDue} 判为立即到期并当场刷新,
* 于是「铺开」在第一个 tick 就失效了(迁移后首批账号会全部集中刷新)。
*/
private void schedule(SubscriptionAccount account, long nextRefreshAt) {
subMapper.updateNextRefreshAt(account.getId(), nextRefreshAt);
account.setNextRefreshAt(nextRefreshAt);
}
/** 陈旧账号告警:同一账号在连续陈旧期间只推一次,刷新成功后重置。 */
private void reportStale(List<SubscriptionAccount> enabled, long now) {
long threshold = (long) (planner.window().toMillis() * staleAlertMultiplier);
for (SubscriptionAccount account : enabled) {
Long lastSuccess = account.getLastSuccessEpoch();
if (lastSuccess == null || now - lastSuccess < threshold)
continue;
if (!alertedAccounts.add(account.getId()))
continue;
long hours = (now - lastSuccess) / 3_600_000L;
String message = String.format("订阅账号 %d(%s)已 %d 小时未成功刷新,最近错误:%s",
account.getId(), account.getName(), hours,
account.getLastError() == null ? "无" : account.getLastError());
log.error("{}", message);
pushService.sendToMe(message);
}
// 已删除或停用的账号不再保留告警状态,避免集合无限增长。
Set<Integer> alive = new HashSet<>();
for (SubscriptionAccount account : enabled)
alive.add(account.getId());
alertedAccounts.retainAll(alive);
}
SubscriptionRefreshPlanner planner() {
return planner;
}
}
@@ -2,6 +2,9 @@ package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.SubMapper; import com.lion.lionwebsite.Dao.normal.SubMapper;
import com.lion.lionwebsite.Domain.SubscriptionAccount; import com.lion.lionwebsite.Domain.SubscriptionAccount;
import com.lion.lionwebsite.Util.ClashSubscriptionParser;
import com.lion.lionwebsite.Util.SubscriptionClientProfile;
import com.lion.lionwebsite.Util.SubscriptionCacheStore;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.hc.client5.http.classic.methods.HttpGet; import org.apache.hc.client5.http.classic.methods.HttpGet;
@@ -18,6 +21,7 @@ import java.io.*;
import java.net.URLDecoder; import java.net.URLDecoder;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.nio.file.*; import java.nio.file.*;
import java.time.Clock;
import java.util.*; import java.util.*;
import java.util.regex.Matcher; import java.util.regex.Matcher;
import java.util.regex.Pattern; import java.util.regex.Pattern;
@@ -52,14 +56,14 @@ public class SubscriptionRefreshService {
@Value("${subscription.cache-root:sub/accounts}") @Value("${subscription.cache-root:sub/accounts}")
String cacheRoot; String cacheRoot;
public boolean refreshAll() { /** 时间源,测试可替换以固定成功时刻。 */
boolean success = true; Clock clock = Clock.systemDefaultZone();
for (SubscriptionAccount account : subMapper.selectAllSubscriptionAccounts()) {
if (account.isEnabled() && !refresh(account.getId())) /**
success = false; * 分代缓存存储。默认按 {@link #cacheRoot} 懒构造;测试可覆写为受控子类来模拟写入失败。
} * 保持为普通字段而不是构造参数,是为了不改动既有两参构造器(Spring 与老测试都在用)。
return success; */
} SubscriptionCacheStore cacheStore;
private long refreshSequence; private long refreshSequence;
// Accessed only while holding the coordinator write lock. // Accessed only while holding the coordinator write lock.
@@ -82,17 +86,20 @@ public class SubscriptionRefreshService {
// Network access and parsing never hold the shared subscription lock. // Network access and parsing never hold the shared subscription lock.
try { try {
String v2 = processV2(firstLine(download(v2Url(account))), account.isFilterHighMultiplier(), highMultiplierThreshold); boolean filter = account.isFilterHighMultiplier();
List<String> clash = processClash(download(clashUrl(account)), account.isFilterHighMultiplier(), highMultiplierThreshold); String v2 = processV2(firstLine(download(v2Url(account), SubscriptionClientProfile.forAccount(accountId, false))),
filter, highMultiplierThreshold);
// 按 YAML 语义过滤高倍节点并同步清理 proxy-groups 引用,不再靠行缩进猜测。
String clash = ClashSubscriptionParser.transform(
download(clashUrl(account), SubscriptionClientProfile.forAccount(accountId, true)),
filter, highMultiplierThreshold);
stateLock.lock(); stateLock.lock();
try { try {
if (!isCurrent(account, version)) if (!isCurrent(account, version))
return false; return false;
Path dir = Paths.get(cacheRoot, String.valueOf(accountId)); // 两份内容写进同一分代,校验通过后指针最后发布:任一步失败都保留上次完整版本。
Files.createDirectories(dir); store().publish(accountId, v2.getBytes(StandardCharsets.UTF_8), clash.getBytes(StandardCharsets.UTF_8));
atomicWrite(dir.resolve("v2ray.txt"), v2.getBytes(StandardCharsets.UTF_8)); subMapper.markSubscriptionRefreshSuccess(accountId, clock.millis());
atomicWrite(dir.resolve("clash.yaml"), String.join("\n", clash).concat("\n").getBytes(StandardCharsets.UTF_8));
subMapper.markSubscriptionRefreshSuccess(accountId);
return true; return true;
} finally { } finally {
stateLock.unlock(); stateLock.unlock();
@@ -164,63 +171,29 @@ public class SubscriptionRefreshService {
return URLDecoder.decode(node.substring(hash + 1), StandardCharsets.UTF_8); return URLDecoder.decode(node.substring(hash + 1), StandardCharsets.UTF_8);
} }
private List<String> processClash(List<String> source, boolean filter, double threshold) {
Set<String> removed = new HashSet<>();
List<String> result = new ArrayList<>();
boolean inProxies = false;
boolean skipNode = false;
for (String line : source) {
if (line.equals("proxies:")) {
inProxies = true;
skipNode = false;
result.add(line);
continue;
}
if (line.equals("proxy-groups:")) {
inProxies = false;
skipNode = false;
result.add(line);
continue;
}
if (inProxies && line.matches("^\\s{2}-\\s+name:.*")) {
String name = clashName(line);
skipNode = filter && isHigh(name, threshold);
if (skipNode)
removed.add(name);
else
result.add(line);
continue;
}
if (skipNode)
continue;
if (!inProxies && !removed.isEmpty() && line.trim().startsWith("- ")) {
String ref = line.trim().substring(2).trim();
if (removed.contains(unquote(ref)))
continue;
}
result.add(line);
}
return result;
}
private static String clashName(String line) {
int index = line.indexOf("name:");
return unquote(line.substring(index + 5).trim());
}
private static String unquote(String value) {
if (value.length() >= 2 && ((value.startsWith("\"") && value.endsWith("\"")) || (value.startsWith("'") && value.endsWith("'"))))
return value.substring(1, value.length() - 1);
return value;
}
private static boolean isHigh(String name, double threshold) { private static boolean isHigh(String name, double threshold) {
Matcher matcher = MULTIPLIER.matcher(name); Matcher matcher = MULTIPLIER.matcher(name);
return matcher.find() && Double.parseDouble(matcher.group(1)) > threshold; return matcher.find() && Double.parseDouble(matcher.group(1)) > threshold;
} }
List<String> download(String url) throws IOException { /**
* 下载上游订阅正文。
*
* @param profile 要伪装的客户端身份;为 null 时保留 HttpClient 默认头(仅测试使用)。
* 生产路径必须传入,否则上游会看到 {@code Apache-HttpClient/... (Java/...)}。
*/
List<String> download(String url, SubscriptionClientProfile profile) throws IOException {
HttpGet get = new HttpGet(url); HttpGet get = new HttpGet(url);
if (profile != null) {
get.addHeader("User-Agent", profile.userAgent());
get.addHeader("Accept", profile.accept());
get.addHeader("Accept-Language", profile.acceptLanguage());
// 实测:显式设置 Accept-Encoding 不会破坏 HttpClient5 的透明解压,
// ContentCompressionExec 仍按 Content-Encoding 正确解码。
get.addHeader("Accept-Encoding", "gzip, deflate");
// 订阅客户端是长连接复用,且不会发送 Referer。
get.addHeader("Connection", "keep-alive");
}
try (CloseableHttpResponse response = HTTP_CLIENT.execute(get)) { try (CloseableHttpResponse response = HTTP_CLIENT.execute(get)) {
if (response.getCode() != 200) if (response.getCode() != 200)
throw new IOException("上游 HTTP 状态码 " + response.getCode()); throw new IOException("上游 HTTP 状态码 " + response.getCode());
@@ -237,22 +210,17 @@ public class SubscriptionRefreshService {
} }
} }
private static void atomicWrite(Path target, byte[] data) throws IOException { /**
Path temp = target.resolveSibling(target.getFileName() + ".tmp"); * 当前应分发的缓存文件。只在一整个版本有效时返回真实路径;否则返回一个保证不存在的
Files.write(temp, data); * 哨兵路径,使分发端按「订阅暂不可用」处理,而不是发出半份旧缓存。
try { */
Files.move(temp, target, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
} catch (AtomicMoveNotSupportedException e) {
Files.move(temp, target, StandardCopyOption.REPLACE_EXISTING);
}
}
public Path cachedPath(Integer accountId, String client) { public Path cachedPath(Integer accountId, String client) {
return Paths.get(cacheRoot, String.valueOf(accountId), client.equals("v2") ? "v2ray.txt" : "clash.yaml"); SubscriptionCacheStore store = store();
return store.resolve(accountId, client).orElseGet(() -> store.unavailablePath(accountId, client));
} }
public boolean hasCompleteCache(Integer accountId) { public boolean hasCompleteCache(Integer accountId) {
return Files.isRegularFile(cachedPath(accountId, "v2")) && Files.isRegularFile(cachedPath(accountId, "cat")); return store().hasComplete(accountId);
} }
public void invalidateCache(Integer accountId) { public void invalidateCache(Integer accountId) {
@@ -260,12 +228,15 @@ public class SubscriptionRefreshService {
stateLock.lock(); stateLock.lock();
try { try {
latestRefresh.remove(accountId); // In-flight responses must not restore invalidated content. latestRefresh.remove(accountId); // In-flight responses must not restore invalidated content.
Files.deleteIfExists(cachedPath(accountId, "v2")); store().invalidate(accountId);
Files.deleteIfExists(cachedPath(accountId, "cat"));
} catch (IOException e) {
log.warn("清理失效订阅缓存失败 accountId={}", accountId, e);
} finally { } finally {
stateLock.unlock(); stateLock.unlock();
} }
} }
private synchronized SubscriptionCacheStore store() {
if (cacheStore == null)
cacheStore = new SubscriptionCacheStore(cacheRoot);
return cacheStore;
}
} }
@@ -6,6 +6,7 @@ import com.lion.lionwebsite.Domain.SubBind;
import com.lion.lionwebsite.Domain.SubscriptionAccount; import com.lion.lionwebsite.Domain.SubscriptionAccount;
import com.lion.lionwebsite.Message.*; import com.lion.lionwebsite.Message.*;
import com.lion.lionwebsite.Util.CustomUtil; import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.SubscriptionCacheStore;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -16,8 +17,6 @@ import javax.crypto.spec.SecretKeySpec;
import java.io.ByteArrayOutputStream; import java.io.ByteArrayOutputStream;
import java.io.IOException; import java.io.IOException;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest; import java.security.MessageDigest;
import java.util.*; import java.util.*;
import java.util.zip.GZIPOutputStream; import java.util.zip.GZIPOutputStream;
@@ -39,6 +38,12 @@ public class SubscriptionStandbySnapshotService {
@Value("${subscription.cache-root:sub/accounts}") @Value("${subscription.cache-root:sub/accounts}")
String cacheRoot; String cacheRoot;
/**
* 与 {@link SubscriptionRefreshService} 共用的分代缓存解析器。
* 测试可直接覆写;未设置时按 {@link #cacheRoot} 懒构造。
*/
SubscriptionCacheStore cacheStore;
@PostConstruct @PostConstruct
void validateConfiguration() { void validateConfiguration() {
if (syncEnabled && (syncSecret == null || syncSecret.isBlank())) if (syncEnabled && (syncSecret == null || syncSecret.isBlank()))
@@ -51,12 +56,14 @@ public class SubscriptionStandbySnapshotService {
try { try {
Map<Integer, SubscriptionAccountSnapshot> accountMap = new HashMap<>(); Map<Integer, SubscriptionAccountSnapshot> accountMap = new HashMap<>();
for (SubscriptionAccount account : subMapper.selectAllSubscriptionAccounts()) { for (SubscriptionAccount account : subMapper.selectAllSubscriptionAccounts()) {
Path v2Path = cachedPath(account.getId(), "v2"); if (!account.isEnabled())
Path clashPath = cachedPath(account.getId(), "cat");
if (!account.isEnabled() || !Files.isRegularFile(v2Path) || !Files.isRegularFile(clashPath))
continue; continue;
byte[] v2 = Files.readAllBytes(v2Path); // 从同一分代读出成对内容,保证快照里两份文件同版本;无完整版本则跳过该账号。
byte[] clash = Files.readAllBytes(clashPath); Optional<SubscriptionCacheStore.Pair> pair = store().read(account.getId());
if (pair.isEmpty())
continue;
byte[] v2 = pair.get().v2();
byte[] clash = pair.get().clash();
SubscriptionAccountSnapshot snapshot = new SubscriptionAccountSnapshot(); SubscriptionAccountSnapshot snapshot = new SubscriptionAccountSnapshot();
snapshot.setAccountId(account.getId()); snapshot.setAccountId(account.getId());
snapshot.setEnabled(true); snapshot.setEnabled(true);
@@ -132,7 +139,9 @@ public class SubscriptionStandbySnapshotService {
private static String hex(byte[] input) { return HexFormat.of().formatHex(input); } private static String hex(byte[] input) { return HexFormat.of().formatHex(input); }
private Path cachedPath(Integer accountId, String client) { private synchronized SubscriptionCacheStore store() {
return Path.of(cacheRoot, String.valueOf(accountId), "v2".equals(client) ? "v2ray.txt" : "clash.yaml"); if (cacheStore == null)
cacheStore = new SubscriptionCacheStore(cacheRoot);
return cacheStore;
} }
} }
@@ -0,0 +1,23 @@
package com.lion.lionwebsite.Service;
import java.util.UUID;
/** 协议 v2 的操作 ID 生成与解析。稳定 ID 是「节点重发/重试不重复执行副作用」的依据。 */
public final class TaskProtocol {
private TaskProtocol() {
}
/** 生成一个新的稳定操作 ID(下载或删除),形如 {@code op-<uuid>}。 */
public static String newOperationId(String action) {
return action + "-" + UUID.randomUUID();
}
public static String newDownloadOperationId() {
return newOperationId("dl");
}
public static String newDeleteOperationId() {
return newOperationId("del");
}
}
@@ -0,0 +1,37 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Domain.Gallery;
import com.lion.lionwebsite.Domain.GalleryTask;
import com.lion.lionwebsite.Message.TaskReportMessage;
import java.util.List;
/**
* 协议 v2 状态上报的消费入口。
*
* <p>定义成接口是为了断开依赖环:RemoteService 需要把上报交给生命周期服务,
* 而生命周期服务需要有 RemoteService 才能下发命令。RemoteService 通过可选的 setter
* 注入本接口(见 {@code RemoteService#setTaskReportHandler}),因此构造期没有环。
*/
public interface TaskReportHandler {
/**
* 提交一份上报。
*
* @return 已成功提交(可回 ACK)时返回提交结果;数据库失败等「不可回 ACK」的情况返回 null。
* 返回 null 时节点会保留快照并稍后重投,这正是「失败写入不得清任务」的要求。
*/
CommittedReport apply(TaskReportMessage report);
/**
* 已提交的结果。
*
* @param reportId 本次上报 ID(回执用)
* @param newlyCompleted 本次「首次」完成的任务,通知与 WebSocket 由独立执行器在事务之后处理,
* 因此慢通知不会阻塞数据库消费线程;重复上报不会重复出现在这里
* @param acceptedTasks 事务内实际被接受的进度更新(含下载中/压缩中等中间状态,已排除过期、
* 旧世代与待删除任务),WebSocket 只推这一批,避免前端看到被丢弃的迟到状态
*/
record CommittedReport(String reportId, List<Gallery> newlyCompleted, List<GalleryTask> acceptedTasks) {
}
}
@@ -0,0 +1,45 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Domain.GalleryTask;
/**
* 任务业务状态码到展示文案的唯一映射点。
*
* <p>状态以稳定代码表达,中文文案只出现在展示层(WebSocket/列表),
* 避免把「失败/待重试/结果未知」塞进成功失败提示字符串。
*/
public final class TaskStatus {
public static final String SUBMITTED = "已提交";
/** 已登记创建意图、尚未确认上游提交;用于让任务列表在提交期间也能看到占位任务。 */
public static final String PENDING_SUBMIT = "提交中";
public static final String DOWNLOADING = "下载中";
public static final String WAITING_COMPRESS = "等待压缩";
public static final String COMPRESSING = "压缩中";
public static final String COMPLETE = "下载完成";
public static final String UNKNOWN = "提交未知";
public static final String DELETE_PENDING = "待删除";
public static final String DELETE_FAILED = "删除失败";
private TaskStatus() {
}
/**
* 节点上报的进度状态映射为业务文案;未知状态码返回 null,由调用方忽略。
*
* <p>GalleryTask 里的状态码是可变 static byte,不是编译期常量,因此不能用 switch case;
* 这里用整数比较,避免为改常量引入风险。
*/
public static String fromReport(byte reportStatus) {
int status = reportStatus;
if (status == GalleryTask.DOWNLOADING)
return DOWNLOADING;
if (status == GalleryTask.DOWNLOAD_COMPLETE)
return WAITING_COMPRESS;
if (status == GalleryTask.COMPRESSING)
return COMPRESSING;
if (status == GalleryTask.COMPRESS_COMPLETE)
return COMPLETE;
return null;
}
}
@@ -0,0 +1,47 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Domain.Gallery;
import com.lion.lionwebsite.Exception.ResolutionNotMatchException;
import com.lion.lionwebsite.Util.GalleryUtil;
import com.lion.lionwebsite.Util.PreparedGallery;
import org.springframework.stereotype.Component;
import java.io.IOException;
/**
* 上游图站的读写边界。
*
* <p>分成两步是协议 v2 的核心约束:
* <ol>
* <li>{@link #prepare} 只做读取(页面、分辨率),绝不产生下载副作用;</li>
* <li>调用方必须在持久化 SUBMITTING 之后才调用 {@link #submit},因为那一步才真正 POST。</li>
* </ol>
*
* <p>拆成接口是为了让生命周期服务可以在测试里注入假实现,也方便后续把抓取/解析/提交
* 进一步分开;默认实现 {@link GalleryUtilAdapter} 直接转发到现有 GalleryUtil。
*/
public interface UpstreamGateway {
/** 只读解析:失败返回 null 表示链接无效或页面不可用,不产生任何副作用。 */
PreparedGallery prepare(String link) throws IOException;
/** 真正提交下载请求。抛异常表示结果未知,由调用方落 UNKNOWN,不做自动重发。 */
Gallery submit(PreparedGallery prepared, String targetResolution)
throws IOException, ResolutionNotMatchException;
/** 默认实现:转发现有 GalleryUtil 的拆分方法。 */
@Component
class GalleryUtilAdapter implements UpstreamGateway {
@Override
public PreparedGallery prepare(String link) throws IOException {
return GalleryUtil.prepare(link);
}
@Override
public Gallery submit(PreparedGallery prepared, String targetResolution)
throws IOException, ResolutionNotMatchException {
return GalleryUtil.submit(prepared, targetResolution);
}
}
}
@@ -3,16 +3,23 @@ package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CollectMapper; import com.lion.lionwebsite.Dao.normal.CollectMapper;
import com.lion.lionwebsite.Dao.normal.GalleryMapper; import com.lion.lionwebsite.Dao.normal.GalleryMapper;
import com.lion.lionwebsite.Dao.normal.UserMapper; import com.lion.lionwebsite.Dao.normal.UserMapper;
import com.lion.lionwebsite.Dao.normal.SubMapper;
import com.lion.lionwebsite.Domain.User; import com.lion.lionwebsite.Domain.User;
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor; import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
import com.lion.lionwebsite.Util.CustomUtil; import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.Response; import com.lion.lionwebsite.Util.Response;
import tools.jackson.databind.ObjectMapper;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.transaction.support.TransactionTemplate;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.concurrent.locks.Lock;
import java.util.function.Supplier;
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
@Service @Service
@Slf4j @Slf4j
@@ -27,6 +34,11 @@ public class UserService{
final TaskHandlerInterceptor taskHandlerInterceptor; final TaskHandlerInterceptor taskHandlerInterceptor;
@Autowired SubMapper subMapper;
@Autowired SubscriptionStateCoordinator subscriptionState;
@Autowired RemoteService remoteService;
@Autowired @Qualifier("mainTransactionTemplate") TransactionTemplate transactions;
public String addAuthCode(String targetAuthCode, String people) { public String addAuthCode(String targetAuthCode, String people) {
Response response = Response.generateResponse(); Response response = Response.generateResponse();
@@ -65,12 +77,22 @@ public class UserService{
Response response = Response.generateResponse(); Response response = Response.generateResponse();
try{ try{
if(userMapper.isExist(targetAuthCode) != 0){ return coordinated(() -> {
userMapper.updateUsername(targetAuthCode, newUsername); if (userMapper.isExist(targetAuthCode) == 0)
response.success("修改成功"); return response.failure("授权码不存在").toJSONString();
}else { if (newUsername == null || newUsername.isBlank())
response.failure("授权码不存在"); return response.failure("用户名不能为空").toJSONString();
} String name = newUsername.trim();
User current = userMapper.selectUserByAuthCode(targetAuthCode);
User existing = userMapper.selectUserByUsername(name);
if (existing != null && (current == null || existing.getId() != current.getId()))
return response.failure("用户名已存在").toJSONString();
userMapper.updateUsername(targetAuthCode, name);
if (subMapper != null && current != null)
subMapper.renameBoundUser(current.getId(), current.getUsername(), name);
if (remoteService != null) remoteService.requestSubscriptionSync();
return response.success("修改成功").toJSONString();
});
}catch (Exception e){ }catch (Exception e){
log.error("修改用户名失败", e); log.error("修改用户名失败", e);
response.failure("修改失败"); response.failure("修改失败");
@@ -84,17 +106,22 @@ public class UserService{
if(userMapper.isExist(targetAuthCode) != 0) if(userMapper.isExist(targetAuthCode) != 0)
try{ try{
coordinated(() -> {
User user = userMapper.selectUserByAuthCode(targetAuthCode); User user = userMapper.selectUserByAuthCode(targetAuthCode);
//修改下载人以及取消收藏
ArrayList<Integer> galleries = collectMapper.selectGidByCollector(user.getId()); ArrayList<Integer> galleries = collectMapper.selectGidByCollector(user.getId());
for(Integer gallery: galleries) for(Integer gallery: galleries)
collectMapper.disCollect(gallery, user.getId()); collectMapper.disCollect(gallery, user.getId());
galleryMapper.updateGalleryDownloader(3, user.getId()); galleryMapper.updateGalleryDownloader(ADMIN_USER_ID, user.getId());
if (subMapper != null) {
subMapper.deleteSubUpdateRecord(user.getUsername());
subMapper.deleteSubBind(user.getUsername());
}
userMapper.deleteUserByAuthCode(targetAuthCode); userMapper.deleteUserByAuthCode(targetAuthCode);
return "deleted";
});
response.success("删除成功"); response.success("删除成功");
taskHandlerInterceptor.updateAuthCodes(); taskHandlerInterceptor.updateAuthCodes();
if (remoteService != null) remoteService.requestSubscriptionSync();
}catch (Exception e){ }catch (Exception e){
log.error("删除授权码失败", e); log.error("删除授权码失败", e);
response.failure("删除失败"); response.failure("删除失败");
@@ -107,27 +134,29 @@ public class UserService{
} }
public String alterStatus(String AuthCode, boolean isEnable){ public String alterStatus(String AuthCode, boolean isEnable){
Response response = Response.generateResponse(); try {
String result = coordinated(() -> {
if(userMapper.isExist(AuthCode) == 1){ if (userMapper.isExist(AuthCode) != 1)
return Response._failure("该用户不存在");
int id = userMapper.selectUserByAuthCode(AuthCode).getId(); int id = userMapper.selectUserByAuthCode(AuthCode).getId();
userMapper.updateIsEnableById(id, isEnable); userMapper.updateIsEnableById(id, isEnable);
galleryMapper.updateGalleryDownloader(3, id); if (!isEnable) galleryMapper.updateGalleryDownloader(ADMIN_USER_ID, id);
response.success("修改用户状态成功"); return Response._success("修改用户状态成功");
taskHandlerInterceptor.updateAuthCodes(); });
// Refresh the in-memory identity view only after the database commits.
if (result.contains("\"result\":\"success\"")) taskHandlerInterceptor.updateAuthCodes();
return result;
} catch (RuntimeException error) {
log.warn("修改用户状态失败: {}", error.getClass().getSimpleName());
return Response._failure("修改用户状态失败");
} }
else{
response.failure("该用户不存在");
}
return response.toJSONString();
} }
public String getAllUser() { public String getAllUser() {
Response response = Response.generateResponse(); Response response = Response.generateResponse();
User[] users = userMapper.selectAllUser(); User[] users = userMapper.selectAllUser();
response.success(new ObjectMapper().valueToTree(users).toString()); response.success(objectMapper.valueToTree(users));
return response.toJSONString(); return response.toJSONString();
} }
@@ -135,4 +164,28 @@ public class UserService{
public int getUserId(String AuthCode){ public int getUserId(String AuthCode){
return userMapper.selectUserByAuthCode(AuthCode).getId(); return userMapper.selectUserByAuthCode(AuthCode).getId();
} }
/**
* 管理员判定。项目没有角色表,历史上以 id=3(狮子)作为管理员,
* 前端也是用 userId === 3 判断;这里把这个约定收敛到一处,
* 权限相关的服务端校验统一走它。
*/
public static final int ADMIN_USER_ID = 3;
public boolean isAdmin(String AuthCode){
if(AuthCode == null)
return false;
User user = userMapper.selectUserByAuthCode(AuthCode);
return user != null && user.getId() == ADMIN_USER_ID;
}
private String coordinated(Supplier<String> action) {
Lock lock = subscriptionState == null ? null : subscriptionState.writeLock();
if (lock != null) lock.lock();
try {
return transactions == null ? action.get() : transactions.execute(ignored -> action.get());
} finally {
if (lock != null) lock.unlock();
}
}
} }
@@ -0,0 +1,45 @@
package com.lion.lionwebsite.Util;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.DumperOptions;
import java.util.*;
import java.util.regex.Pattern;
/** YAML semantics rather than indentation determine nodes and references. */
public final class ClashSubscriptionParser {
private static final Pattern MULTIPLIER = Pattern.compile("(\\d+(?:\\.\\d+)?)x\\s*$", Pattern.CASE_INSENSITIVE);
private ClashSubscriptionParser() { }
public static String transform(List<String> lines, boolean filter, double threshold) {
Object loaded = new Yaml().load(String.join("\n", lines));
if (!(loaded instanceof Map<?, ?> source)) throw new IllegalArgumentException("Clash 上游必须为 YAML 对象");
if (!(source.get("proxies") instanceof List<?> proxies)) throw new IllegalArgumentException("Clash 上游缺少 proxies 列表");
Map<Object, Object> output = new LinkedHashMap<>(source);
Set<String> removed = new HashSet<>();
List<Object> kept = new ArrayList<>();
for (Object node : proxies) {
if (!(node instanceof Map<?, ?> fields) || !(fields.get("name") instanceof String name))
throw new IllegalArgumentException("Clash 节点缺少名称");
var multiplier = MULTIPLIER.matcher(name);
if (filter && multiplier.find() && Double.parseDouble(multiplier.group(1)) > threshold)
removed.add(name);
else kept.add(node);
}
output.put("proxies", kept);
if (source.get("proxy-groups") instanceof List<?> groups) {
List<Object> result = new ArrayList<>();
for (Object group : groups) {
if (!(group instanceof Map<?, ?> fields)) throw new IllegalArgumentException("Clash 分组必须为对象");
Map<Object, Object> copy = new LinkedHashMap<>(fields);
if (fields.get("proxies") instanceof List<?> references)
copy.put("proxies", references.stream().filter(ref -> !removed.contains(ref)).toList());
result.add(copy);
}
output.put("proxy-groups", result);
}
DumperOptions options = new DumperOptions();
options.setDefaultFlowStyle(DumperOptions.FlowStyle.BLOCK);
options.setPrettyFlow(true);
return new Yaml(options).dump(output);
}
}
@@ -59,7 +59,9 @@ public class FileDownload {
if ("HEAD".equalsIgnoreCase(request.getMethod())) if ("HEAD".equalsIgnoreCase(request.getMethod()))
return; return;
input.seek(start); input.seek(start);
BufferedOutputStream output = new BufferedOutputStream(response.getOutputStream()); // 关闭响应流:Tomcat 为大响应体创建的临时文件会在流关闭时删除,
// 不关闭会一直堆积到 Full GC。flushBuffer 只能把内容送出去,不能触发清理。
try (BufferedOutputStream output = new BufferedOutputStream(response.getOutputStream())) {
byte[] buffer = new byte[8192]; byte[] buffer = new byte[8192];
while (remaining > 0) { while (remaining > 0) {
int count = input.read(buffer, 0, (int) Math.min(buffer.length, remaining)); int count = input.read(buffer, 0, (int) Math.min(buffer.length, remaining));
@@ -70,6 +72,7 @@ public class FileDownload {
} }
output.flush(); output.flush();
response.flushBuffer(); response.flushBuffer();
}
} catch (ClientAbortException e) { } catch (ClientAbortException e) {
// The client cancelled its download. // The client cancelled its download.
} catch (IOException e) { } catch (IOException e) {
@@ -0,0 +1,97 @@
package com.lion.lionwebsite.Util;
import com.lion.lionwebsite.Domain.Gallery;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import java.io.IOException;
import java.util.*;
import java.util.regex.Pattern;
/** Pure parsing: upstream requests and download submission live outside this class. */
public final class GalleryPageParser {
private GalleryPageParser() { }
/** 归档页 #db 单元格开头:分辨率标签 + 正数体积 + 受支持单位;后缀费用文本(Free、50GP)忽略。 */
private static final Pattern RESOLUTION_ENTRY =
Pattern.compile("^(Original|(\\d+)x)\\s+(\\d+(?:\\.\\d+)?)\\s*([A-Za-z]+)(?=\\s|$)", Pattern.CASE_INSENSITIVE);
public static Gallery metadata(Document document, String link, int gid) throws IOException {
String title = document.select("#gn").text();
if (title.isBlank()) title = document.select("#gj").text();
if (title.isBlank()) throw new IOException("上游页面缺少画廊名称,可能是错误页");
Map<String, String> fields = new HashMap<>();
for (Element row : document.select("#gdd tr")) {
var cells = row.select("td");
if (cells.size() < 2) continue;
String name = cells.get(0).text().replace(":", "").strip().toLowerCase(Locale.ROOT);
fields.put(name, cells.get(1).text().strip());
}
var pages = Pattern.compile("\\d+").matcher(fields.getOrDefault("length", ""));
if (!pages.find()) throw new IOException("上游页面缺少画廊页数");
Gallery result = new Gallery();
result.setGid(gid); result.setLink(link); result.setName(title + " [" + gid + "]");
result.setCreateTime(System.currentTimeMillis() / 1000); result.setProceeding(0);
result.setLanguage(fields.getOrDefault("language", "").replace("\u00a0", " "));
try {
result.setPages(Integer.parseInt(pages.group()));
result.setFileSize(CustomUtil.stringToFileSize(fields.getOrDefault("file size", "").replace(" ", "")));
} catch (RuntimeException failure) { throw new IOException("上游画廊页数或文件大小格式不支持", failure); }
if (result.getPages() <= 0) throw new IOException("上游画廊页数必须为正数");
result.setDisplayFileSize(CustomUtil.fileSizeToString(result.getFileSize()));
Element thumbnail = document.selectFirst("#gd1 [style*=url]");
var image = Pattern.compile("url\\(\\s*['\"]?(.*?)['\"]?\\s*\\)").matcher(thumbnail == null ? "" : thumbnail.attr("style"));
String imageUrl = image.find() ? image.group(1) : document.select("#gd1 img").attr("src");
result.setThumb_link(imageUrl.replace("https://s.exhentai.org", ""));
return result;
}
public static String archiveLink(Document document) throws IOException {
for (Element anchor : document.select("#gd5 a")) {
var matcher = Pattern.compile("https?://[^'\"\\s]+").matcher(anchor.attr("onclick"));
if (matcher.find()) return matcher.group();
String href = anchor.absUrl("href");
if (!href.isBlank() && (anchor.text().toLowerCase(Locale.ROOT).contains("download") || href.contains("archive")))
return href;
}
throw new IOException("上游页面缺少归档入口");
}
/**
* 解析归档页 #db 的可选分辨率。每个单元格合并后的文本形如 {@code Original 36.31 MiB Free}、
* {@code 800x 5.65 MiB Free}、{@code 2560x N/A N/A},单段布局形如 {@code 1280x 12.5 MiB},
* 都由同一条锚定规则覆盖;只看开头(分辨率 + 正数体积 + 单位),故 N/A 与费用文本不匹配,
* 配对也不会跨 {@code <td>} 借用相邻格。体积须能被 {@link CustomUtil#stringToFileSize(String)}
* 解析为正数,宽度须为正且不溢出。结果按宽度升序、{@code Original} 最后;无可用项抛 IOException。
*/
public static Map<String, String> resolutions(Document document) throws IOException {
TreeMap<Integer, String> sizes = new TreeMap<>();
String original = null;
for (Element cell : document.select("#db td")) {
var matcher = RESOLUTION_ENTRY.matcher(cell.text().replace('\u00a0', ' ').strip());
if (!matcher.find()) continue;
String size = matcher.group(3) + matcher.group(4);
try {
if (CustomUtil.stringToFileSize(size) <= 0) continue;
} catch (RuntimeException unsupported) {
continue;
}
if (matcher.group(1).equalsIgnoreCase("Original")) {
original = size;
continue;
}
int width;
try {
width = Integer.parseInt(matcher.group(2));
} catch (NumberFormatException outOfRange) {
continue;
}
if (width <= 0) continue;
sizes.put(width, size);
}
Map<String, String> result = new LinkedHashMap<>();
sizes.forEach((width, size) -> result.put(width + "x", size));
if (original != null) result.put("Original", original);
if (result.isEmpty()) throw new IOException("上游页面没有可用分辨率");
return result;
}
}
@@ -0,0 +1,74 @@
package com.lion.lionwebsite.Util;
import com.lion.lionwebsite.Domain.GalleryForQuery;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;
/** Search entries support table and thumbnail layouts; metadata excludes title/tag text. */
public final class GallerySearchParser {
private GallerySearchParser() { }
public static List<GalleryForQuery> parse(Document document) throws IOException {
List<GalleryForQuery> result = new ArrayList<>();
for (Element row : document.select("tr, .gl1t")) {
if (row.tagName().equals("tr") && !row.select(".gl1t").isEmpty()) continue;
Element link = row.selectFirst("a[href*=/g/]");
if (link == null) continue;
String href = link.attr("href");
Integer gid = GalleryUtil.parseGid(href);
if (gid == null) continue;
GalleryForQuery gallery = new GalleryForQuery();
gallery.setGid(gid.toString()); gallery.setLink(href);
Element title = row.selectFirst(".glink");
String name = title == null ? "" : title.text();
if (name.isBlank()) {
for (Element candidate : row.select("a[href*=/g/]")) {
if (!candidate.text().isBlank()) {
name = candidate.text();
break;
}
}
}
gallery.setName(name);
Element type = row.selectFirst(".cn, .cs");
Element firstCell = row.selectFirst("td");
gallery.setType(type != null ? type.text() : firstCell == null ? "" : firstCell.text());
Element image = row.selectFirst("img");
String thumbnail = image == null ? "" : image.attr("src");
if (thumbnail.isBlank() || thumbnail.startsWith("data:")) thumbnail = image == null ? "" : image.attr("data-src");
gallery.setThumbnailUrl(thumbnail.replace("https://s.exhentai.org", ""));
gallery.setPage(pageCount(row, gid));
Element date = row.selectFirst("[id^=posted_]");
if (date != null) gallery.setUploadTime(date.text());
else {
var dates = Pattern.compile("\\d{4}-\\d{2}-\\d{2}(?:\\s+\\d{2}:\\d{2})?").matcher(row.text());
gallery.setUploadTime(dates.find() ? dates.group() : "");
}
if (gallery.getName().isBlank()) throw new IOException("上游搜索结果缺少名称 gid=" + gid);
result.add(gallery);
}
return result;
}
private static int pageCount(Element row, int gid) throws IOException {
Element metadata = row.clone();
// Extended rows can put title and metadata in the same cell: remove only
// the title links/tags, retaining their containing cells and page counts.
metadata.select("a[href*=/g/], .glink, .gt, .gtl, .gtd, .glthumb").remove();
var expression = Pattern.compile("^(\\d+)\\s+pages?$", Pattern.CASE_INSENSITIVE);
for (Element element : metadata.getAllElements()) {
var pages = expression.matcher(element.text().strip());
if (!pages.matches()) continue;
try {
int count = Integer.parseInt(pages.group(1));
if (count > 0) return count;
} catch (NumberFormatException invalid) {
throw new IOException("上游搜索结果页数格式不支持 gid=" + gid, invalid);
}
}
throw new IOException("上游搜索结果缺少页数 gid=" + gid);
}
}
@@ -29,7 +29,6 @@ import java.io.File;
import java.io.IOException; import java.io.IOException;
import java.io.InputStreamReader; import java.io.InputStreamReader;
import java.util.*; import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicInteger;
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper; import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
@@ -45,7 +44,17 @@ public class GalleryUtil {
static String JSON = "json"; static String JSON = "json";
static ConcurrentHashMap<String, String> gid2MpvKey = new ConcurrentHashMap<>(); /**
* gid → mpvkey 的内存缓存。key 会轮换,代码里对失配也做了刷新兜底,
* 因此这里只需限制为有界 LRU,避免长期运行无上限增长。
*/
static final Map<String, String> gid2MpvKey = Collections.synchronizedMap(
new LinkedHashMap<>(64, 0.75f, true) {
@Override
protected boolean removeEldestEntry(Map.Entry<String, String> eldest) {
return size() > 2048;
}
});
/** Reusable HTTP client —不要每次请求新建 */ /** Reusable HTTP client —不要每次请求新建 */
private static final CloseableHttpClient httpClient = HttpClients.custom() private static final CloseableHttpClient httpClient = HttpClients.custom()
@@ -73,6 +82,12 @@ public class GalleryUtil {
* @throws ResolutionNotMatchException 没有目标分辨率 * @throws ResolutionNotMatchException 没有目标分辨率
*/ */
public static Gallery parse(String url, boolean isDownload, String targetResolution) throws IOException, ResolutionNotMatchException { public static Gallery parse(String url, boolean isDownload, String targetResolution) throws IOException, ResolutionNotMatchException {
PreparedGallery prepared = prepare(url);
if (prepared == null) return null;
return isDownload ? submit(prepared, targetResolution) : prepared.gallery();
}
public static PreparedGallery prepare(String url) throws IOException {
String origin = verifyLink(url); String origin = verifyLink(url);
if(origin == null){ if(origin == null){
@@ -80,66 +95,31 @@ public class GalleryUtil {
} }
//初始化图片 //初始化图片
Gallery gallery = new Gallery(); Integer gid = parseGid(url);
gallery.setLink(url); if (gid == null) {
gallery.setCreateTime(System.currentTimeMillis()/1000); log.warn("链接无法解析出 gid,按无效链接处理");
gallery.setGid(Integer.parseInt(url.split("/")[4])); return null;
gallery.setProceeding(0); }
Document galleryDoc = Jsoup.parse(requests(url, GET, null, null), url);
//访问图片页面 Gallery gallery = GalleryPageParser.metadata(galleryDoc, url, gid);
String galleryPage = requests(url, GET, null, null); String archiveLink = GalleryPageParser.archiveLink(galleryDoc);
Document galleryDoc = Jsoup.parse(galleryPage);
//收集图片基本信息
gallery.setName(galleryDoc.select("#gn").text() + " [" + gallery.getGid() + ']');
gallery.setLanguage(galleryDoc.select("#gdd > table > tbody > tr:nth-child(4) > td.gdt2").text().replace("%nbps", ""));
gallery.setPages(Integer.parseInt(galleryDoc.select("#gdd > table > tbody > tr:nth-child(6) > td.gdt2").text().split(" ")[0]));
gallery.setFileSize(CustomUtil.stringToFileSize(galleryDoc.select("#gdd > table > tbody > tr:nth-child(5) > td.gdt2").text().replace(" ", "")));
gallery.setDisplayFileSize(CustomUtil.fileSizeToString(gallery.getFileSize()));
String thumb_link = galleryDoc.select("#gd1 > div").getFirst().attr("style");
gallery.setThumb_link(thumb_link.substring(thumb_link.indexOf("url(") + 4, thumb_link.lastIndexOf(")")).replace("https://s.exhentai.org", ""));
//查找下载页面链接并初始化参数
String download_link = galleryDoc.select("#gd5 > p:nth-child(2) > a").attr("onclick").split("'")[1];
HashMap<String, String> headers = new HashMap<>(); HashMap<String, String> headers = new HashMap<>();
headers.put("origin", origin); headers.put("origin", origin);
headers.put("referer", download_link); headers.put("referer", archiveLink);
Document downloadDoc = Jsoup.parse(requests(archiveLink, GET, headers, null), archiveLink);
//访问下载页面,获取分辨率 String download_link = downloadDoc.select("#hathdl_form").attr("abs:action");
String downloadPage = requests(download_link, GET, headers, null); if (download_link.isBlank()) throw new IOException("上游页面缺少下载提交表单");
Document downloadDoc = Jsoup.parse(downloadPage); gallery.setAvailableResolution(GalleryPageParser.resolutions(downloadDoc));
Elements resolutions = downloadDoc.select("#db > div > table > tbody > tr > td");
download_link = downloadDoc.select("#hathdl_form").attr("action");
Map<String, String> availableResolution = new LinkedHashMap<>();
Map<String, String> tempResolutions = new LinkedHashMap<>();
ArrayList<Integer> tempResolutionKeys = new ArrayList<>();
for(Element element: resolutions){
String[] parameter = element.select("td > p").text().split(" ");
if(parameter[1].contains("N/A"))
continue;
tempResolutions.put(parameter[0], parameter[1] + parameter[2]);
if(parameter[0].contains("x")){
tempResolutionKeys.add(Integer.valueOf(parameter[0].replace("x", "")));
}
}
tempResolutionKeys.sort(Comparator.comparingInt(o -> o));
for(Integer resolution: tempResolutionKeys){
availableResolution.put(resolution.toString()+"x", tempResolutions.get(resolution+"x"));
}
availableResolution.put("Original", tempResolutions.get("Original"));
gallery.setAvailableResolution(availableResolution);
//如果不是下载,则直接返回
if(!isDownload){
gallery.setStatus("等待确认下载"); gallery.setStatus("等待确认下载");
return gallery; return new PreparedGallery(gallery, download_link, origin);
} }
public static Gallery submit(PreparedGallery prepared, String targetResolution) throws IOException, ResolutionNotMatchException {
Gallery gallery = prepared.gallery();
String download_link = prepared.formUrl();
HashMap<String, String> headers = new HashMap<>();
headers.put("origin", prepared.origin());
headers.put("referer", download_link);
//如果目标分辨率不存在,抛出错误 //如果目标分辨率不存在,抛出错误
if(!gallery.getAvailableResolution().containsKey(targetResolution)){ if(!gallery.getAvailableResolution().containsKey(targetResolution)){
log.warn("目标分辨率不存在,可用分辨率: {}", gallery.getAvailableResolution()); log.warn("目标分辨率不存在,可用分辨率: {}", gallery.getAvailableResolution());
@@ -147,15 +127,15 @@ public class GalleryUtil {
} }
gallery.setResolution(targetResolution); gallery.setResolution(targetResolution);
gallery.setFileSize(CustomUtil.stringToFileSize(availableResolution.get(targetResolution).replace(" ", ""))); gallery.setFileSize(CustomUtil.stringToFileSize(gallery.getAvailableResolution().get(targetResolution).replace(" ", "")));
gallery.setDisplayFileSize(CustomUtil.fileSizeToString(gallery.getFileSize())); gallery.setDisplayFileSize(CustomUtil.fileSizeToString(gallery.getFileSize()));
//提交下载请求 //提交下载请求
HashMap<String, String> body = new HashMap<>(); HashMap<String, String> body = new HashMap<>();
body.put("payload", FORM_DATA); body.put("payload", FORM_DATA);
body.put("hathdl_xres", targetResolution.replace("x", "").replace("Original", "org")); body.put("hathdl_xres", targetResolution.replace("x", "").replace("Original", "org"));
downloadPage = requests(download_link, POST, headers, body); String downloadPage = requests(download_link, POST, headers, body);
downloadDoc = Jsoup.parse(downloadPage); Document downloadDoc = Jsoup.parse(downloadPage);
//判断下载请求是否提交成功 //判断下载请求是否提交成功
if(downloadDoc.select("#db > p:nth-child(2) > strong").text().startsWith("#")) if(downloadDoc.select("#db > p:nth-child(2) > strong").text().startsWith("#"))
@@ -178,14 +158,12 @@ public class GalleryUtil {
if(content.trim().isEmpty()) if(content.trim().isEmpty())
return null; return null;
Document document = Jsoup.parse(content); Document document = Jsoup.parse(content);
if(document.select("body > script").size() < 2) String script = document.select("script").stream().map(Element::data).collect(java.util.stream.Collectors.joining("\n"));
return null; if (!script.contains("imagelist") || !script.contains("mpvkey")) return null;
Element script = document.select("body > script").get(1);
String[] scripts = script.html().split("\n");
ArrayList<ImageKeyCache> imageKeyCaches = new ArrayList<>(); ArrayList<ImageKeyCache> imageKeyCaches = new ArrayList<>();
AtomicInteger page = new AtomicInteger(1); AtomicInteger page = new AtomicInteger(1);
gid2MpvKey.put(gid, scripts[1].split("=")[1].replace(";", "").replace("\"", "").replace(" ", "")); gid2MpvKey.put(gid, JavaScriptAssignments.string(script, "mpvkey"));
JsonNode nodes = objectMapper.readValue(scripts[2].replace("var imagelist = ", ""), JsonNode.class); JsonNode nodes = objectMapper.readTree(JavaScriptAssignments.array(script, "imagelist"));
nodes.forEach((n) -> { nodes.forEach((n) -> {
ImageKeyCache imageKeyCache = new ImageKeyCache(); ImageKeyCache imageKeyCache = new ImageKeyCache();
imageKeyCache.setGid(gid); imageKeyCache.setGid(gid);
@@ -196,6 +174,19 @@ public class GalleryUtil {
return imageKeyCaches; return imageKeyCaches;
} }
/**
* 解析 mpv 页里的 imagelist 行。
* 该行是 JS 赋值语句(`var imagelist = [...];`),行尾分号属 JavaScript 语法而非 JSON,
* 因此先剥掉「var imagelist = 」前缀与行尾分号,再交给 JSON 解析。
* Jackson 3 默认开启 FAIL_ON_TRAILING_TOKENS,若把分号留给它会在解析时报错。
*/
private static JsonNode parseImagelist(String scriptLine) {
String json = scriptLine.replace("var imagelist = ", "").trim();
if (json.endsWith(";"))
json = json.substring(0, json.length() - 1).trim();
return objectMapper.readValue(json, JsonNode.class);
}
public static String getMpvKey(String url){ public static String getMpvKey(String url){
String gid = String.valueOf(parseGid(url)); String gid = String.valueOf(parseGid(url));
String key = gid2MpvKey.get(gid); String key = gid2MpvKey.get(gid);
@@ -220,12 +211,14 @@ public class GalleryUtil {
gid2MpvKey.remove(parseGid(url) + ""); gid2MpvKey.remove(parseGid(url) + "");
return; return;
} }
Document document = Jsoup.parse(content); try {
Element script = document.select("body > script").get(1); String scripts = Jsoup.parse(content).select("script").stream().map(Element::data)
String[] scripts = script.html().split("\n"); .collect(java.util.stream.Collectors.joining("\n"));
String mpvKey = scripts[1].split("=")[1].replace(";", "").replace("\"", "").replace(" ", ""); gid2MpvKey.put(parseGid(url) + "", JavaScriptAssignments.string(scripts, "mpvkey"));
gid2MpvKey.put(parseGid(url) + "", mpvKey); } catch (IOException e) {
log.info("刷新key:{}", mpvKey); gid2MpvKey.remove(parseGid(url) + "");
log.warn("上游mpv脚本无法解析,缓存已失效");
}
} }
public static String getImageUrl(String mpvKey, ImageKeyCache imageKeyCache) { public static String getImageUrl(String mpvKey, ImageKeyCache imageKeyCache) {
@@ -367,10 +360,27 @@ public class GalleryUtil {
} }
} }
/**
* 从链接里安全提取 gid。
*
* <p>历史实现只捕 {@code IndexOutOfBoundsException}:非数字段会抛
* {@code NumberFormatException},{@code link == null} 会抛 NPE,两者都会在
* 「无 @ControllerAdvice」的项目里穿透成 500。这里与
* {@code GalleryManageService.parseGidFromLink} 保持一致的宽松语义:
* 任何畸形输入都返回 null,由调用方转成业务失败。
*/
public static Integer parseGid(String link){ public static Integer parseGid(String link){
if (link == null)
return null;
String[] parts = link.split("/g/");
if (parts.length < 2)
return null;
String[] segments = parts[1].split("/");
if (segments.length == 0)
return null;
try { try {
return Integer.parseInt(link.split("/g/")[1].split("/")[0]); return Integer.parseInt(segments[0]);
}catch (IndexOutOfBoundsException e){ } catch (NumberFormatException e) {
return null; return null;
} }
} }
@@ -0,0 +1,34 @@
package com.lion.lionwebsite.Util;
import java.io.IOException;
import java.util.regex.Pattern;
/** Extracts data assignments without depending on script position or line layout. */
public final class JavaScriptAssignments {
private JavaScriptAssignments() { }
public static String string(String source, String name) throws IOException {
var matcher = Pattern.compile("\\b" + Pattern.quote(name) + "\\s*=\\s*(['\"])(.*?)\\1", Pattern.DOTALL).matcher(source);
if (!matcher.find()) throw new IOException("上游脚本缺少 " + name);
return matcher.group(2);
}
public static String array(String source, String name) throws IOException {
var matcher = Pattern.compile("\\b" + Pattern.quote(name) + "\\s*=\\s*\\[").matcher(source);
if (!matcher.find()) throw new IOException("上游脚本缺少 " + name);
int start = matcher.end() - 1, depth = 0; char quote = 0; boolean escaped = false;
for (int i = start; i < source.length(); i++) {
char c = source.charAt(i);
if (quote != 0) {
if (escaped) escaped = false;
else if (c == '\\') escaped = true;
else if (c == quote) quote = 0;
continue;
}
if (c == '"' || c == '\'') quote = c;
else if (c == '[') depth++;
else if (c == ']' && --depth == 0) return source.substring(start, i + 1);
}
throw new IOException("上游脚本的 " + name + " 不完整");
}
}
@@ -0,0 +1,107 @@
package com.lion.lionwebsite.Util;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.time.Duration;
import java.time.Instant;
import java.util.HexFormat;
/**
* 机器人一键登录票据:与 PersonalHub 共享密钥的 HMAC 自签串。
*
* <p>形状为 {@code v1.<签发秒级时间戳>.<HMAC-SHA256(密钥, "v1.<时间戳>") 的十六进制>}。
* 机器人本地用同一把密钥签出,LionWebsite 侧只做验签与新鲜度判断,因此不需要
* 机器人到主站的网络调用,两端任一侧临时不可达都不影响链接生成。
*
* <p>设计取舍:票据不携带一次性状态,时间窗内可重放(用户明确接受这一点)。
* 窗口外的票据一律拒绝,所以泄露的链接在 {@code ticket-ttl-seconds} 之后自动失效。
*
* <p>票据只签发给主人:密钥只配置在主人自己的机器人上,不随用户分发。
*/
@Component
@Slf4j
public class PanelLoginTicket {
private static final String VERSION = "v1";
private static final String ALGORITHM = "HmacSHA256";
private final byte[] secret;
private final Duration ttl;
public PanelLoginTicket(
@Value("${personal.login.secret:}") String secret,
@Value("${personal.login.ticket-ttl-seconds:300}") long ttlSeconds) {
this.secret = secret == null ? new byte[0] : secret.getBytes(StandardCharsets.UTF_8);
this.ttl = Duration.ofSeconds(Math.max(30, ttlSeconds));
if (this.secret.length == 0)
log.warn("未配置 personal.login.secret,一键登录票据将一律拒绝");
}
/** 密钥未配置时无法签发也无法校验,调用方据此给出可读提示。 */
public boolean configured() {
return secret.length > 0;
}
public String issue() {
return issue(Instant.now());
}
/** 供测试注入固定时钟。 */
public String issue(Instant now) {
if (!configured())
throw new IllegalStateException("未配置 personal.login.secret");
String stamp = String.valueOf(now.getEpochSecond());
return VERSION + "." + stamp + "." + sign(stamp);
}
public boolean verify(String ticket) {
return verify(ticket, Instant.now());
}
public boolean verify(String ticket, Instant now) {
if (!configured() || ticket == null)
return false;
String[] parts = ticket.split("\\.", -1);
if (parts.length != 3 || !VERSION.equals(parts[0]) || parts[1].isEmpty() || parts[2].isEmpty())
return false;
if (!MessageDigest.isEqual(sign(parts[1]).getBytes(StandardCharsets.UTF_8),
parts[2].getBytes(StandardCharsets.UTF_8)))
return false;
long issuedAt;
try {
issuedAt = Long.parseLong(parts[1]);
} catch (NumberFormatException e) {
return false;
}
// 只接受 [now - ttl, now]:未来时间戳一律拒绝,避免伪造者用远期时间换取长期有效。
long ageSeconds = now.getEpochSecond() - issuedAt;
return ageSeconds >= 0 && ageSeconds <= ttl.getSeconds();
}
public long ttlSeconds() {
return ttl.getSeconds();
}
private String sign(String stamp) {
try {
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(new SecretKeySpec(secret, ALGORITHM));
return HexFormat.of().formatHex(
mac.doFinal((VERSION + "." + stamp).getBytes(StandardCharsets.UTF_8)));
} catch (Exception e) {
// 算法名固定且密钥非空,正常不会走到这里;一旦发生必须视为校验失败。
log.error("计算登录票据签名失败", e);
return "";
}
}
}
@@ -0,0 +1,6 @@
package com.lion.lionwebsite.Util;
import com.lion.lionwebsite.Domain.Gallery;
/** A read-only upstream preparation; creating this value never submits a download. */
public record PreparedGallery(Gallery gallery, String formUrl, String origin) { }
@@ -0,0 +1,328 @@
package com.lion.lionwebsite.Util;
import lombok.extern.slf4j.Slf4j;
import java.io.IOException;
import java.io.StringReader;
import java.nio.charset.StandardCharsets;
import java.nio.file.AtomicMoveNotSupportedException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.security.MessageDigest;
import java.util.Comparator;
import java.util.HexFormat;
import java.util.List;
import java.util.Optional;
import java.util.Properties;
import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Stream;
/**
* 订阅缓存的分代存储:v2ray 与 clash 两份内容必须同时生效。
*
* <p>旧实现把两份文件各自原子替换,第二份失败时用户会拿到「新 v2ray + 旧 clash」的
* 混合版本。这里把两份文件写进同一个不可变分代目录,校验通过后用一次原子替换发布
* 指针(manifest),因此抓取、解析、写入或发布任何一步失败,都只会保留上一个完整分代。
*
* <p>目录结构:
* <pre>
* &lt;cacheRoot&gt;/&lt;accountId&gt;/current 指针(原子替换,最后发布)
* &lt;cacheRoot&gt;/&lt;accountId&gt;/generations/&lt;generation&gt;/v2ray.txt
* &lt;cacheRoot&gt;/&lt;accountId&gt;/generations/&lt;generation&gt;/clash.yaml
* &lt;cacheRoot&gt;/&lt;accountId&gt;/v2ray.txt + clash.yaml 旧结构,两份都在才按 legacy 读
* </pre>
*
* <p>读取始终从指针指向的同一个分代里取两份文件,所以拿到的一定是同版本内容。
*/
@Slf4j
public class SubscriptionCacheStore {
private static final String MANIFEST_FILE = "current";
private static final String GENERATION_DIR = "generations";
private static final String V2_FILE = "v2ray.txt";
private static final String CLASH_FILE = "clash.yaml";
/** 保留目录名,结构上永远不存在,用于表达「当前没有可分发版本」。 */
private static final String UNAVAILABLE_DIR = ".unavailable";
private static final String MANIFEST_GENERATION = "generation";
private static final String MANIFEST_V2_SHA = "v2-sha256";
private static final String MANIFEST_CLASH_SHA = "clash-sha256";
/** 旧结构读出来的版本的 generation 标记;真实分代名是定宽时间戳,不会与之碰撞。 */
private static final String LEGACY_GENERATION = "legacy";
/** 保留分代数:当前与上一个,足够回退且不会无限增长。 */
private static final int RETAINED_GENERATIONS = 2;
private static final AtomicLong SEQUENCE = new AtomicLong();
private final Path root;
public SubscriptionCacheStore(String cacheRoot) {
this(Paths.get(cacheRoot));
}
public SubscriptionCacheStore(Path root) {
this.root = root;
}
/** 同一个分代目录里读出的一对内容;两份文件必然同版本。 */
public record Pair(String generation, byte[] v2, byte[] clash) { }
public Path accountDir(int accountId) {
return root.resolve(String.valueOf(accountId));
}
/** 旧结构的账号根文件路径。仅供兼容测试定位 legacy 文件,不作为回退目标。 */
public Path legacyPath(int accountId, String client) {
return accountDir(accountId).resolve(fileName(client));
}
/**
* 没有完整可分发版本时返回的哨兵路径。
*
* <p>它保证不存在,因此调用方用 {@code Files.isRegularFile} 判定就会走「订阅暂不可用」
* 分支,而不会被半份 legacy 文件骗过。
*/
public Path unavailablePath(int accountId, String client) {
return accountDir(accountId).resolve(UNAVAILABLE_DIR).resolve(fileName(client));
}
private static String fileName(String client) {
return "v2".equals(client) ? V2_FILE : CLASH_FILE;
}
/**
* 写入一份新分代并在最后发布指针。
*
* <p>失败时抛出异常,且不改变指针与上一个分代,调用方据此保留上次的完整版本。
*/
public Pair publish(int accountId, byte[] v2, byte[] clash) throws IOException {
Path account = accountDir(accountId);
Files.createDirectories(account);
String generation = nextGeneration();
Path dir = account.resolve(GENERATION_DIR).resolve(generation);
boolean published = false;
try {
Files.createDirectories(dir);
writeFile(dir.resolve(V2_FILE), v2);
writeFile(dir.resolve(CLASH_FILE), clash);
// 发布前校验:两份文件都必须能按写入内容读回,避免把半截内容挂上指针。
verify(dir.resolve(V2_FILE), v2);
verify(dir.resolve(CLASH_FILE), clash);
publishManifest(account, generation, sha256(v2), sha256(clash));
published = true;
} finally {
if (!published)
deleteRecursively(dir);
}
prune(account, generation);
removeLegacy(account);
return new Pair(generation, v2, clash);
}
/** 读取当前发布内容;只有构成一整个有效版本时才返回。 */
public Optional<Pair> read(int accountId) {
return Optional.ofNullable(load(accountDir(accountId)));
}
/**
* 解析当前应分发的单个文件。
*
* <p>与 {@link #read}、{@link #hasComplete} 使用同一份配对契约:只有两份文件
* 齐全且哈希校验通过时才返回路径,因此分发路径不会发出半份或已损坏的缓存。
*/
public Optional<Path> resolve(int accountId, String client) {
Path account = accountDir(accountId);
Pair pair = load(account);
if (pair == null)
return Optional.empty();
Path path = LEGACY_GENERATION.equals(pair.generation())
? account.resolve(fileName(client))
: account.resolve(GENERATION_DIR).resolve(pair.generation()).resolve(fileName(client));
return Files.isRegularFile(path) ? Optional.of(path) : Optional.empty();
}
/** 是否已有可分发的一对内容(分代两份都在且哈希匹配,或旧结构两份都在)。 */
public boolean hasComplete(int accountId) {
return load(accountDir(accountId)) != null;
}
/** 删除该账号的全部缓存(指针、分代、旧结构文件),使后续请求判定为无缓存。 */
public void invalidate(int accountId) {
Path account = accountDir(accountId);
deleteIfExists(account.resolve(MANIFEST_FILE));
deleteRecursively(account.resolve(GENERATION_DIR));
deleteIfExists(account.resolve(V2_FILE));
deleteIfExists(account.resolve(CLASH_FILE));
}
/** 写入接缝:测试覆写以模拟第二份文件写入失败。 */
protected void writeFile(Path target, byte[] data) throws IOException {
atomicWrite(target, data);
}
/** 指针发布接缝:测试覆写以模拟 manifest 发布失败。 */
protected void publishManifest(Path accountDir, String generation, String v2Sha256, String clashSha256)
throws IOException {
atomicWrite(accountDir.resolve(MANIFEST_FILE),
manifestText(generation, v2Sha256, clashSha256).getBytes(StandardCharsets.UTF_8));
}
/**
* 读取「一整个版本」:一次读入指针快照,再按该快照定位分代并校验双文件哈希。
*
* <p>指针只读一次,因此并发 {@link #publish} 切换指针时不会出现「用新分代名配旧哈希」
* 或哈希字段缺失导致的空指针;要么拿到完整的新版本,要么拿到完整的旧版本。
*/
private Pair load(Path account) {
Properties manifest = readManifest(account);
if (manifest == null) {
// 指针文件不存在才是「旧结构」。指针存在但读不出来说明它已损坏,同样按无缓存处理。
return Files.isRegularFile(account.resolve(MANIFEST_FILE)) ? null : loadLegacy(account);
}
Pair generation = loadGeneration(account, manifest);
if (generation != null)
return generation;
// 指针存在却读不出完整版本(半写、被篡改、内容缺失)时不得回退到 legacy:
// 那会发出一个比指针更旧的版本,掩盖损坏。此时按无可用缓存处理。
return null;
}
private Pair loadGeneration(Path account, Properties manifest) {
String generation = manifest.getProperty(MANIFEST_GENERATION);
if (generation == null || generation.isBlank() || generation.contains("/") || generation.contains("..")) {
log.warn("订阅缓存指针内容非法");
return null;
}
String v2Sha256 = manifest.getProperty(MANIFEST_V2_SHA);
String clashSha256 = manifest.getProperty(MANIFEST_CLASH_SHA);
if (v2Sha256 == null || clashSha256 == null) {
log.warn("订阅缓存指针缺少哈希字段 generation={}", generation);
return null;
}
Path dir = account.resolve(GENERATION_DIR).resolve(generation);
byte[] v2 = readFile(dir.resolve(V2_FILE));
byte[] clash = readFile(dir.resolve(CLASH_FILE));
if (v2 == null || clash == null) {
log.warn("订阅缓存分代内容缺失 generation={}", generation);
return null;
}
if (!matches(v2, v2Sha256) || !matches(clash, clashSha256)) {
log.warn("订阅缓存分代内容校验失败 generation={}", generation);
return null;
}
return new Pair(generation, v2, clash);
}
/** 旧结构:只有账号根下两份文件都在才构成一个有效版本。 */
private Pair loadLegacy(Path account) {
byte[] v2 = readFile(account.resolve(V2_FILE));
byte[] clash = readFile(account.resolve(CLASH_FILE));
if (v2 == null || clash == null)
return null;
return new Pair(LEGACY_GENERATION, v2, clash);
}
/** 指针读取接缝:包内可见,便于测试统计「每次查询只读一次指针」。 */
Properties readManifest(Path account) {
Path manifest = account.resolve(MANIFEST_FILE);
if (!Files.isRegularFile(manifest))
return null;
try {
Properties properties = new Properties();
properties.load(new StringReader(Files.readString(manifest, StandardCharsets.UTF_8)));
return properties;
} catch (IOException e) {
log.warn("读取订阅缓存指针失败: {}", e.getMessage());
return null;
}
}
private static byte[] readFile(Path path) {
try {
return Files.isRegularFile(path) ? Files.readAllBytes(path) : null;
} catch (IOException e) {
log.warn("读取订阅缓存文件失败 path={} error={}", path, e.getMessage());
return null;
}
}
private static void verify(Path path, byte[] expected) throws IOException {
byte[] written = Files.readAllBytes(path);
if (written.length != expected.length || !MessageDigest.isEqual(written, expected))
throw new IOException("订阅缓存写入校验失败 " + path.getFileName());
}
private static boolean matches(byte[] content, String expectedSha256) {
return expectedSha256 != null && expectedSha256.equals(sha256(content));
}
private static String manifestText(String generation, String v2Sha256, String clashSha256) {
return MANIFEST_GENERATION + "=" + generation + "\n"
+ MANIFEST_V2_SHA + "=" + v2Sha256 + "\n"
+ MANIFEST_CLASH_SHA + "=" + clashSha256 + "\n";
}
/** 分代名定宽可排序:时间戳在前保证单调,序号消除同毫秒碰撞。 */
private static String nextGeneration() {
return String.format("%013d-%06d", System.currentTimeMillis(), SEQUENCE.incrementAndGet() % 1_000_000);
}
private void prune(Path account, String current) {
Path dir = account.resolve(GENERATION_DIR);
List<Path> generations;
try (Stream<Path> entries = Files.list(dir)) {
generations = entries.filter(Files::isDirectory)
.sorted(Comparator.comparing(path -> path.getFileName().toString(), Comparator.reverseOrder()))
.toList();
} catch (IOException e) {
log.warn("清理订阅缓存分代失败 path={} error={}", dir, e.getMessage());
return;
}
for (int index = RETAINED_GENERATIONS; index < generations.size(); index++) {
Path stale = generations.get(index);
if (!stale.getFileName().toString().equals(current))
deleteRecursively(stale);
}
}
/** 分代发布成功后旧结构文件被取代,避免两份来源同时存在。 */
private void removeLegacy(Path account) {
deleteIfExists(account.resolve(V2_FILE));
deleteIfExists(account.resolve(CLASH_FILE));
}
private static void atomicWrite(Path target, byte[] data) throws IOException {
Path temp = target.resolveSibling(target.getFileName() + ".tmp");
Files.write(temp, data);
try {
Files.move(temp, target, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
} catch (AtomicMoveNotSupportedException e) {
Files.move(temp, target, StandardCopyOption.REPLACE_EXISTING);
}
}
private static void deleteIfExists(Path path) {
try {
Files.deleteIfExists(path);
} catch (IOException e) {
log.warn("删除订阅缓存文件失败 path={} error={}", path, e.getMessage());
}
}
private static void deleteRecursively(Path path) {
if (!Files.exists(path))
return;
try (Stream<Path> entries = Files.walk(path)) {
entries.sorted(Comparator.reverseOrder()).forEach(SubscriptionCacheStore::deleteIfExists);
} catch (IOException e) {
log.warn("删除订阅缓存目录失败 path={} error={}", path, e.getMessage());
}
}
private static String sha256(byte[] input) {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(input));
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
}
@@ -0,0 +1,65 @@
package com.lion.lionwebsite.Util;
import java.util.List;
/**
* 抓取上游订阅时伪装的客户端身份。
*
* <p>此前请求只带 HttpClient 的默认头,实测上游收到的是
* {@code User-agent: Apache-HttpClient/5.6.4 (Java/25.0.4)},
* 对订阅站点而言等同于「这是一个 Java 程序在批量拉取」。
* 这里按订阅格式给出与真实客户端一致的常见头组合。
*
* <p>注意:只能伪装到 HTTP 头这一层。真实 Clash/Mihomo 使用 Go 的 TLS 栈与
* HTTP/2,TLS 指纹与 Java HttpClient 仍有差异;若上游做主动指纹识别,
* 需要另外引入真实客户端进程,本枚举解决不了。
*/
public enum SubscriptionClientProfile {
MIHOMO("mihomo/v1.19.2", "application/yaml, */*", "zh-CN,zh;q=0.9"),
CLASH_VERGE("clash-verge/v2.0.3", "application/yaml, */*", "zh-CN,zh;q=0.9"),
CLASH_WINDOWS("ClashforWindows/0.20.39", "*/*", "zh-CN,zh;q=0.9"),
V2RAYN("v2rayN/6.45", "*/*", "zh-CN,zh;q=0.9"),
V2RAYNG("v2rayNG/1.9.16", "*/*", "zh-CN,zh;q=0.9"),
SHADOWROCKET("Shadowrocket/2.2.39", "*/*", "zh-CN,zh;q=0.9");
static final List<SubscriptionClientProfile> CLASH_PROFILES = List.of(MIHOMO, CLASH_VERGE, CLASH_WINDOWS);
static final List<SubscriptionClientProfile> V2_PROFILES = List.of(V2RAYN, V2RAYNG, SHADOWROCKET);
private final String userAgent;
private final String accept;
private final String acceptLanguage;
SubscriptionClientProfile(String userAgent, String accept, String acceptLanguage) {
this.userAgent = userAgent;
this.accept = accept;
this.acceptLanguage = acceptLanguage;
}
public String userAgent() {
return userAgent;
}
public String accept() {
return accept;
}
public String acceptLanguage() {
return acceptLanguage;
}
/**
* 按账号与订阅格式确定性地选择一个身份。
*
* <p>刻意不使用随机数:同一个账号每次都用同一个客户端,才符合真实用户的样子;
* 逐请求更换 User-Agent 本身就是更明显的机器特征。不同账号使用不同客户端,
* 又便于在必要时按账号区分上游流量。
*
* @param accountId 子账号 ID,可为 null(视为 0)
* @param clash true 表示 Clash 格式,false 表示 V2Ray 格式
*/
public static SubscriptionClientProfile forAccount(Integer accountId, boolean clash) {
List<SubscriptionClientProfile> candidates = clash ? CLASH_PROFILES : V2_PROFILES;
int id = accountId == null ? 0 : accountId;
return candidates.get(Math.floorMod(id, candidates.size()));
}
}
+39 -1
View File
@@ -1,5 +1,15 @@
server: server:
port: 8888 port: 8888
servlet:
session:
# nginx 的 location / 会把浏览器请求改写成后端 /personal/…,而浏览器地址栏仍是
# /user 这类路径。若沿用容器按请求路径推导的 Cookie Path(/personal),浏览器
# 判定 /user 不匹配就不会带上会话,面板会一直 401。这里固定为 /。
cookie:
path: /
http-only: true
same-site: lax
timeout: 14d
tomcat: tomcat:
max-swallow-size: 10000MB max-swallow-size: 10000MB
http2: http2:
@@ -12,6 +22,12 @@ spring:
datasource-cache: datasource-cache:
driver-class-name: org.sqlite.JDBC driver-class-name: org.sqlite.JDBC
jdbc-url: jdbc:sqlite:cache.db jdbc-url: jdbc:sqlite:cache.db
# 定时任务默认只有 1 个线程。订阅刷新一轮最多 2 个账号、每个 2 次请求(各 15 秒超时),
# 最坏可占用约 1 分钟;单线程会让连接自检、cookie 检测等任务在此期间全部排队。
task:
scheduling:
pool:
size: 2
mvc: mvc:
view: view:
prefix: /resources/templates/ prefix: /resources/templates/
@@ -38,6 +54,19 @@ local:
subscription: subscription:
cache-root: sub/accounts cache-root: sub/accounts
# 分散刷新:每个账号在 window-hours 内至少刷新一次,两两之间至少相隔 min-gap-minutes;
# max-per-tick 限制单个 tick 刷新几个账号,避免重启后一批到期账号同时开火。
refresh:
window-hours: 24
min-gap-minutes: 60
# 5 分钟一个轻量 tick,只处理已到期账号。
tick-interval-ms: 300000
# 重启后延迟 2 分钟再开始,先让应用完成启动。
initial-delay-ms: 120000
max-per-tick: 2
# 失败后的重试间隔,必须小于 window-hours。
retry-delay-minutes: 60
stale-alert-multiplier: 2.0
upstream: upstream:
# Use environment variables or an external config file in production. # Use environment variables or an external config file in production.
v2-url-template: "https://aaaa.gay/link/{key}?client=v2" v2-url-template: "https://aaaa.gay/link/{key}?client=v2"
@@ -47,7 +76,16 @@ subscription:
standby: standby:
sync-enabled: "${SUBSCRIPTION_STANDBY_SYNC_ENABLED:false}" sync-enabled: "${SUBSCRIPTION_STANDBY_SYNC_ENABLED:false}"
sync-secret: "${SUBSCRIPTION_SYNC_SECRET:}" sync-secret: "${SUBSCRIPTION_SYNC_SECRET:}"
retry-interval-ms: 60000 # 快照只在订阅内容变化时推送(另在节点上线时补推一次),没有周期性重发。
# 备机的过期语义是「主站失联」,由节点按「收到主站的任何消息」判定新鲜度,
# 主站每 30 分钟的可用性检查即可持续刷新,无需为续期周期性传输快照。
bot: bot:
token: "5222939329:AAHa6l9ZuVVdNSDLPI_H-c8O_VgeOEw5plA" token: "5222939329:AAHa6l9ZuVVdNSDLPI_H-c8O_VgeOEw5plA"
# 一键登录:机器人用同一密钥自签 HMAC 票据,主站只做验签与时间窗校验。
# secret 必须通过环境变量注入,不得写入仓库;为空时所有票据一律拒绝。
personal:
login:
secret: "${PERSONAL_LOGIN_SECRET:}"
ticket-ttl-seconds: 300
@@ -0,0 +1,169 @@
package com.lion.lionwebsite.Configuration;
import com.lion.lionwebsite.Interceptor.HumanInterceptor;
import com.lion.lionwebsite.Interceptor.PersonalInterceptor;
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
import com.lion.lionwebsite.Service.WebSocketService;
import org.junit.jupiter.api.Test;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.config.annotation.InterceptorRegistration;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
/**
* 拦截器与 WebSocket 的注册关系。
* 这是整个应用的「安全地图」:若某条路径漏挂鉴权拦截器,接口就会在无授权码
* 的情况下对外可用(9/8 那批改动正是栽在 /GalleryManage/** 的漏挂上),
* 所以这里把路径与拦截器的对应关系显式锁进测试。
*/
class ConfigurationWiringTest {
/** 记录 addInterceptors 调用顺序与实际挂载路径的假注册表。 */
private static final class RecordingRegistry extends InterceptorRegistry {
final List<String> order = new ArrayList<>();
final Map<String, List<String>> paths = new LinkedHashMap<>();
@Override
public InterceptorRegistration addInterceptor(HandlerInterceptor interceptor) {
String name = interceptor.getClass().getSimpleName();
order.add(name);
return new RecordingRegistration(name, paths);
}
}
/** 只记录路径,其余注册动作不做真实处理。 */
private static final class RecordingRegistration extends InterceptorRegistration {
private final String name;
private final Map<String, List<String>> paths;
RecordingRegistration(String name, Map<String, List<String>> paths) {
super(mock(HandlerInterceptor.class));
this.name = name;
this.paths = paths;
}
@Override
public InterceptorRegistration addPathPatterns(String... patterns) {
paths.computeIfAbsent(name, k -> new ArrayList<>()).addAll(List.of(patterns));
return this;
}
}
private static RecordingRegistry registryOf(InterceptorConfiguration config) {
RecordingRegistry registry = new RecordingRegistry();
config.addInterceptors(registry);
return registry;
}
/** TaskHandlerInterceptor 必须覆盖 /GalleryManage 全子路径与 /validate。 */
@Test
void taskHandlerGuardsGalleryManageAndValidate() {
var config = new InterceptorConfiguration(mock(TaskHandlerInterceptor.class));
var registry = registryOf(config);
List<String> guarded = registry.paths.get("TaskHandlerInterceptor");
assertNotNull(guarded, "TaskHandlerInterceptor 必须被注册");
assertTrue(guarded.contains("/GalleryManage"), "精确路径必须挂着");
assertTrue(guarded.contains("/GalleryManage/**"), "子路径必须挂着(历史漏挂点)");
assertTrue(guarded.contains("/validate"));
}
/** PersonalInterceptor 必须覆盖 /personal/** 与 /remote/**。 */
@Test
void personalInterceptorGuardsPrivateAreas() {
var config = new InterceptorConfiguration(mock(TaskHandlerInterceptor.class));
var registry = registryOf(config);
List<String> guarded = registry.paths.get("PersonalInterceptor");
assertNotNull(guarded, "PersonalInterceptor 必须被注册");
assertTrue(guarded.contains("/personal/**"));
assertTrue(guarded.contains("/remote/**"));
}
/** HumanInterceptor 只管首页与移动端入口。 */
@Test
void humanInterceptorGuardsEntryPointsOnly() {
var config = new InterceptorConfiguration(mock(TaskHandlerInterceptor.class));
var registry = registryOf(config);
assertEquals(List.of("/", "/mobile"), registry.paths.get("HumanInterceptor"));
}
/** @Bean 暴露的拦截器类型必须与注册时一致(写错类型会让鉴权静默失效)。 */
@Test
void exposedInterceptorBeansHaveExpectedTypes() {
var config = new InterceptorConfiguration(mock(TaskHandlerInterceptor.class));
assertInstanceOf(PersonalInterceptor.class, config.getPersonalInterceptor());
assertInstanceOf(HumanInterceptor.class, config.getHumanInterceptor());
}
/** WebSocket 处理器必须挂在 /ws/,并放开跨域(前端部署在不同源)。 */
@Test
void websocketHandlerIsRegisteredAtWsPath() {
var service = new WebSocketService();
var config = new WebsocketConfiguration(service);
var registration = mock(org.springframework.web.socket.config.annotation.WebSocketHandlerRegistration.class,
RETURNS_SELF);
var registry = mock(org.springframework.web.socket.config.annotation.WebSocketHandlerRegistry.class);
when(registry.addHandler(any(), anyString())).thenReturn(registration);
config.registerWebSocketHandlers(registry);
verify(registry).addHandler(eq(service), eq("/ws/"));
verify(registration).setAllowedOriginPatterns("*");
}
// ---------- CORS ----------
/** 暴露 CorsRegistry 的 protected 配置表,便于断言实际生效的规则。 */
private static final class InspectableCorsRegistry
extends org.springframework.web.servlet.config.annotation.CorsRegistry {
@Override
public java.util.Map<String, org.springframework.web.cors.CorsConfiguration> getCorsConfigurations() {
return super.getCorsConfigurations();
}
}
/** CORS 配置必须允许凭据,且不得使用通配来源(两者不能同时成立)。 */
@Test
void corsAllowsCredentialsWithOriginPatterns() {
var registry = new InspectableCorsRegistry();
new CorsConfig().addCorsMappings(registry);
var configs = registry.getCorsConfigurations();
assertEquals(1, configs.size(), "应只注册一条 /** 的映射");
assertTrue(configs.containsKey("/**"));
var mapping = configs.get("/**");
assertTrue(mapping.getAllowCredentials(), "前端带 Cookie 时需要允许凭据");
assertTrue(mapping.getAllowedMethods().containsAll(List.of("GET", "POST", "PUT", "DELETE")),
"四种方法都应放开");
assertEquals(List.of("*"), mapping.getAllowedHeaders());
}
/**
* allowCredentials + allowedOrigins("*") 是非法组合(Spring 会抛异常),
* 因此必须走 allowedOriginPatterns。这里锁定该实现方式不被改回去。
*/
@Test
void corsUsesOriginPatternsRatherThanWildcardOrigins() {
var registry = new InspectableCorsRegistry();
new CorsConfig().addCorsMappings(registry);
var mapping = registry.getCorsConfigurations().get("/**");
assertTrue(mapping.getAllowedOriginPatterns().contains("*"),
"来源应通过 allowedOriginPatterns 放开");
assertNull(mapping.getAllowedOrigins(),
"不应设置 allowedOrigins,否则与 allowCredentials 冲突");
}
}
@@ -0,0 +1,302 @@
package com.lion.lionwebsite.Configuration;
import com.lion.lionwebsite.Dao.normal.GalleryMapper;
import com.lion.lionwebsite.Dao.normal.UserMapper;
import org.junit.jupiter.api.Test;
import org.mybatis.spring.mapper.MapperFactoryBean;
import org.mybatis.spring.mapper.MapperScannerConfigurer;
import org.springframework.aot.hint.MemberCategory;
import org.springframework.aot.hint.RuntimeHints;
import org.springframework.aot.hint.predicate.RuntimeHintsPredicates;
import org.springframework.beans.factory.support.RegisteredBean;
import org.springframework.beans.factory.support.RootBeanDefinition;
import java.lang.reflect.Method;
import java.util.Collection;
import static org.junit.jupiter.api.Assertions.*;
/**
* GraalVM 原生镜像的 MyBatis 运行时提示(AOT hints)。
*
* 这些代码在 JVM 模式下几乎不执行,但一旦原生构建,缺少 hint 就会在运行期抛
* ClassNotFoundException / 反射失败——而这类问题只在原生产物上暴露,回归成本极高。
* 因此这里直接调用这些处理器,断言「该注册的反射与代理都注册了」。
*/
class MyBatisNativeConfigurationTest {
/** 运行期提示注册器应把 MyBatis 关键类全部登记为可反射,并注册 dtd/xsd 资源。 */
@Test
void runtimeHintsRegistrarRegistersMyBatisReflectionAndResources() {
RuntimeHints hints = new RuntimeHints();
new MyBatisNativeConfiguration.MyBaitsRuntimeHintsRegistrar()
.registerHints(hints, getClass().getClassLoader());
// 语言驱动与日志实现等都会被 MyBatis 反射实例化
assertTrue(RuntimeHintsPredicates.reflection()
.onType(org.apache.ibatis.scripting.xmltags.XMLLanguageDriver.class).test(hints),
"XMLLanguageDriver 应可反射");
assertTrue(RuntimeHintsPredicates.reflection()
.onType(org.apache.ibatis.logging.slf4j.Slf4jImpl.class).test(hints),
"Slf4jImpl 应可反射");
assertTrue(RuntimeHintsPredicates.reflection()
.onType(org.apache.ibatis.session.SqlSessionFactory.class).test(hints));
assertTrue(RuntimeHintsPredicates.reflection()
.onType(java.util.ArrayList.class).test(hints),
"集合类型也应登记(MyBatis 需要实例化)");
// XML 映射文件的 DTD/XSD 解析依赖这些资源
assertTrue(RuntimeHintsPredicates.resource()
.forResource("org/apache/ibatis/builder/xml/mybatis-3-mapper.dtd").test(hints));
assertTrue(RuntimeHintsPredicates.resource()
.forResource("org/apache/ibatis/builder/xml/mybatis-3-config.xsd").test(hints));
}
/** 反射条目应包含全部成员类别,而不只是构造器。 */
@Test
void reflectionHintsIncludeAllMemberCategories() {
RuntimeHints hints = new RuntimeHints();
new MyBatisNativeConfiguration.MyBaitsRuntimeHintsRegistrar()
.registerHints(hints, getClass().getClassLoader());
assertTrue(RuntimeHintsPredicates.reflection()
.onType(org.apache.ibatis.session.SqlSessionFactory.class)
.withMemberCategory(MemberCategory.INVOKE_DECLARED_METHODS).test(hints),
"应登记方法调用权限");
assertTrue(RuntimeHintsPredicates.reflection()
.onType(org.apache.ibatis.session.SqlSessionFactory.class)
.withMemberCategory(MemberCategory.ACCESS_DECLARED_FIELDS).test(hints),
"应登记字段访问权限");
}
// ---------- MyBatisMapperTypeUtils ----------
/** 返回类型解析:泛型 T 应解析成 mapper 接口声明的具体类型。 */
@Test
void resolveReturnClassUnwrapsGenerics() throws Exception {
Method genericList = Holder.class.getMethod("genericList");
assertEquals(String.class,
MyBatisNativeConfiguration.MyBatisMapperTypeUtils
.resolveReturnClass(Holder.class, genericList));
Method plain = Holder.class.getMethod("plain");
assertEquals(int.class,
MyBatisNativeConfiguration.MyBatisMapperTypeUtils
.resolveReturnClass(Holder.class, plain));
}
/** Map 泛型取 value 类型(index 1),其他泛型取第一个参数。 */
@Test
void resolveReturnClassPrefersMapValueType() throws Exception {
assertEquals(Integer.class,
MyBatisNativeConfiguration.MyBatisMapperTypeUtils
.resolveReturnClass(Holder.class, Holder.class.getMethod("genericMap")));
}
/** 数组返回类型应解析为组件类型。 */
@Test
void resolveReturnClassUnwrapsArrays() throws Exception {
assertEquals(String.class,
MyBatisNativeConfiguration.MyBatisMapperTypeUtils
.resolveReturnClass(Holder.class, Holder.class.getMethod("arrayReturn")));
}
/** 参数类型解析:所有参数类都应收集到。 */
@Test
void resolveParameterClassesCollectsEveryParameter() throws Exception {
Collection<Class<?>> params = MyBatisNativeConfiguration.MyBatisMapperTypeUtils
.resolveParameterClasses(Holder.class, Holder.class.getMethod("twoArgs", String.class, Integer.class));
assertTrue(params.contains(String.class));
assertTrue(params.contains(Integer.class));
}
/** 无参数方法应得到空集合而不是异常。 */
@Test
void resolveParameterClassesHandlesNoArguments() throws Exception {
Collection<Class<?>> params = MyBatisNativeConfiguration.MyBatisMapperTypeUtils
.resolveParameterClasses(Holder.class, Holder.class.getMethod("plain"));
assertTrue(params.isEmpty(), "无参方法应返回空集合");
}
// ---------- MapperFactoryBean 后置处理器 ----------
/**
* 泛型未解析时,应把 mapper 接口注入为构造器泛型参数并设置 targetType,
* 从而避免容器提前初始化 MapperFactoryBean。
*/
@Test
void factoryBeanPostProcessorResolvesUnresolvableMapperType() {
var processor = new MyBatisNativeConfiguration.MyBatisMapperFactoryBeanPostProcessor();
processor.setBeanFactory(new org.springframework.beans.factory.support.DefaultListableBeanFactory());
RootBeanDefinition definition = new RootBeanDefinition(MapperFactoryBean.class);
definition.setTargetType(org.springframework.core.ResolvableType.forClass(MapperFactoryBean.class));
definition.getPropertyValues().add("mapperInterface", GalleryMapper.class);
processor.postProcessMergedBeanDefinition(definition, MapperFactoryBean.class, "galleryMapper");
assertEquals(GalleryMapper.class,
definition.getConstructorArgumentValues().getGenericArgumentValue(Class.class).getValue(),
"mapper 接口应被注入为构造器泛型参数");
assertFalse(definition.getResolvableType().hasUnresolvableGenerics(),
"targetType 设置后泛型应可解析");
}
/** 非 MapperFactoryBean 的 bean 不应被改动。 */
@Test
void factoryBeanPostProcessorIgnoresOtherBeans() {
var processor = new MyBatisNativeConfiguration.MyBatisMapperFactoryBeanPostProcessor();
processor.setBeanFactory(new org.springframework.beans.factory.support.DefaultListableBeanFactory());
RootBeanDefinition definition = new RootBeanDefinition(String.class);
processor.postProcessMergedBeanDefinition(definition, String.class, "someString");
assertTrue(definition.getConstructorArgumentValues().isEmpty(),
"无关 bean 不应被注入构造器参数");
}
/** 取不到映射器接口时应静默返回,不影响启动。 */
@Test
void factoryBeanPostProcessorToleratesMissingMapperInterface() {
var processor = new MyBatisNativeConfiguration.MyBatisMapperFactoryBeanPostProcessor();
processor.setBeanFactory(new org.springframework.beans.factory.support.DefaultListableBeanFactory());
RootBeanDefinition definition = new RootBeanDefinition(MapperFactoryBean.class);
definition.setTargetType(org.springframework.core.ResolvableType.forClass(MapperFactoryBean.class));
// 故意不设置 mapperInterface 属性
assertDoesNotThrow(() ->
processor.postProcessMergedBeanDefinition(definition, MapperFactoryBean.class, "broken"));
}
// ---------- AOT 处理器 ----------
/** MapperScannerConfigurer 必须被排除在 AOT 处理之外(否则会被提前实例化)。 */
@Test
void aotProcessorExcludesMapperScannerConfigurer() {
var processor = new MyBatisNativeConfiguration.MyBatisBeanFactoryInitializationAotProcessor();
RegisteredBean registered = registeredBean("configurer", MapperScannerConfigurer.class);
assertTrue(processor.isExcludedFromAotProcessing(registered));
}
@Test
void aotProcessorDoesNotExcludeOrdinaryBeans() {
var processor = new MyBatisNativeConfiguration.MyBatisBeanFactoryInitializationAotProcessor();
RegisteredBean registered = registeredBean("service", GalleryMapper.class);
assertFalse(processor.isExcludedFromAotProcessing(registered));
}
/**
* 没有 MapperFactoryBean 时不应产出 AOT contribution(返回 null),
* 避免为无 MyBatis 的上下文生成多余代码。
*/
@Test
void aotProcessorReturnsNullWithoutMappers() {
var processor = new MyBatisNativeConfiguration.MyBatisBeanFactoryInitializationAotProcessor();
// 空工厂:没有任何 MapperFactoryBean
var beanFactory = new org.springframework.beans.factory.support.DefaultListableBeanFactory();
assertNull(processor.processAheadOfTime(beanFactory));
}
/**
* 存在 MapperFactoryBean 时应产出 contribution,并在应用时注册
* mapper 接口的反射、JDK 代理与同名 XML 资源。
*/
@Test
void aotProcessorRegistersMapperProxyAndResources() {
var processor = new MyBatisNativeConfiguration.MyBatisBeanFactoryInitializationAotProcessor();
// 真实工厂:注册一个 MapperFactoryBean,getBeanNamesForType 会带上 & 前缀
var beanFactory = new org.springframework.beans.factory.support.DefaultListableBeanFactory();
RootBeanDefinition definition = new RootBeanDefinition(MapperFactoryBean.class);
definition.getPropertyValues().add("mapperInterface", UserMapper.class);
beanFactory.registerBeanDefinition("userMapper", definition);
var contribution = processor.processAheadOfTime(beanFactory);
assertNotNull(contribution, "有 mapper 时应产出 AOT contribution");
RuntimeHints hints = new RuntimeHints();
contribution.applyTo(new StubGenerationContext(hints), new NoOpInitializationCode());
assertTrue(RuntimeHintsPredicates.reflection().onType(UserMapper.class).test(hints),
"mapper 接口应登记反射");
assertTrue(RuntimeHintsPredicates.proxies().forInterfaces(UserMapper.class).test(hints),
"mapper 接口应登记 JDK 代理");
assertTrue(RuntimeHintsPredicates.resource()
.forResource("com/lion/lionwebsite/Dao/normal/UserMapper.xml").test(hints),
"同名 XML 映射文件应登记");
}
/** applyTo 只用 runtimeHints,初始化代码侧给一个记录型空实现即可。 */
private static final class NoOpInitializationCode
implements org.springframework.beans.factory.aot.BeanFactoryInitializationCode {
final java.util.List<org.springframework.aot.generate.MethodReference> initializers = new java.util.ArrayList<>();
@Override
public org.springframework.aot.generate.GeneratedMethods getMethods() {
throw new UnsupportedOperationException("测试不断言生成方法");
}
@Override
public org.springframework.javapoet.ClassName getClassName() {
return org.springframework.javapoet.ClassName.bestGuess("com.example.Generated");
}
@Override
public void addInitializer(org.springframework.aot.generate.MethodReference methodReference) {
initializers.add(methodReference);
}
}
/**
* RegisteredBean.of(beanFactory, name) 要求该 bean 已注册在工厂里,
* 因此先用一个真实的 DefaultListableBeanFactory 注册定义再取。
*/
private static RegisteredBean registeredBean(String name, Class<?> type) {
var factory = new org.springframework.beans.factory.support.DefaultListableBeanFactory();
factory.registerBeanDefinition(name, new RootBeanDefinition(type));
return RegisteredBean.of(factory, name);
}
/** 只提供 applyTo 所需能力的极简生成上下文替身(只用到 runtimeHints)。 */
private record StubGenerationContext(RuntimeHints hints)
implements org.springframework.aot.generate.GenerationContext {
@Override
public RuntimeHints getRuntimeHints() {
return hints;
}
@Override
public org.springframework.aot.generate.GeneratedClasses getGeneratedClasses() {
throw new UnsupportedOperationException("测试不需要生成类");
}
@Override
public org.springframework.aot.generate.GeneratedFiles getGeneratedFiles() {
throw new UnsupportedOperationException("测试不需要生成文件");
}
@Override
public org.springframework.aot.generate.GenerationContext withName(String name) {
return this;
}
}
/** 承载各类返回/参数类型的样例接口,用于驱动类型解析。 */
@SuppressWarnings("unused")
interface Holder {
int plain();
String arrayReturn();
java.util.List<String> genericList();
java.util.Map<String, Integer> genericMap();
void twoArgs(String a, Integer b);
}
}
@@ -0,0 +1,143 @@
package com.lion.lionwebsite.Controller;
import com.lion.lionwebsite.Service.SubService;
import com.lion.lionwebsite.Service.UserService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
/**
* /personal/subBind 与 /personal/user 的路由契约。
* 两个控制器都是直通转调,测试价值在于路径/方法/参数的映射正确
* (方法写错会让前端拿到 405,参数名写错会静默传 null)。
*/
class AccountControllersTest {
private SubService subService;
private UserService userService;
private MockMvc mockMvc;
@BeforeEach
void setUp() {
subService = mock(SubService.class);
userService = mock(UserService.class);
}
private MockMvc subMvc() {
return MockMvcBuilders.standaloneSetup(new SubController(subService)).build();
}
private MockMvc userMvc() {
return MockMvcBuilders.standaloneSetup(new UserController(userService)).build();
}
// ---------- SubController ----------
@Test
void subBindEndpointsRouteToService() throws Exception {
var mvc = subMvc();
when(subService.insertSubBind("alice", 1)).thenReturn("{\"result\":\"success\"}");
mvc.perform(post("/personal/subBind/").param("user", "alice").param("accountId", "1"))
.andExpect(status().isOk());
verify(subService).insertSubBind("alice", 1);
mvc.perform(put("/personal/subBind/").param("user", "alice"));
verify(subService).resetKey("alice");
mvc.perform(get("/personal/subBind/all"));
verify(subService).selectAllSubBind();
mvc.perform(get("/personal/subBind/allRecord"));
verify(subService).SelectAllSubUpdateRecord();
mvc.perform(delete("/personal/subBind/").param("user", "alice"));
verify(subService).deleteSubBind("alice");
mvc.perform(put("/personal/subBind/alice/account").param("accountId", "3"));
verify(subService).rebind("alice", 3);
}
/** 账号增删改查与刷新。 */
@Test
void subscriptionAccountEndpointsRouteToService() throws Exception {
var mvc = subMvc();
mvc.perform(get("/personal/subBind/accounts"));
verify(subService).listSubscriptionAccounts();
mvc.perform(post("/personal/subBind/accounts")
.param("name", "n").param("upstreamKey", "k"));
verify(subService).insertSubscriptionAccount("n", "k", true, true);
mvc.perform(put("/personal/subBind/accounts/7")
.param("name", "n2").param("upstreamKey", "k2"));
verify(subService).updateSubscriptionAccount(7, "n2", "k2", true, true);
mvc.perform(post("/personal/subBind/accounts/7/refresh"));
verify(subService).refreshSubscriptionAccount(7);
mvc.perform(delete("/personal/subBind/accounts/7"));
verify(subService).deleteSubscriptionAccount(7);
}
/** filterHighMultiplier / enabled 的默认值为 true,显式传 false 必须被尊重。 */
@Test
void accountFlagsHonourExplicitValuesAndDefaults() throws Exception {
var mvc = subMvc();
mvc.perform(post("/personal/subBind/accounts")
.param("name", "n").param("upstreamKey", "k")
.param("filterHighMultiplier", "false").param("enabled", "false"));
verify(subService).insertSubscriptionAccount("n", "k", false, false);
// 不传两个开关时使用默认 true(与前端表单默认勾选一致)
mvc.perform(put("/personal/subBind/accounts/9")
.param("name", "n").param("upstreamKey", "k"));
verify(subService).updateSubscriptionAccount(9, "n", "k", true, true);
}
// ---------- UserController ----------
@Test
void userEndpointsRouteToService() throws Exception {
var mvc = userMvc();
mvc.perform(get("/personal/user"));
verify(userService).getAllUser();
mvc.perform(post("/personal/user")
.param("targetAuthCode", "code").param("username", "alice"));
verify(userService).addAuthCode("code", "alice");
mvc.perform(put("/personal/user/AuthCode")
.param("targetAuthCode", "old").param("newAuthCode", "new"));
verify(userService).alterAuthCode("old", "new");
mvc.perform(put("/personal/user/Username")
.param("targetAuthCode", "code").param("newUsername", "bob"));
verify(userService).alterUsername("code", "bob");
mvc.perform(delete("/personal/user").param("targetAuthCode", "code"));
verify(userService).deleteAuthCode("code");
mvc.perform(put("/personal/user/status")
.param("AuthCode", "code").param("isEnable", "false"));
verify(userService).alterStatus("code", false);
}
/** 停用状态的布尔绑定:不传时按 false 处理,这是 Spring 的既有行为。 */
@Test
void userStatusDefaultsToFalseWhenFlagOmitted() throws Exception {
var mvc = userMvc();
mvc.perform(put("/personal/user/status").param("AuthCode", "code"));
verify(userService).alterStatus("code", false);
}
}
@@ -0,0 +1,293 @@
package com.lion.lionwebsite.Controller;
import com.lion.lionwebsite.Service.CollectService;
import com.lion.lionwebsite.Service.GalleryManageService;
import com.lion.lionwebsite.Service.RemoteService;
import com.lion.lionwebsite.Service.UserService;
import com.lion.lionwebsite.Util.Response;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
/**
* /GalleryManage 的 HTTP 契约。
* 这里重点是「参数不全时必须在控制器层就拦住」,不能被透传到服务层去碰数据库;
* 以及 type 分发到正确的查询方法(写错就查错人)。
*/
class GalleryManageControllerTest {
private GalleryManageService galleryManageService;
private CollectService collectService;
private UserService userService;
private RemoteService remoteService;
private MockMvc mockMvc;
@BeforeEach
void setUp() {
galleryManageService = mock(GalleryManageService.class);
collectService = mock(CollectService.class);
userService = mock(UserService.class);
remoteService = mock(RemoteService.class);
mockMvc = MockMvcBuilders.standaloneSetup(new GalleryManageController(
galleryManageService, collectService, userService, remoteService))
// standaloneSetup 默认按 ISO-8859-1 输出字符串,会把中文写成 "?";
// 生产环境 Spring Boot 用 UTF-8,这里对齐以免断言到被破坏的响应体。
.setMessageConverters(new org.springframework.http.converter.StringHttpMessageConverter(
java.nio.charset.StandardCharsets.UTF_8))
.build();
}
// ---------- create_task ----------
/** link 为空必须在控制器层拒绝,不能进服务层。 */
@Test
void createTaskRejectsMissingLink() throws Exception {
mockMvc.perform(post("/GalleryManage").param("targetResolution", "1280x720"))
.andExpect(status().isOk())
.andExpect(content().string(containsFailure()));
verifyNoInteractions(galleryManageService);
}
@Test
void createTaskRejectsMissingResolution() throws Exception {
mockMvc.perform(post("/GalleryManage").param("link", "https://exhentai.org/g/1/abc/"))
.andExpect(status().isOk())
.andExpect(content().string(containsFailure()));
verifyNoInteractions(galleryManageService);
}
@Test
void createTaskDelegatesWhenParametersComplete() throws Exception {
when(galleryManageService.createTask(anyString(), anyString(), anyString()))
.thenReturn("{\"result\":\"success\"}");
mockMvc.perform(post("/GalleryManage")
.param("link", "https://exhentai.org/g/1/abc/")
.param("targetResolution", "1280x720")
.param("AuthCode", "code"))
.andExpect(status().isOk())
.andExpect(content().string("{\"result\":\"success\"}"));
verify(galleryManageService).createTask("https://exhentai.org/g/1/abc/", "1280x720", "code");
}
// ---------- selectGallery 的 type 分发 ----------
/**
* 缺 type 时回「参数不全」,但注意执行顺序:控制器先解析授权码,再校验 type。
* 也就是说非法授权码会在更早处失败(生产上由 TaskHandlerInterceptor 先拒掉),
* 这个顺序本身是契约的一部分,故一并锁定。
*/
@Test
void selectGalleryRejectsMissingTypeAfterResolvingUser() throws Exception {
when(userService.getUserId("code")).thenReturn(7);
var result = mockMvc.perform(get("/GalleryManage").param("AuthCode", "code"))
.andExpect(status().isOk())
.andReturn();
assertTrue(body(result).contains("failure"), "实际输出: " + body(result));
verify(userService).getUserId("code");
verifyNoInteractions(galleryManageService);
}
/** 未知 type 应回「参数错误」,不得落到任何查询分支。 */
@Test
void selectGalleryRejectsUnknownType() throws Exception {
when(userService.getUserId("code")).thenReturn(7);
var result = mockMvc.perform(get("/GalleryManage")
.param("type", "bogus").param("AuthCode", "code"))
.andExpect(status().isOk())
.andReturn();
// MockMvc 默认按 ISO-8859-1 解码响应体,中文需显式按 UTF-8 还原
assertTrue(body(result).contains("参数错误"), "实际输出: " + body(result));
verifyNoInteractions(galleryManageService);
}
/** 每种 type 都要路由到对应方法,且 all/downloader 必须带上解析出的 userId/授权码。 */
@Test
void selectGalleryRoutesEachTypeToItsQuery() throws Exception {
when(userService.getUserId("code")).thenReturn(7);
mockMvc.perform(get("/GalleryManage").param("type", "link")
.param("param", "https://e/g/1/").param("AuthCode", "code"));
verify(galleryManageService).selectTaskByLink("https://e/g/1/");
mockMvc.perform(get("/GalleryManage").param("type", "gid")
.param("param", "123").param("AuthCode", "code"));
verify(galleryManageService).selectTaskByGid(123);
mockMvc.perform(get("/GalleryManage").param("type", "all").param("AuthCode", "code"));
verify(galleryManageService).selectAllGallery(7);
mockMvc.perform(get("/GalleryManage").param("type", "name")
.param("param", "sakura").param("AuthCode", "code"));
verify(galleryManageService).selectGalleryByName("sakura");
mockMvc.perform(get("/GalleryManage").param("type", "downloader").param("AuthCode", "code"));
verify(galleryManageService).selectGalleryByDownloader("code");
}
/** gid 非数字应抛绑定异常而不是静默查 0 号。 */
@Test
void selectGalleryRejectsNonNumericGid() throws Exception {
when(userService.getUserId("code")).thenReturn(7);
try {
mockMvc.perform(get("/GalleryManage")
.param("type", "gid").param("param", "not-a-number").param("AuthCode", "code"));
fail("非数字 gid 应抛出 NumberFormatException");
} catch (Exception e) {
assertInstanceOf(NumberFormatException.class, e.getCause() == null ? e : e.getCause());
}
}
// ---------- deleteTask ----------
// ---------- 下载人查询 ----------
/** 缺 gid 时在控制器层拦住,不进服务层。 */
@Test
void selectDownloaderRejectsMissingGid() throws Exception {
mockMvc.perform(get("/GalleryManage/downloader").param("AuthCode", "code"))
.andExpect(status().isOk())
.andExpect(content().string(containsFailure()));
verifyNoInteractions(galleryManageService);
}
/** 正常请求必须把 gid 与授权码原样传给服务层。 */
@Test
void selectDownloaderDelegatesWithGidAndAuthCode() throws Exception {
when(galleryManageService.selectDownloaderByGid(500, "code")).thenReturn("{}");
mockMvc.perform(get("/GalleryManage/downloader").param("gid", "500").param("AuthCode", "code"))
.andExpect(status().isOk());
verify(galleryManageService).selectDownloaderByGid(500, "code");
}
@Test
void deleteTaskRejectsMissingGid() throws Exception {
mockMvc.perform(delete("/GalleryManage").param("AuthCode", "code"))
.andExpect(status().isOk())
.andExpect(content().string(containsFailure()));
verifyNoInteractions(galleryManageService);
}
@Test
void deleteTaskDelegatesWithGidAndAuthCode() throws Exception {
when(galleryManageService.deleteGalleryByGid(55, "code")).thenReturn("{\"result\":\"success\"}");
mockMvc.perform(delete("/GalleryManage")
.param("gid", "55").param("AuthCode", "code"))
.andExpect(status().isOk())
.andExpect(content().string("{\"result\":\"success\"}"));
verify(galleryManageService).deleteGalleryByGid(55, "code");
}
// ---------- 收藏 ----------
/** 收藏使用授权码解析出的 userId,而不是请求里的任意值。 */
@Test
void collectUsesResolvedUserId() throws Exception {
when(userService.getUserId("code")).thenReturn(7);
when(collectService.collectGallery(9, 7)).thenReturn("{\"result\":\"success\"}");
mockMvc.perform(post("/GalleryManage/collect")
.param("gid", "9").param("AuthCode", "code"))
.andExpect(status().isOk());
verify(collectService).collectGallery(9, 7);
}
@Test
void disCollectUsesResolvedUserId() throws Exception {
when(userService.getUserId("code")).thenReturn(7);
when(collectService.disCollectGallery(9, 7)).thenReturn("{\"result\":\"success\"}");
mockMvc.perform(post("/GalleryManage/disCollect")
.param("gid", "9").param("AuthCode", "code"))
.andExpect(status().isOk());
verify(collectService).disCollectGallery(9, 7);
}
// ---------- 其余直通接口 ----------
@Test
void simpleEndpointsDelegateToService() throws Exception {
mockMvc.perform(post("/GalleryManage/reconnect"));
verify(galleryManageService).reconnect();
mockMvc.perform(post("/GalleryManage/test"));
verify(remoteService).checkAvailability();
mockMvc.perform(get("/GalleryManage/weekUsedAmount"));
verify(galleryManageService).getWeekUsedAmount();
mockMvc.perform(post("/GalleryManage/cache").param("url", "https://e/g/1/x/"));
verify(galleryManageService).cacheImagesKey("https://e/g/1/x/");
mockMvc.perform(post("/GalleryManage/reset"));
verify(galleryManageService).resetUndone();
}
@Test
void retryRejectsMissingGidButAcceptsPresentOne() throws Exception {
mockMvc.perform(post("/GalleryManage/retry"))
.andExpect(status().isOk())
.andExpect(content().string(containsFailure()));
verify(galleryManageService, never()).retryGallery(anyInt());
mockMvc.perform(post("/GalleryManage/retry").param("gid", "12"));
verify(galleryManageService).retryGallery(12);
}
/** 在线图片接口返回 Callable(异步),控制器必须原样交回而不立即执行。 */
@Test
void onlineImageReturnsCallableUnresolved() throws Exception {
java.util.concurrent.atomic.AtomicBoolean invoked = new java.util.concurrent.atomic.AtomicBoolean();
when(galleryManageService.getCachedImage(eq("123"), eq(2), any(), any()))
.thenReturn(() -> {
invoked.set(true);
return null;
});
mockMvc.perform(get("/GalleryManage/onlineImage/2").param("gid", "123"))
.andExpect(request().asyncStarted());
verify(galleryManageService).getCachedImage(eq("123"), eq(2),
any(HttpServletRequest.class), any(HttpServletResponse.class));
}
private static org.hamcrest.Matcher<String> containsFailure() {
return org.hamcrest.Matchers.containsString("\"failure\"");
}
/** 按 UTF-8 还原响应体(MockMvc 默认字符集会把中文解成乱码)。 */
private static String body(org.springframework.test.web.servlet.MvcResult result) throws Exception {
return result.getResponse().getContentAsString(java.nio.charset.StandardCharsets.UTF_8);
}
/** 供断言使用的极小响应体,避免测试里散落魔法字符串。 */
@SuppressWarnings("unused")
private static String failureBody(String reason) {
return Response._failure(reason);
}
}
@@ -0,0 +1,125 @@
package com.lion.lionwebsite.Controller;
import com.lion.lionwebsite.Service.LocalService;
import com.lion.lionwebsite.Service.PersonalService;
import com.lion.lionwebsite.Interceptor.PersonalInterceptor;
import com.lion.lionwebsite.Util.PanelLoginTicket;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.mock.web.MockHttpSession;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
/**
* /personal 的 HTTP 契约。
* 文件管理与分享接口下线后,这里只剩订阅更新时间、家里 IP 与留言转发;
* 权限由 PersonalInterceptor 另外把关(见 PersonalInterceptorTest)。
*/
class PersonalControllerTest {
private PersonalService personalService;
private LocalService localService;
private PanelLoginTicket panelLoginTicket;
private MockMvc mockMvc;
private static final String SECRET = "test-secret-for-login-ticket";
@BeforeEach
void setUp() {
personalService = mock(PersonalService.class);
localService = mock(LocalService.class);
// 用真实票据实现(不是 mock),这样验签与时间窗的开销也在链路里被覆盖。
panelLoginTicket = new PanelLoginTicket(SECRET, 300);
mockMvc = MockMvcBuilders
.standaloneSetup(new PersonalController(personalService, localService, panelLoginTicket))
.build();
}
@Test
void indexRedirectsToIndexPage() throws Exception {
mockMvc.perform(get("/personal/"))
.andExpect(status().is3xxRedirection())
.andExpect(redirectedUrl("/index"));
}
// ---------- 一键登录 ----------
/** 合法票据:建立会话并跳到面板入口。 */
@Test
void loginAcceptsFreshTicketAndStartsSession() throws Exception {
var result = mockMvc.perform(get("/personal/login").param("t", panelLoginTicket.issue()))
.andExpect(status().is3xxRedirection())
.andExpect(redirectedUrl("/index"))
.andReturn();
var session = (MockHttpSession) result.getRequest().getSession(false);
assertNotNull(session, "登录成功必须建立会话");
assertTrue(Boolean.TRUE.equals(session.getAttribute(PersonalInterceptor.SESSION_ATTRIBUTE)));
}
/** 票据缺失、被篡改、或来自未来的时间戳,都必须退回提示页。 */
@Test
void loginRejectsInvalidTickets() throws Exception {
String valid = panelLoginTicket.issue();
for (String bad : new String[]{null, "", "v1.1.2", valid + "x", valid.replace(".", "")}) {
var request = get("/personal/login");
if (bad != null)
request = request.param("t", bad);
mockMvc.perform(request)
.andExpect(status().is3xxRedirection())
.andExpect(redirectedUrl("/denied"))
.andExpect(result -> assertNull(result.getRequest().getSession(false),
"非法票据不得建立会话"));
}
}
/** 退出必须销毁会话。 */
@Test
void logoutInvalidatesSession() throws Exception {
var session = new MockHttpSession();
session.setAttribute(PersonalInterceptor.SESSION_ATTRIBUTE, Boolean.TRUE);
mockMvc.perform(get("/personal/login/logout").session(session))
.andExpect(status().is3xxRedirection())
.andExpect(redirectedUrl("/denied"));
assertTrue(session.isInvalid(), "退出后会话必须失效");
}
@Test
void simpleOperationsDelegateWithTheirParameters() throws Exception {
mockMvc.perform(get("/personal/lastUpdate"));
verify(personalService).lastUpdate();
mockMvc.perform(get("/personal/ip"));
verify(personalService).getIp();
mockMvc.perform(post("/personal/message2me").param("message", "hello"));
verify(personalService).message2me("hello");
}
/** 手动更新订阅:成功与失败必须映射成不同的 result,前端据此提示。 */
@Test
void updateSubReflectsServiceOutcome() throws Exception {
when(localService.updateSub()).thenReturn(true);
var ok = mockMvc.perform(post("/personal/updateSub"))
.andExpect(status().isOk())
.andReturn();
assertTrue(ok.getResponse().getContentAsString().contains("\"result\":\"success\""));
when(localService.updateSub()).thenReturn(false);
var failed = mockMvc.perform(post("/personal/updateSub"))
.andExpect(status().isOk())
.andReturn();
assertTrue(failed.getResponse().getContentAsString().contains("\"result\":\"failure\""));
verify(localService, times(2)).updateSub();
}
}
@@ -0,0 +1,173 @@
package com.lion.lionwebsite.Controller;
import com.lion.lionwebsite.Domain.User;
import com.lion.lionwebsite.Service.PublicService;
import com.lion.lionwebsite.Service.QueryService;
import com.lion.lionwebsite.Service.RemoteService;
import com.lion.lionwebsite.Service.SubService;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
import static org.junit.jupiter.api.Assertions.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
/**
* 公开路由的 HTTP 契约:路径、方法、参数绑定与响应内容。
* 全部用 standaloneSetup(不加载 Spring 上下文、不连数据库、不占端口),
* 拦截器另行单测,这里只锁控制器自身的分发与转发行为。
*/
class PublicControllerTest {
private PublicService publicService;
private RemoteService remoteService;
private SubService subService;
private QueryService queryService;
private MockMvc mockMvc;
@BeforeEach
void setUp() {
publicService = mock(PublicService.class);
remoteService = mock(RemoteService.class);
subService = mock(SubService.class);
queryService = mock(QueryService.class);
mockMvc = MockMvcBuilders
.standaloneSetup(new PublicController(publicService, remoteService, subService, queryService))
.build();
}
@Test
void indexRedirectsToIndexPage() throws Exception {
mockMvc.perform(get("/"))
.andExpect(status().is3xxRedirection())
.andExpect(redirectedUrl("/index"));
}
/** 有 X-Forwarded-For 时以它为准(反向代理后的真实客户端 IP)。 */
@Test
void ipPrefersForwardedForHeader() throws Exception {
mockMvc.perform(get("/ip").header("X-Forwarded-For", "203.0.113.7"))
.andExpect(status().isOk())
.andExpect(content().string("203.0.113.7"));
verify(publicService, never()).logIpAddress(anyString());
}
/** 没有转发头时回退到 remoteAddr。 */
@Test
void ipFallsBackToRemoteAddress() throws Exception {
mockMvc.perform(get("/ip").with(request -> {
request.setRemoteAddr("198.51.100.9");
return request;
}))
.andExpect(status().isOk())
.andExpect(content().string("198.51.100.9"));
}
/** 只有 auth=ip 时才记录家里 IP,其他取值不得写库。 */
@Test
void ipOnlyLogsWhenAuthIsIp() throws Exception {
mockMvc.perform(get("/ip").param("auth", "ip").header("X-Forwarded-For", "203.0.113.7"))
.andExpect(status().isOk());
verify(publicService).logIpAddress("203.0.113.7");
mockMvc.perform(get("/ip").param("auth", "other").header("X-Forwarded-For", "203.0.113.8"))
.andExpect(status().isOk());
verify(publicService, times(1)).logIpAddress(anyString());
}
/** 身份直接内嵌为 JSON 对象,保留 result/data 字段。 */
@Test
void validateReturnsIdentityAndNodeAvailability() throws Exception {
when(publicService.getUserId("code")).thenReturn(new User(7, "code", "alice", null, true));
when(remoteService.isDead()).thenReturn(false);
mockMvc.perform(post("/validate").param("AuthCode", "code"))
.andExpect(status().isOk())
.andExpect(content().string(org.hamcrest.Matchers.containsString("\"userId\":7")))
.andExpect(content().string(org.hamcrest.Matchers.containsString("\"username\":\"alice\"")))
.andExpect(content().string(org.hamcrest.Matchers.containsString("\"isAvailable\":true")))
// 普通用户 isAdmin=false,前端据此隐藏下载人信息。
.andExpect(content().string(org.hamcrest.Matchers.containsString("\"isAdmin\":false")));
}
/** 管理员(id=3)必须带上 isAdmin=true,前端据此显示下载人信息与筛选。 */
@Test
void validateMarksAdminAccount() throws Exception {
when(publicService.getUserId("admin")).thenReturn(new User(3, "admin", "狮子", null, true));
when(remoteService.isDead()).thenReturn(false);
mockMvc.perform(post("/validate").param("AuthCode", "admin"))
.andExpect(status().isOk())
.andExpect(content().string(org.hamcrest.Matchers.containsString("\"isAdmin\":true")));
}
/**
* 用户名含引号/反斜杠时,内层 data 仍必须是合法 JSON。
* 回归:原先手工 String.format 拼接会在这种输入下产出非法 JSON,
* 前端 JSON.parse(res.data.data) 会直接失败,用户卡在登录态。
*/
@Test
void validateEscapesSpecialCharactersInUsername() throws Exception {
when(publicService.getUserId("code")).thenReturn(new User(7, "code", "a\"b\\c", null, true));
when(remoteService.isDead()).thenReturn(false);
var result = mockMvc.perform(post("/validate").param("AuthCode", "code"))
.andExpect(status().isOk())
.andReturn();
// 直接读取结构化 data,特殊字符保持原样。
var outer = com.lion.lionwebsite.Util.CustomUtil.objectMapper
.readTree(result.getResponse().getContentAsString());
var inner = outer.get("data");
assertEquals("a\"b\\c", inner.get("username").asText(), "特殊字符必须原样保留且 JSON 合法");
}
/** 存储节点掉线时 isAvailable 必须为 false,前端据此提示。 */
@Test
void validateReportsUnavailableWhenNodeIsDead() throws Exception {
when(publicService.getUserId("code")).thenReturn(new User(7, "code", "alice", null, true));
when(remoteService.isDead()).thenReturn(true);
mockMvc.perform(post("/validate").param("AuthCode", "code"))
.andExpect(status().isOk())
.andExpect(content().string(org.hamcrest.Matchers.containsString("\"isAvailable\":false")));
}
@Test
void alterAuthCodeDelegatesToService() throws Exception {
when(publicService.alterAuthCode("old", "new")).thenReturn("{\"result\":\"success\"}");
mockMvc.perform(put("/AuthCode").param("AuthCode", "old").param("newAuthCode", "new"))
.andExpect(status().isOk())
.andExpect(content().string("{\"result\":\"success\"}"));
verify(publicService).alterAuthCode("old", "new");
}
/** 订阅分发必须把 client 与 key 原样交给服务层(含路径中的 key)。 */
@Test
void publicSubPassesClientAndKeyToService() throws Exception {
mockMvc.perform(get("/sub/v2/abcd1234"))
.andExpect(status().isOk());
verify(subService).updateSub(any(HttpServletResponse.class), any(HttpServletRequest.class),
eq("v2"), eq("abcd1234"));
}
@Test
void ehThumbnailDelegatesToQueryService() throws Exception {
mockMvc.perform(get("/GalleryManage/ehThumbnail").param("path", "123/abc.jpg"))
.andExpect(status().isOk());
verify(queryService).getEhThumbnail(eq("123/abc.jpg"),
any(HttpServletRequest.class), any(HttpServletResponse.class));
}
}
@@ -0,0 +1,57 @@
package com.lion.lionwebsite.Controller;
import com.lion.lionwebsite.Service.QueryService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
/**
* /query 搜索入口的路由契约:三个可选参数(keyword/prev/next)必须原样透传,
* 因为分页完全依赖 prev/next 拼 URL。
*/
class QueryControllerTest {
private QueryService queryService;
private MockMvc mockMvc;
@BeforeEach
void setUp() {
queryService = mock(QueryService.class);
mockMvc = MockMvcBuilders.standaloneSetup(new QueryController(queryService)).build();
}
@Test
void queryForwardsAllThreeParameters() throws Exception {
when(queryService.query("sakura", "P", "N")).thenReturn("{\"result\":\"success\"}");
mockMvc.perform(get("/query")
.param("keyword", "sakura").param("prev", "P").param("next", "N"))
.andExpect(status().isOk())
.andExpect(content().string("{\"result\":\"success\"}"));
verify(queryService).query("sakura", "P", "N");
}
/** 分页参数缺省时应传 null(服务层据此决定是否拼分页参数)。 */
@Test
void queryPassesNullsForAbsentPagination() throws Exception {
mockMvc.perform(get("/query").param("keyword", "sakura"))
.andExpect(status().isOk());
verify(queryService).query("sakura", null, null);
}
/** 完全不带参数也应正常到达服务层,由它决定如何应对空关键词。 */
@Test
void queryWithoutKeywordReachesService() throws Exception {
mockMvc.perform(get("/query"))
.andExpect(status().isOk());
verify(queryService).query(null, null, null);
}
}
@@ -0,0 +1,219 @@
package com.lion.lionwebsite.Dao.cache;
import com.lion.lionwebsite.Domain.GidToKey;
import com.lion.lionwebsite.Domain.ImageKeyCache;
import org.apache.ibatis.datasource.unpooled.UnpooledDataSource;
import org.apache.ibatis.mapping.Environment;
import org.apache.ibatis.session.Configuration;
import org.apache.ibatis.session.SqlSession;
import org.apache.ibatis.session.SqlSessionFactory;
import org.apache.ibatis.session.SqlSessionFactoryBuilder;
import org.apache.ibatis.transaction.jdbc.JdbcTransactionFactory;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.*;
/**
* ImageCacheMapper 的实际 SQL 验证。
*
* <p>服务层测试用的是内存替身,无法覆盖注解 SQL 本身(尤其是批量 INSERT 的
* {@code <script>} 动态语句与唯一索引冲突)。这里用一个真实的 MyBatis
* SqlSessionFactory 打到临时 SQLite 文件上,不依赖 Spring 容器,也不碰生产配置。
*
* <p>表结构刻意保持与线上一致(两列条件唯一索引),这样「唯一约束确实拦住了重复写入」
* 也是在真实 SQLite 上被验证的。
*/
class ImageCacheMapperSqlTest {
private static final String GID = "1596929";
@TempDir
Path tempDir;
private Path database;
private SqlSessionFactory factory;
@BeforeEach
void setUp() throws Exception {
database = tempDir.resolve("cache.db");
try (Connection connection = DriverManager.getConnection("jdbc:sqlite:" + database);
Statement statement = connection.createStatement()) {
statement.execute("create table gidToKey (gid varchar(20), key varchar(20), pages integer)");
statement.execute("create table ImageKeyCache (gid varchar(20), imgkey varchar(20), page integer)");
// 与迁移脚本最终建立的一致:按 gid 唯一、按 (gid,page) 唯一。
statement.execute("create unique index ux_gidToKey_gid on gidToKey (gid)");
statement.execute("create unique index ux_ImageKeyCache_gid_page on ImageKeyCache (gid, page)");
}
UnpooledDataSource dataSource = new UnpooledDataSource("org.sqlite.JDBC", "jdbc:sqlite:" + database, null, null);
Environment environment = new Environment("cache-test", new JdbcTransactionFactory(), dataSource);
Configuration configuration = new Configuration(environment);
configuration.addMapper(ImageCacheMapper.class);
factory = new SqlSessionFactoryBuilder().build(configuration);
}
@AfterEach
void tearDown() {
factory = null;
}
private <T> T inSession(SessionWork<T> work) {
try (SqlSession session = factory.openSession(true)) {
return work.run(session.getMapper(ImageCacheMapper.class));
}
}
private interface SessionWork<T> {
T run(ImageCacheMapper mapper);
}
// ---------- 批量写入 ----------
@Test
void batchInsertWritesEveryRowIncludingMultipleBatches() {
List<ImageKeyCache> rows = new ArrayList<>();
for (int page = 1; page <= 250; page++)
rows.add(page(GID, page, "k" + page));
inSession(mapper -> {
mapper.insertGidToKey(header(GID, "f08534d87d", 250));
for (int start = 0; start < rows.size(); start += 200) {
int end = Math.min(start + 200, rows.size());
mapper.insertImageKeyCacheBatch(new ArrayList<>(rows.subList(start, end)));
}
return null;
});
inSession(mapper -> {
assertEquals(250, mapper.countImageKeyCacheByGid(GID));
assertEquals(250, mapper.selectPageNumbersByGid(GID).size());
assertEquals("k1", mapper.selectImageKeyCacheByGidAndPage(GID, 1).getImgkey());
assertEquals("k250", mapper.selectImageKeyCacheByGidAndPage(GID, 250).getImgkey());
return null;
});
}
@Test
void pageNumbersComeBackAscendingEvenWhenRowsWereInsertedOutOfOrder() {
inSession(mapper -> {
mapper.insertGidToKey(header(GID, "key", 3));
mapper.insertImageKeyCacheBatch(List.of(
page(GID, 3, "c"), page(GID, 1, "a"), page(GID, 2, "b")));
return null;
});
inSession(mapper -> {
assertEquals(List.of(1, 2, 3), mapper.selectPageNumbersByGid(GID));
return null;
});
}
// ---------- 完整性辅助查询 ----------
@Test
void invalidImageKeyCountCatchesNullAndBlank() {
inSession(mapper -> {
mapper.insertGidToKey(header(GID, "key", 3));
mapper.insertImageKeyCacheBatch(List.of(
page(GID, 1, "ok"), page(GID, 2, " "), page(GID, 3, "")));
assertEquals(2, mapper.countInvalidImageKeysByGid(GID),
"空白与空串都必须被算作无效 imgkey");
return null;
});
}
@Test
void pageNumbersForUnknownGidIsEmptyNotNull() {
inSession(mapper -> {
assertNotNull(mapper.selectPageNumbersByGid("999"));
assertTrue(mapper.selectPageNumbersByGid("999").isEmpty());
assertNull(mapper.selectKeyByGid("999"));
return null;
});
}
// ---------- 唯一约束 ----------
@Test
void uniqueIndexRejectsDuplicatePageAndDuplicateHeader() {
inSession(mapper -> {
mapper.insertGidToKey(header(GID, "key", 1));
mapper.insertImageKeyCacheBatch(List.of(page(GID, 1, "a")));
return null;
});
assertThrows(RuntimeException.class, () -> inSession(mapper -> {
mapper.insertImageKeyCacheBatch(List.of(page(GID, 1, "duplicate")));
return null;
}), "重复 (gid,page) 必须被唯一索引拒绝");
assertThrows(RuntimeException.class, () -> inSession(mapper -> {
mapper.insertGidToKey(header(GID, "other", 1));
return null;
}), "重复 gid 必须被唯一索引拒绝");
}
@Test
void deleteByGidRemovesBothHeaderAndPages() {
inSession(mapper -> {
mapper.insertGidToKey(header(GID, "key", 2));
mapper.insertImageKeyCacheBatch(List.of(page(GID, 1, "a"), page(GID, 2, "b")));
return null;
});
inSession(mapper -> {
mapper.deleteImageKeyCacheByGid(GID);
mapper.deleteGidToKey(GID);
return null;
});
inSession(mapper -> {
assertNull(mapper.selectKeyByGid(GID));
assertEquals(0, mapper.countImageKeyCacheByGid(GID));
return null;
});
}
@Test
void singleRowInsertStillWorksForLegacyCallers() {
inSession(mapper -> {
mapper.insertGidToKey(header(GID, "key", 1));
mapper.insertImageKeyCache(page(GID, 1, "legacy"));
return null;
});
inSession(mapper -> {
assertEquals("legacy", mapper.selectImageKeyCacheByGidAndPage(GID, 1).getImgkey());
return null;
});
}
// ---------- 辅助 ----------
private static GidToKey header(String gid, String key, int pages) {
GidToKey header = new GidToKey();
header.setGid(gid);
header.setKey(key);
header.setPages(pages);
return header;
}
private static ImageKeyCache page(String gid, int page, String imgkey) {
ImageKeyCache row = new ImageKeyCache();
row.setGid(gid);
row.setPage(page);
row.setImgkey(imgkey);
return row;
}
}
@@ -0,0 +1,66 @@
package com.lion.lionwebsite.Filter;
import com.lion.lionwebsite.Dao.normal.UserMapper;
import jakarta.servlet.FilterChain;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.mock.web.MockHttpServletRequest;
import org.springframework.mock.web.MockHttpServletResponse;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
/**
* AccessFilter 挂在 /validate 上,用于更新用户最后访问时间。
* 关键约束:只有带了 AuthCode 才落库并继续过滤链;没带时必须直接返回
* (不调用 chain.doFilter),否则匿名请求会污染访问时间统计。
*/
class AccessFilterTest {
private UserMapper userMapper;
private AccessFilter filter;
private FilterChain chain;
@BeforeEach
void setUp() {
userMapper = mock(UserMapper.class);
filter = new AccessFilter(userMapper);
chain = mock(FilterChain.class);
}
/** 带授权码:记录访问时间并放行。 */
@Test
void recordsLastAccessTimeAndContinues() throws Exception {
var request = new MockHttpServletRequest();
request.setParameter("AuthCode", "code-1");
var response = new MockHttpServletResponse();
filter.doFilter(request, response, chain);
verify(userMapper).updateLastAccessTime(anyString(), eq("code-1"));
verify(chain).doFilter(request, response);
}
/** 不带授权码:既不写库也不继续过滤链。 */
@Test
void missingAuthCodeStopsChainWithoutWriting() throws Exception {
var request = new MockHttpServletRequest();
var response = new MockHttpServletResponse();
filter.doFilter(request, response, chain);
verifyNoInteractions(userMapper);
verify(chain, never()).doFilter(any(), any());
}
/** 空字符串授权码视为非法,同样不写库。 */
@Test
void emptyAuthCodeIsIgnored() throws Exception {
var request = new MockHttpServletRequest();
request.setParameter("AuthCode", "");
filter.doFilter(request, new MockHttpServletResponse(), chain);
verify(userMapper).updateLastAccessTime(anyString(), eq(""));
}
}
@@ -0,0 +1,183 @@
package com.lion.lionwebsite.Interceptor;
import com.lion.lionwebsite.Dao.normal.UserMapper;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.mock.web.MockHttpServletRequest;
import org.springframework.mock.web.MockHttpServletResponse;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
/**
* 拦截器是授权之外的第二道闸门:
* - HumanInterceptor 决定无 UA 的请求能否进入首页(挡机器人);
* - PersonalInterceptor 决定能否触达 /personal 与 /remote(必须 AuthCode=alone)。
* 两者都返回 boolean,一旦写错方向就是「全放行」或「全拦截」,后果极端,
* 所以每个分支都单独锁定。
*/
class InterceptorsTest {
// ---------- HumanInterceptor ----------
private HumanInterceptor human;
@BeforeEach
void setUp() {
human = new HumanInterceptor();
}
@Test
void humanAllowsRequestWithUserAgent() {
var request = new MockHttpServletRequest();
request.addHeader("User-Agent", "Mozilla/5.0");
assertTrue(human.preHandle(request, new MockHttpServletResponse(), new Object()));
}
/** 无 User-Agent 一律拒绝(爬虫通常不带)。 */
@Test
void humanRejectsRequestWithoutUserAgent() {
var request = new MockHttpServletRequest();
assertFalse(human.preHandle(request, new MockHttpServletResponse(), new Object()));
}
/** 空字符串也算「有 UA 头」,按现状放行(与 null 区分)。 */
@Test
void humanTreatsEmptyHeaderAsPresent() {
var request = new MockHttpServletRequest();
request.addHeader("User-Agent", "");
assertTrue(human.preHandle(request, new MockHttpServletResponse(), new Object()));
}
// ---------- PersonalInterceptor ----------
private PersonalInterceptor personal;
@BeforeEach
void setUpPersonal() {
personal = new PersonalInterceptor();
}
/** 只有 AuthCode=alone 才放行。 */
@Test
void personalAllowsOnlyAloneAuthCode() {
var ok = new MockHttpServletRequest();
ok.setParameter("AuthCode", "alone");
assertTrue(personal.preHandle(ok, new MockHttpServletResponse(), new Object()));
}
@Test
void personalRejectsMissingOrDifferentAuthCode() {
var missing = new MockHttpServletRequest();
assertFalse(personal.preHandle(missing, new MockHttpServletResponse(), new Object()));
var wrong = new MockHttpServletRequest();
wrong.setParameter("AuthCode", "user-code");
assertFalse(personal.preHandle(wrong, new MockHttpServletResponse(), new Object()));
var empty = new MockHttpServletRequest();
empty.setParameter("AuthCode", "");
assertFalse(personal.preHandle(empty, new MockHttpServletResponse(), new Object()));
}
/** 大小写敏感:ALONE 不是 alone。 */
@Test
void personalIsCaseSensitive() {
var request = new MockHttpServletRequest();
request.setParameter("AuthCode", "ALONE");
assertFalse(personal.preHandle(request, new MockHttpServletResponse(), new Object()));
}
/** 带已登录会话时无需 AuthCode 即可放行(一键登录换来的会话)。 */
@Test
void personalAllowsAuthenticatedSessionWithoutAuthCode() {
var request = new MockHttpServletRequest();
request.getSession(true).setAttribute(PersonalInterceptor.SESSION_ATTRIBUTE, Boolean.TRUE);
assertTrue(personal.preHandle(request, new MockHttpServletResponse(), new Object()));
}
/** 会话存在但没有登录标记时不得放行:只有服务端写入的标记才算数。 */
@Test
void personalRejectsSessionWithoutMarker() {
var request = new MockHttpServletRequest();
request.getSession(true);
assertFalse(personal.preHandle(request, new MockHttpServletResponse(), new Object()));
}
/** 拒绝时必须回 401,前端据此提示去机器人要新链接。 */
@Test
void personalReturnsUnauthorizedWhenRejected() {
var response = new MockHttpServletResponse();
assertFalse(personal.preHandle(new MockHttpServletRequest(), response, new Object()));
assertEquals(401, response.getStatus());
}
// ---------- TaskHandlerInterceptor ----------
/**
* 授权码集合在启动时加载一次,之后靠 updateAuthCodes() 刷新。
* 这里重点验证:合法码放行、非法/缺失码拒绝、刷新后立即生效。
*/
@Test
void taskHandlerAcceptsKnownCodeAndRejectsOthers() {
var userMapper = mock(UserMapper.class);
when(userMapper.selectAllAuthCode()).thenReturn(new String[]{"code-a", "code-b"});
var interceptor = new TaskHandlerInterceptor(userMapper);
interceptor.init();
var valid = new MockHttpServletRequest();
valid.setParameter("AuthCode", "code-a");
assertTrue(interceptor.preHandle(valid, new MockHttpServletResponse(), new Object()));
var invalid = new MockHttpServletRequest();
invalid.setParameter("AuthCode", "code-x");
assertFalse(interceptor.preHandle(invalid, new MockHttpServletResponse(), new Object()));
var absent = new MockHttpServletRequest();
assertFalse(interceptor.preHandle(absent, new MockHttpServletResponse(), new Object()));
}
/** updateAuthCodes 取的是「启用」集合;刷新后旧码必须立即失效。 */
@Test
void taskHandlerRefreshTakesEffectImmediately() {
var userMapper = mock(UserMapper.class);
when(userMapper.selectAllAuthCode()).thenReturn(new String[]{"old-code"});
when(userMapper.selectEnableAuthCode()).thenReturn(new String[]{"new-code"});
var interceptor = new TaskHandlerInterceptor(userMapper);
interceptor.init();
var stale = new MockHttpServletRequest();
stale.setParameter("AuthCode", "old-code");
assertTrue(interceptor.preHandle(stale, new MockHttpServletResponse(), new Object()));
interceptor.updateAuthCodes();
assertFalse(interceptor.preHandle(stale, new MockHttpServletResponse(), new Object()),
"刷新后旧授权码应立即失效");
var fresh = new MockHttpServletRequest();
fresh.setParameter("AuthCode", "new-code");
assertTrue(interceptor.preHandle(fresh, new MockHttpServletResponse(), new Object()));
}
/** 数据库无任何授权码时,任何请求都必须被拒(不能因空数组而误放行)。 */
@Test
void taskHandlerRejectsEverythingWhenNoCodesExist() {
var userMapper = mock(UserMapper.class);
when(userMapper.selectAllAuthCode()).thenReturn(new String[0]);
var interceptor = new TaskHandlerInterceptor(userMapper);
interceptor.init();
var request = new MockHttpServletRequest();
request.setParameter("AuthCode", "any");
assertFalse(interceptor.preHandle(request, new MockHttpServletResponse(), new Object()));
}
}
@@ -0,0 +1,249 @@
package com.lion.lionwebsite.Maintenance;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.*;
/**
* 离线体检与修复工具的行为约束。
*
* <p>用临时 SQLite 文件建出真实表结构,因此这里验证的是工具实际发出的 SQL
* (含判定条件与删除范围),而不是内存替身。
*/
class GalleryIndexMaintenanceTest {
private static final String SCHEMA = """
create table gidToKey (gid varchar(20), key varchar(20), pages integer);
create table ImageKeyCache (gid varchar(20), imgkey varchar(20), page integer);
""";
@TempDir
Path tempDir;
private Path database;
private Connection open() throws SQLException {
return DriverManager.getConnection("jdbc:sqlite:" + database);
}
private void exec(Connection connection, String sql) throws SQLException {
try (Statement statement = connection.createStatement()) {
statement.executeUpdate(sql);
}
}
/** 建一份「一好三坏」的缓存库:正常索引、半截索引、重复 gid、孤儿页 key。 */
private void seed() throws Exception {
database = tempDir.resolve("cache.db");
try (Connection connection = open()) {
for (String statement : SCHEMA.split(";"))
if (!statement.isBlank())
exec(connection, statement);
// 1. 正常:3 页,页码 1..3
exec(connection, "insert into gidToKey values ('100', 'sk100', 3)");
exec(connection, "insert into ImageKeyCache values ('100', 'a', 1)");
exec(connection, "insert into ImageKeyCache values ('100', 'b', 2)");
exec(connection, "insert into ImageKeyCache values ('100', 'c', 3)");
// 2. 半截:声称 3 页,只有 1、3 两页
exec(connection, "insert into gidToKey values ('200', 'sk200', 3)");
exec(connection, "insert into ImageKeyCache values ('200', 'x', 1)");
exec(connection, "insert into ImageKeyCache values ('200', 'y', 3)");
// 3. 重复 gid:两条头部记录
exec(connection, "insert into gidToKey values ('300', 'sk300', 1)");
exec(connection, "insert into gidToKey values ('300', 'sk300b', 1)");
exec(connection, "insert into ImageKeyCache values ('300', 'z', 1)");
// 4. 孤儿页 key:没有头部记录
exec(connection, "insert into ImageKeyCache values ('400', 'orphan', 1)");
}
}
private String scalar(Connection connection, String sql) throws SQLException {
try (Statement statement = connection.createStatement();
ResultSet rows = statement.executeQuery(sql)) {
return rows.next() ? rows.getString(1) : null;
}
}
@Test
void dryRunInspectReportsProblemsWithoutWriting() throws Exception {
seed();
String before;
try (Connection connection = open()) {
before = scalar(connection, "select count(*) || '/' || (select count(*) from ImageKeyCache) from gidToKey");
}
GalleryIndexMaintenance.Report report;
try (Connection connection = open()) {
report = GalleryIndexMaintenance.inspect(connection);
assertFalse(report.isHealthy());
assertEquals(4, report.headerRows, "gidToKey 应有 4 行(含重复 gid)");
assertEquals(7, report.pageRows);
assertEquals(1, report.duplicateHeaderGids, "gid=300 是重复组");
assertEquals(List.of("200", "300"), report.badGids, "半截 200 与重复 300 都应被判为待修复");
assertEquals(List.of("400"), report.orphanGids);
assertTrue(report.badReasons.get("200").contains("页 key 行数"),
"原因应说明行数不符,实际: " + report.badReasons.get("200"));
// gid=200 的页码是 1 和 3:区间端点看起来正常,必须靠「去重后个数」这条判断抓出来。
assertTrue(report.badReasons.get("200").contains("页码去重后"),
"原因应说明页码数量不足,实际: " + report.badReasons.get("200"));
assertTrue(report.badReasons.get("300").contains("页 key 行数")
|| report.badReasons.get("300").contains("重复"),
"重复 gid 必须给出可读原因,实际: " + report.badReasons.get("300"));
assertTrue(report.render("体检结果").contains("需要修复"));
}
try (Connection connection = open()) {
assertEquals(before, scalar(connection,
"select count(*) || '/' || (select count(*) from ImageKeyCache) from gidToKey"),
"体检必须是只读的");
}
}
@Test
void repairRemovesOnlyBrokenGroupsAndLeavesHealthyIndexIntact() throws Exception {
seed();
try (Connection connection = open()) {
GalleryIndexMaintenance.Report before = GalleryIndexMaintenance.inspect(connection);
connection.setAutoCommit(false);
int touched = GalleryIndexMaintenance.repair(connection, before.badGids, before.orphanGids);
connection.commit();
assertEquals(3, touched, "应处理 200、300、400 三个 gid");
GalleryIndexMaintenance.Report after = GalleryIndexMaintenance.inspect(connection);
assertTrue(after.isHealthy(), "修复后应健康,原因: " + after.render("修复后"));
assertEquals(1, after.headerRows, "只应剩下正常索引 gid=100");
assertEquals(3, after.pageRows);
assertTrue(after.badGids.isEmpty());
assertTrue(after.orphanGids.isEmpty());
// 正常索引必须逐行保留,不能被误删。
assertEquals("3", scalar(connection, "select pages from gidToKey where gid='100'"));
assertEquals("a", scalar(connection, "select imgkey from ImageKeyCache where gid='100' and page=1"));
assertEquals("c", scalar(connection, "select imgkey from ImageKeyCache where gid='100' and page=3"));
assertEquals("0", scalar(connection, "select count(*) from ImageKeyCache where gid in ('200','300','400')"));
}
}
@Test
void emptyDatabaseIsHealthy() throws Exception {
database = tempDir.resolve("empty.db");
try (Connection connection = open()) {
for (String statement : SCHEMA.split(";"))
if (!statement.isBlank())
exec(connection, statement);
GalleryIndexMaintenance.Report report = GalleryIndexMaintenance.inspect(connection);
assertTrue(report.isHealthy());
assertEquals(0, report.headerRows);
}
}
@Test
void missingTablesAreReportedInsteadOfCrashing() throws Exception {
database = tempDir.resolve("nothing.db");
try (Connection connection = open()) {
GalleryIndexMaintenance.Report report = GalleryIndexMaintenance.inspect(connection);
assertFalse(report.isHealthy());
assertFalse(report.hasHeaderTable);
assertFalse(report.hasPageTable);
assertTrue(report.render("体检结果").contains("缺少缓存表"));
}
}
@Test
void productionPathsAreRecognisedSoTheToolCanRefuseToWrite() {
assertTrue(GalleryIndexMaintenance.isProductionPath(Path.of("/home/lionwebsite/cache.db")));
assertTrue(GalleryIndexMaintenance.isProductionPath(Path.of("/home/lionwebsite/sub/deep/cache.db")));
assertFalse(GalleryIndexMaintenance.isProductionPath(Path.of("/tmp/cache.db")));
assertFalse(GalleryIndexMaintenance.isProductionPath(tempDir.resolve("cache.db")));
}
@Test
void gapInPageNumbersIsDetectedEvenWhenRowCountMatches() throws Exception {
// 2 行、page 分别是 1 和 3:只比行数会漏判。
database = tempDir.resolve("gap.db");
try (Connection connection = open()) {
for (String statement : SCHEMA.split(";"))
if (!statement.isBlank())
exec(connection, statement);
exec(connection, "insert into gidToKey values ('500', 'sk500', 2)");
exec(connection, "insert into ImageKeyCache values ('500', 'p1', 1)");
exec(connection, "insert into ImageKeyCache values ('500', 'p3', 3)");
GalleryIndexMaintenance.Report report = GalleryIndexMaintenance.inspect(connection);
assertFalse(report.isHealthy());
assertEquals(List.of("500"), report.badGids);
}
}
@Test
void blankImgKeyMakesGroupUnhealthy() throws Exception {
database = tempDir.resolve("blank.db");
try (Connection connection = open()) {
for (String statement : SCHEMA.split(";"))
if (!statement.isBlank())
exec(connection, statement);
exec(connection, "insert into gidToKey values ('600', 'sk600', 2)");
exec(connection, "insert into ImageKeyCache values ('600', 'ok', 1)");
exec(connection, "insert into ImageKeyCache values ('600', ' ', 2)");
GalleryIndexMaintenance.Report report = GalleryIndexMaintenance.inspect(connection);
assertEquals(List.of("600"), report.badGids);
assertTrue(report.badReasons.get("600").contains("imgkey 为空"),
"实际: " + report.badReasons.get("600"));
}
}
@Test
void reportRenderingIsReadableAndTruncatesLongLists() throws Exception {
database = tempDir.resolve("many.db");
try (Connection connection = open()) {
for (String statement : SCHEMA.split(";"))
if (!statement.isBlank())
exec(connection, statement);
for (int i = 0; i < 25; i++)
exec(connection, "insert into gidToKey values ('7" + i + "', 'k', 1)");
GalleryIndexMaintenance.Report report = GalleryIndexMaintenance.inspect(connection);
String rendered = report.render("体检结果");
assertEquals(25, report.badGids.size());
assertTrue(rendered.contains("其余 5 个略"), "应截断超长列表,实际:\n" + rendered);
}
}
/** 修复是幂等的:对已健康的库再跑一次不应改变任何东西。 */
@Test
void repairIsIdempotent() throws Exception {
seed();
try (Connection connection = open()) {
GalleryIndexMaintenance.Report first = GalleryIndexMaintenance.inspect(connection);
connection.setAutoCommit(false);
GalleryIndexMaintenance.repair(connection, first.badGids, first.orphanGids);
connection.commit();
GalleryIndexMaintenance.Report second = GalleryIndexMaintenance.inspect(connection);
connection.setAutoCommit(false);
int touched = GalleryIndexMaintenance.repair(connection, second.badGids, second.orphanGids);
connection.commit();
assertEquals(0, touched, "已健康的库再次修复不应删除任何东西");
assertTrue(GalleryIndexMaintenance.inspect(connection).isHealthy());
}
}
}
@@ -0,0 +1,39 @@
package com.lion.lionwebsite.Message;
import tools.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Test;
import java.util.Set;
import java.util.HashSet;
import static org.junit.jupiter.api.Assertions.*;
/** Both ends assert this exact wire shape; helper getters must not leak onto the wire. */
class ReliableWireContractTest {
private final ObjectMapper mapper = new ObjectMapper();
private void fields(Object message, Set<String> allowed) throws Exception {
var tree = mapper.readTree(mapper.writeValueAsString(message));
Set<String> actual = new HashSet<>();
tree.propertyNames().forEach(actual::add);
assertEquals(allowed, actual);
}
@Test void commandWireShapeAndGoldenInput() throws Exception {
var command = new TaskCommandMessage();
command.setMessageId(71); command.setOperationId("op-download");
command.setAction("DOWNLOAD"); command.setGid(17); command.setName("中文画廊");
fields(command, Set.of("messageType","messageId","operationId","action","gid","name"));
var input = mapper.readValue("{\"messageType\":11,\"messageId\":71,\"operationId\":\"op-download\",\"action\":\"DOWNLOAD\",\"gid\":17,\"name\":\"中文画廊\"}", TaskCommandMessage.class);
assertEquals(command.getOperationId(), input.getOperationId());
assertEquals(command.getGid(), input.getGid());
}
@Test void reportWireShapeAndGoldenInput() throws Exception {
var report = mapper.readValue("{\"messageType\":12,\"messageId\":72,\"reportId\":\"report-1\",\"revision\":9007199254,\"galleryTasks\":[{\"gid\":17,\"name\":\"中文画廊\",\"status\":4,\"proceeding\":20}],\"operationIds\":{\"17\":\"op-download\"}}", TaskReportMessage.class);
fields(report, Set.of("messageType","messageId","reportId","revision","galleryTasks","operationIds"));
assertEquals(9007199254L, report.getRevision());
assertEquals("op-download", report.getOperationIds().get(17));
assertEquals(20, report.getGalleryTasks()[0].getProceeding());
}
@Test void ackWireShapeAndGoldenInput() throws Exception {
var ack = mapper.readValue("{\"messageType\":13,\"messageId\":73,\"reportId\":\"report-1\",\"result\":0}", TaskReportAckMessage.class);
fields(ack, Set.of("messageType","messageId","reportId","result"));
assertEquals(0, ack.getResult());
}
}
@@ -0,0 +1,152 @@
package com.lion.lionwebsite.Message;
import com.lion.lionwebsite.Domain.GalleryTask;
import io.netty.buffer.ByteBuf;
import io.netty.channel.embedded.EmbeddedChannel;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import java.nio.charset.StandardCharsets;
import java.util.LinkedHashMap;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.*;
/**
* 协议 v2(类型 10..13)的线格式。
*
* <p>旧类型 0..9 的字段布局不变(旧节点 Jackson 会拒绝未知字段),新类型沿用
* 「1 字节类型 + 4 字节大端长度 + JSON」的既有帧格式,因此这里同时锁住帧头与字段名,
* 作为与 storageNode 的契约样本。
*/
class TaskProtocolV2CodecTest {
private EmbeddedChannel channel;
@AfterEach
void tearDown() {
if (channel != null) channel.finishAndReleaseAll();
}
private EmbeddedChannel channel() {
channel = new EmbeddedChannel(new MessageCodec());
return channel;
}
private static String frameBody(ByteBuf frame) {
byte[] body = new byte[frame.getInt(1)];
frame.getBytes(5, body);
return new String(body, StandardCharsets.UTF_8);
}
private static <T> T roundTrip(EmbeddedChannel ch, AbstractMessage message, Class<T> type) {
assertTrue(ch.writeOutbound(message), "消息应被编码");
ByteBuf frame = ch.readOutbound();
assertEquals(message.messageType, frame.getByte(0), "帧首字节应为 messageType");
assertEquals(frame.getInt(1), frameBody(frame).getBytes(StandardCharsets.UTF_8).length,
"帧头长度须等于正文实际字节数");
assertTrue(ch.writeInbound(frame));
Object decoded = ch.readInbound();
assertInstanceOf(type, decoded);
return type.cast(decoded);
}
/** 能力协商:协议版本与特性列表按合同字段名传输。 */
@Test
void helloCarriesVersionAndFeatures() {
ProtocolHelloMessage hello = new ProtocolHelloMessage();
hello.setMessageId(7);
ProtocolHelloMessage decoded = roundTrip(channel(), hello, ProtocolHelloMessage.class);
assertEquals(2, decoded.getProtocolVersion());
assertTrue(decoded.getFeatures().contains("reliable-task-v2"));
assertTrue(decoded.supportsReliableTasks());
assertEquals(7, decoded.getMessageId(), "既有 messageId 字段必须保留");
}
/** 未声明 v2 的对端不得被当作可靠协议可用。 */
@Test
void helloWithoutV2FeatureDoesNotSupportReliableTasks() {
ProtocolHelloMessage legacy = new ProtocolHelloMessage();
legacy.setProtocolVersion(1);
legacy.setFeatures(java.util.List.of());
assertFalse(legacy.supportsReliableTasks());
}
/** 任务命令:稳定 operationId + 动作 + gid + 名称。 */
@Test
void taskCommandCarriesStableOperationIdAndAction() {
TaskCommandMessage command = new TaskCommandMessage();
command.setMessageId(11);
command.setOperationId("dl-abc");
command.setAction(TaskCommandMessage.ACTION_DELETE);
command.setGid(998);
command.setName("sample [998]");
TaskCommandMessage decoded = roundTrip(channel(), command, TaskCommandMessage.class);
assertEquals("dl-abc", decoded.getOperationId());
assertEquals("DELETE", decoded.getAction());
assertEquals(998, decoded.getGid());
assertEquals("sample [998]", decoded.getName());
}
/** 状态上报:reportId、revision、任务数组与 per-gid 的 operationId 映射。 */
@Test
void taskReportCarriesRevisionTasksAndOperationIds() {
GalleryTask task = new GalleryTask();
task.setGid(5);
task.setName("g [5]");
task.setStatus(GalleryTask.DOWNLOADING);
task.setProceeding(3);
Map<Integer, String> operationIds = new LinkedHashMap<>();
operationIds.put(5, "dl-5");
TaskReportMessage report = new TaskReportMessage();
report.setReportId("r-1");
report.setRevision(42);
report.setGalleryTasks(new GalleryTask[]{task});
report.setOperationIds(operationIds);
TaskReportMessage decoded = roundTrip(channel(), report, TaskReportMessage.class);
assertEquals("r-1", decoded.getReportId());
assertEquals(42L, decoded.getRevision());
assertEquals(1, decoded.getGalleryTasks().length);
assertEquals(5, decoded.getGalleryTasks()[0].getGid());
assertEquals(GalleryTask.DOWNLOADING, decoded.getGalleryTasks()[0].getStatus());
assertEquals("dl-5", decoded.getOperationIds().get(5));
}
/** 回执:只带 reportId 与结果码。 */
@Test
void reportAckCarriesReportIdAndResult() {
TaskReportAckMessage ack = new TaskReportAckMessage("r-9", TaskReportAckMessage.RESULT_SUCCESS);
ack.setMessageId(13);
TaskReportAckMessage decoded = roundTrip(channel(), ack, TaskReportAckMessage.class);
assertEquals("r-9", decoded.getReportId());
assertEquals((byte) 0, decoded.getResult());
assertEquals(13, decoded.getMessageId());
}
/** 旧类型(下载投递)必须保持原样:不能因为新增 v2 类型而改动它的字段。 */
@Test
void legacyDownloadPostMessageKeepsOriginalShape() {
GalleryTask task = new GalleryTask();
task.setGid(1);
task.setName("legacy");
DownloadPostMessage message = new DownloadPostMessage();
message.setGalleryTask(task);
assertTrue(channel().writeOutbound(message));
ByteBuf frame = channel.readOutbound();
assertEquals(AbstractMessage.DOWNLOAD_POST_MESSAGE, frame.getByte(0));
String json = frameBody(frame);
assertTrue(json.contains("galleryTask"), "旧字段名不能改: " + json);
assertFalse(json.contains("operationId"), "旧类型不得被塞入新字段: " + json);
}
}
@@ -0,0 +1,67 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CollectMapper;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
/**
* 画廊收藏与取消收藏。规则很简单,但「重复收藏」与「取消未收藏」两条分支
* 决定落库与否,必须逐一锁死。
*/
class CollectServiceTest {
private CollectMapper collectMapper;
private CollectService service;
@BeforeEach
void setUp() {
collectMapper = mock(CollectMapper.class);
service = new CollectService(collectMapper);
}
private static boolean ok(String json) {
return json.contains("\"result\":\"success\"");
}
@Test
void collectWritesWhenNotYetCollected() {
when(collectMapper.isCollect(100, 5)).thenReturn(0);
assertTrue(ok(service.collectGallery(100, 5)));
verify(collectMapper).collect(100, 5);
}
/** 重复收藏必须拒绝且不得重复落库。 */
@Test
void collectRejectsDuplicate() {
when(collectMapper.isCollect(100, 5)).thenReturn(1);
String json = service.collectGallery(100, 5);
assertFalse(ok(json));
assertTrue(json.contains("已经收藏了"));
verify(collectMapper, never()).collect(anyInt(), anyInt());
}
@Test
void disCollectRemovesWhenCollected() {
when(collectMapper.isCollect(100, 5)).thenReturn(1);
assertTrue(ok(service.disCollectGallery(100, 5)));
verify(collectMapper).disCollect(100, 5);
}
/** 取消一个没收藏的画廊必须拒绝,且不得落库。 */
@Test
void disCollectRejectsWhenNotCollected() {
when(collectMapper.isCollect(100, 5)).thenReturn(0);
String json = service.disCollectGallery(100, 5);
assertFalse(ok(json));
assertTrue(json.contains("没有收藏该图片"));
verify(collectMapper, never()).disCollect(anyInt(), anyInt());
}
}
@@ -0,0 +1,696 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
import com.lion.lionwebsite.Domain.GidToKey;
import com.lion.lionwebsite.Domain.ImageKeyCache;
import com.lion.lionwebsite.Exception.GalleryIndexException;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.transaction.support.SimpleTransactionStatus;
import org.springframework.transaction.support.TransactionCallback;
import org.springframework.transaction.support.TransactionOperations;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import static org.junit.jupiter.api.Assertions.*;
/**
* 图片索引服务的行为约束。
*
* <p>这些用例不依赖 Spring 容器、MyBatis 与网络:DB 用内存假实现,
* 事务用带快照回滚的假实现,上游用可控的假解析器。
* 因此它们验证的是服务自己的编排逻辑,而不是 SQLite 的行为。
*/
class GalleryIndexServiceTest {
private static final String URL = "https://exhentai.org/g/1596929/f08534d87d/";
private static final String GID = "1596929";
private static final String KEY = "f08534d87d";
private FakeMapper mapper;
/** 写事务(发布索引)探针。 */
private FakeTransactions transactions;
/** 读事务(快照读取)探针,独立计数,避免和写事务混淆。 */
private FakeTransactions reads;
private GalleryIndexService service;
@BeforeEach
void setUp() {
mapper = new FakeMapper();
transactions = new FakeTransactions(mapper);
reads = new FakeTransactions(mapper, true);
mapper.transactionsForProbe = transactions;
service = new GalleryIndexService(mapper, transactions, reads, url -> mapper.fetch());
}
// ---------- 正常路径 ----------
@Test
void buildsOnceThenServesFromCache() throws Exception {
GidToKey index = service.ensureIndex(URL);
assertEquals(GID, index.getGid());
assertEquals(KEY, index.getKey());
assertEquals(3, index.getPages());
assertEquals(1, mapper.fetchCount.get(), "首次访问应抓取上游");
assertEquals(1, transactions.commits.get(), "应只发布一次");
GidToKey again = service.ensureIndex(URL);
assertEquals(3, again.getPages());
assertEquals(1, mapper.fetchCount.get(), "二次访问必须命中缓存,不得再抓上游");
assertEquals(1, transactions.commits.get(), "命中缓存不应触发写入");
}
@Test
void concurrentCallersShareOneUpstreamFetch() throws Exception {
int callers = 24;
ExecutorService pool = Executors.newFixedThreadPool(callers);
CountDownLatch start = new CountDownLatch(1);
List<Future<GidToKey>> futures = new ArrayList<>();
for (int i = 0; i < callers; i++) {
futures.add(pool.submit(() -> {
start.await();
return service.ensureIndex(URL);
}));
}
start.countDown();
for (Future<GidToKey> future : futures) {
GidToKey index = future.get(20, TimeUnit.SECONDS);
assertEquals(GID, index.getGid());
assertEquals(3, index.getPages());
}
pool.shutdownNow();
assertEquals(1, mapper.fetchCount.get(), "同 gid 必须合并为一次上游抓取");
assertEquals(1, transactions.commits.get(), "同 gid 必须只发布一次");
}
@Test
void upstreamFetchRunsOutsideTransactionAndAllWritesInsideIt() throws Exception {
service.ensureIndex(URL);
assertEquals(0, mapper.upstreamSawTransaction.get(),
"抓取上游时不应有任何事务在执行,否则会长时间占住 SQLite 写锁");
assertTrue(mapper.insertSawTransaction.get() > 0, "写库必须在事务内");
assertEquals(0, mapper.mutationsOutsideTransaction.get(),
"任何一次删除或写入都不得落在事务之外,否则整组替换就不是原子的");
}
@Test
void wholeReplacementIsOneTransaction() throws Exception {
// 先放一份不完整索引,强制走一次完整重建。
mapper.putHeader(GID, KEY, 9);
mapper.putPage(GID, 1, "old");
service.ensureIndex(URL);
assertEquals(1, transactions.commits.get(), "删除加写入必须在同一个事务里提交");
assertEquals(1, mapper.transactionAttempts.get(), "不应出现多个独立写事务");
}
@Test
void cacheHitReadsInsideOneSnapshotAndOpensNoWriteTransaction() throws Exception {
service.ensureIndex(URL);
int writesAfterBuild = transactions.started.get();
int readsAfterBuild = reads.started.get();
service.ensureIndex(URL);
assertEquals(writesAfterBuild, transactions.started.get(),
"命中缓存不应创建任何写事务");
assertEquals(readsAfterBuild + 1, reads.started.get(),
"每次命中缓存都应恰好用一个读事务读取头部与页码");
}
@Test
void missFirstLooksUpUnderReadSnapshotThenRechecksInsideTheBuildCriticalSection() throws Exception {
service.ensureIndex(URL);
// 首次构建:先做一次一致性读确认未命中,进入临界区后再确认一次。
assertTrue(reads.started.get() >= 2,
"构建前与进入临界区后都应各自读一次快照,实际: " + reads.started.get());
assertEquals(1, transactions.started.get(),
"无论读了几次,写事务只能有一个");
}
// ---------- 索引完整性 ----------
@Test
void incompleteIndexIsRebuilt() throws Exception {
// 头部声称 3 页,实际只有 1、3 两页:半截且跳号。
mapper.putHeader(GID, KEY, 3);
mapper.putPage(GID, 1, "old1");
mapper.putPage(GID, 3, "old3");
GidToKey index = service.ensureIndex(URL);
assertEquals(3, index.getPages());
assertEquals(1, mapper.fetchCount.get(), "半截索引必须重新抓取");
assertEquals(List.of(1, 2, 3), mapper.pageNumbers(GID));
assertEquals("k1", mapper.pageKey(GID, 1));
}
@Test
void pageNumberGapIsTreatedAsIncompleteEvenWhenRowCountMatches() throws Exception {
// 行数刚好等于 pages,但页码是 1 和 3:只比行数会误判为完整。
mapper.putHeader(GID, KEY, 2);
mapper.putPage(GID, 1, "old1");
mapper.putPage(GID, 3, "old3");
mapper.pagesForFetch = List.of("n1", "n2");
GidToKey index = service.ensureIndex(URL);
assertEquals(1, mapper.fetchCount.get(), "页码跳号必须触发重建");
assertEquals(2, index.getPages());
assertEquals(List.of(1, 2), mapper.pageNumbers(GID));
}
@Test
void blankImgKeyIsTreatedAsIncomplete() throws Exception {
mapper.putHeader(GID, KEY, 3);
mapper.putPage(GID, 1, "ok");
mapper.putPage(GID, 2, " ");
mapper.putPage(GID, 3, "ok3");
service.ensureIndex(URL);
assertEquals(1, mapper.fetchCount.get(), "空 imgkey 必须触发重建");
assertEquals("k2", mapper.pageKey(GID, 2));
}
@Test
void headerWithoutPagesOrKeyIsTreatedAsIncomplete() throws Exception {
mapper.putHeader(GID, KEY, 0);
service.ensureIndex(URL);
assertEquals(1, mapper.fetchCount.get(), "pages 非法必须触发重建");
mapper.clear();
mapper.putHeader(GID, " ", 3);
service.ensureIndex(URL);
assertEquals(2, mapper.fetchCount.get(), "key 为空也必须触发重建");
}
// ---------- 上游畸形数据(必须在写事务之前拦下) ----------
@Test
void malformedUpstreamRowsAreRejectedBeforeAnyTransaction() throws Exception {
// 先放一份旧索引,用来断言畸形数据不会把它改掉。
mapper.putHeader(GID, KEY, 2);
mapper.putPage(GID, 1, "old1");
mapper.putPage(GID, 2, "old2");
mapper.forceIncomplete(GID);
Map<String, String> before = mapper.stateFingerprint();
List<List<ImageKeyCache>> malformedCases = List.of(
rows(row(GID, 1, "k"), row("999999", 2, "k")), // gid 不一致
rows(row(GID, 1, "k"), row(GID, 1, "k2")), // 页码重复
rows(row(GID, 1, "k"), row(GID, 3, "k2")), // 页码跳号
rows(row(GID, 1, "k"), row(GID, 2, " ")), // imgkey 为空
rows(row(GID, 0, "k"), row(GID, 1, "k2")), // 页码从 0 开始
rows(row(GID, 1, "k"), row(GID, 5, "k2"))); // 页码越界
for (List<ImageKeyCache> malformed : malformedCases) {
mapper.rawRowsForFetch = malformed;
GalleryIndexException error = assertThrows(GalleryIndexException.class,
() -> service.ensureIndex(URL), "应拒绝畸形数据: " + describe(malformed));
assertEquals(GalleryIndexException.Reason.MALFORMED, error.getReason(),
"应归为上游数据畸形: " + describe(malformed));
assertFalse(error.isPermanent(), "上游数据畸形可以重试");
assertEquals(0, transactions.commits.get(), "畸形数据不得写入任何东西");
assertEquals(0, transactions.rollbacks.get(), "应在开启写事务之前就失败");
assertEquals(before, mapper.stateFingerprint(), "旧索引必须原样保留: " + describe(malformed));
}
}
@Test
void validationRunsBeforeTheWriteTransactionOpens() throws Exception {
mapper.rawRowsForFetch = rows(row(GID, 1, "k"), row(GID, 1, "k2"));
assertThrows(GalleryIndexException.class, () -> service.ensureIndex(URL));
assertEquals(0, mapper.transactionAttempts.get(),
"畸形数据应在事务创建之前就被拒绝,不产生任何写事务");
assertEquals(0, mapper.mutationsOutsideTransaction.get(),
"更不允许事务外写入");
}
@Test
void wellFormedButOutOfOrderRowsAreAccepted() throws Exception {
// 上游行序无保证:页码乱序但集合正确时应当接受。
mapper.rawRowsForFetch = rows(row(GID, 3, "c"), row(GID, 1, "a"), row(GID, 2, "b"));
GidToKey index = service.ensureIndex(URL);
assertEquals(3, index.getPages());
assertEquals(1, mapper.headerCount(GID));
assertEquals("b", mapper.pageKey(GID, 2));
}
@Test
void failedPublishRollsBackAndLeavesExistingIndexUntouched() throws Exception {
// 旧的半截索引:头部 3 页,实际只有 1、2 两页。它会触发重建。
mapper.putHeader(GID, KEY, 3);
mapper.putPage(GID, 1, "old1");
mapper.putPage(GID, 2, "old2");
Map<String, String> before = mapper.stateFingerprint();
// 重建时上游给了新 key,但第一次批量写入就失败。
mapper.pagesForFetch = List.of("n1", "n2", "n3");
mapper.failBatchCallAt.set(1);
GalleryIndexException error = assertThrows(GalleryIndexException.class,
() -> service.ensureIndex(URL));
assertEquals(GalleryIndexException.Reason.TRANSIENT, error.getReason(),
"写库失败应被归为可重试的暂时错误");
assertEquals(1, transactions.rollbacks.get(), "失败必须回滚");
assertEquals(0, transactions.commits.get(), "失败不应留下已提交事务");
assertEquals(before, mapper.stateFingerprint(),
"回滚后必须与失败前完全一致,既不能被清空,也不能只写一半");
}
@Test
void upstreamFailureIsReportedAsTransientAndWritesNothing() {
mapper.upstreamError = new RuntimeException("network down");
GalleryIndexException error = assertThrows(GalleryIndexException.class,
() -> service.ensureIndex(URL));
assertEquals(GalleryIndexException.Reason.TRANSIENT, error.getReason());
assertEquals(0, transactions.commits.get());
assertEquals(0, transactions.rollbacks.get(), "尚未开始写入,不应有事务");
assertNull(mapper.header(GID));
}
@Test
void unavailableGalleryIsReportedAsPermanent() {
mapper.pagesForFetch = List.of();
GalleryIndexException error = assertThrows(GalleryIndexException.class,
() -> service.ensureIndex(URL));
assertEquals(GalleryIndexException.Reason.UNAVAILABLE, error.getReason());
assertTrue(error.isPermanent(), "画廊已下架不应被上层当成可重试错误");
assertEquals(0, transactions.commits.get());
}
@Test
void malformedLinkIsRejectedBeforeAnyWork() {
for (String bad : new String[]{"abc", "https://exhentai.org/g/1/", null, "https://exhentai.org/g/x/y/"}) {
GalleryIndexException error = assertThrows(GalleryIndexException.class,
() -> service.ensureIndex(bad), "输入: " + bad);
assertEquals(GalleryIndexException.Reason.INVALID_LINK, error.getReason(), "输入: " + bad);
}
assertEquals(0, mapper.fetchCount.get(), "链接错误不得触发上游抓取");
assertEquals(0, transactions.commits.get());
}
@Test
void failureIsSharedByConcurrentWaiters() throws Exception {
mapper.upstreamError = new RuntimeException("network down");
ExecutorService pool = Executors.newFixedThreadPool(8);
CountDownLatch start = new CountDownLatch(1);
List<Future<String>> outcomes = new ArrayList<>();
for (int i = 0; i < 8; i++) {
outcomes.add(pool.submit(() -> {
start.await();
try {
service.ensureIndex(URL);
return "SUCCESS";
} catch (GalleryIndexException e) {
return e.getReason().name();
}
}));
}
start.countDown();
for (Future<String> outcome : outcomes)
assertEquals("TRANSIENT", outcome.get(20, TimeUnit.SECONDS),
"失败的构建结果也必须被等待者共享,而不是各自重试");
pool.shutdownNow();
}
/** 重试语义:同一 gid 失败后再次调用应真的重试,而不是永久返回失败。 */
@Test
void retryAfterFailureFetchesAgain() throws Exception {
mapper.upstreamError = new RuntimeException("network down");
assertThrows(GalleryIndexException.class, () -> service.ensureIndex(URL));
assertEquals(1, mapper.fetchCount.get());
mapper.upstreamError = null;
GidToKey index = service.ensureIndex(URL);
assertEquals(2, mapper.fetchCount.get(), "失败不应被永久缓存,必须允许重试");
assertEquals(3, index.getPages());
assertEquals(3, mapper.pageNumbers(GID).size());
}
// ---------- 批量写入分片 ----------
@Test
void largeIndexIsWrittenInBatches() throws Exception {
int pages = GalleryIndexService.BATCH_SIZE * 2 + 7;
List<String> keys = new ArrayList<>();
for (int i = 0; i < pages; i++) keys.add("k" + i);
mapper.pagesForFetch = keys;
GidToKey index = service.ensureIndex(URL);
assertEquals(pages, index.getPages());
assertEquals(3, mapper.batchCalls.get(), "应按 BATCH_SIZE 分片写入");
assertEquals(pages, mapper.pageNumbers(GID).size());
assertEquals("k0", mapper.pageKey(GID, 1));
}
@Test
void republishReplacesPreviousRowsInsteadOfAccumulating() throws Exception {
service.ensureIndex(URL);
assertEquals(3, mapper.pageNumbers(GID).size());
// 上游改版:同一 gid 变成 5 页。
mapper.pagesForFetch = List.of("b1", "b2", "b3", "b4", "b5");
mapper.forceIncomplete(GID);
GidToKey index = service.ensureIndex(URL);
assertEquals(5, index.getPages());
assertEquals(5, mapper.pageNumbers(GID).size(), "旧页 key 必须被替换掉,不能残留");
assertEquals(1, mapper.headerCount(GID), "gidToKey 必须仍只有一条记录");
assertEquals(List.of(1, 2, 3, 4, 5), mapper.pageNumbers(GID));
}
/** 构造一行上游页 key,便于在用例里拼畸形数据。 */
private static ImageKeyCache row(String gid, int page, String imgkey) {
ImageKeyCache row = new ImageKeyCache();
row.setGid(gid);
row.setPage(page);
row.setImgkey(imgkey);
return row;
}
private static List<ImageKeyCache> rows(ImageKeyCache... items) {
return new ArrayList<>(Arrays.asList(items));
}
private static String describe(List<ImageKeyCache> rows) {
StringBuilder out = new StringBuilder();
for (ImageKeyCache row : rows) {
if (out.length() > 0) out.append(", ");
out.append(row.getGid()).append('/').append(row.getPage())
.append('/').append(row.getImgkey());
}
return "[" + out + "]";
}
// ---------- 测试替身 ----------
/**
* 记录事务边界与写入所在状态的假事务。
*
* <p>写事务会做真实快照并支持回滚;读事务({@code readOnly=true})只标记一个快照区间,
* 不计入提交/回滚次数,否则无法把「发布事务」和「读快照」分开断言。
*/
static final class FakeTransactions implements TransactionOperations {
private final FakeMapper mapper;
final AtomicInteger active = new AtomicInteger();
final AtomicInteger commits = new AtomicInteger();
final AtomicInteger rollbacks = new AtomicInteger();
final AtomicInteger started = new AtomicInteger();
private final boolean readOnly;
FakeTransactions(FakeMapper mapper) {
this(mapper, false);
}
FakeTransactions(FakeMapper mapper, boolean readOnly) {
this.mapper = mapper;
this.readOnly = readOnly;
}
@Override
public <T> T execute(TransactionCallback<T> action) {
started.incrementAndGet();
active.incrementAndGet();
if (readOnly) {
try {
return action.doInTransaction(new SimpleTransactionStatus());
} finally {
active.decrementAndGet();
}
}
mapper.transactionAttempts.incrementAndGet();
Map<String, Object> before = mapper.snapshot();
try {
T result = action.doInTransaction(new SimpleTransactionStatus());
commits.incrementAndGet();
return result;
} catch (RuntimeException | Error e) {
mapper.restore(before);
rollbacks.incrementAndGet();
throw e;
} finally {
active.decrementAndGet();
}
}
}
/** 内存版 ImageCacheMapper,无需数据库,且对并发访问安全。 */
static final class FakeMapper implements ImageCacheMapper {
private final Map<String, GidToKey> headers = Collections.synchronizedMap(new LinkedHashMap<>());
private final List<ImageKeyCache> pages = Collections.synchronizedList(new ArrayList<>());
final AtomicInteger fetchCount = new AtomicInteger();
final AtomicInteger batchCalls = new AtomicInteger();
final AtomicInteger transactionAttempts = new AtomicInteger();
final AtomicInteger upstreamSawTransaction = new AtomicInteger();
final AtomicInteger insertSawTransaction = new AtomicInteger();
final AtomicInteger mutationsOutsideTransaction = new AtomicInteger();
List<String> pagesForFetch = List.of("k1", "k2", "k3");
/** 设置后 fetch 直接返回这些行,用于构造上游畸形数据。 */
List<ImageKeyCache> rawRowsForFetch;
RuntimeException upstreamError;
/** 第几次批量写入时抛异常;小于等于 0 表示不失败。 */
final AtomicInteger failBatchCallAt = new AtomicInteger(-1);
FakeTransactions transactionsForProbe;
List<ImageKeyCache> fetch() {
fetchCount.incrementAndGet();
if (transactionsForProbe != null)
upstreamSawTransaction.set(transactionsForProbe.active.get());
if (upstreamError != null)
throw upstreamError;
if (rawRowsForFetch != null)
return new ArrayList<>(rawRowsForFetch);
List<ImageKeyCache> result = new ArrayList<>();
int page = 1;
for (String key : pagesForFetch) {
ImageKeyCache row = new ImageKeyCache();
row.setGid(GID);
row.setPage(page++);
row.setImgkey(key);
result.add(row);
}
return result;
}
/** 任何写操作都调用它:事务外写入会被计数,用于断言原子性。 */
private void noteWrite() {
if (transactionsForProbe == null)
return;
if (transactionsForProbe.active.get() == 0)
mutationsOutsideTransaction.incrementAndGet();
else
insertSawTransaction.set(transactionsForProbe.active.get());
}
void putHeader(String gid, String key, int pageCount) {
GidToKey header = new GidToKey();
header.setGid(gid);
header.setKey(key);
header.setPages(pageCount);
headers.put(gid, header);
}
void putPage(String gid, int page, String imgkey) {
ImageKeyCache row = new ImageKeyCache();
row.setGid(gid);
row.setPage(page);
row.setImgkey(imgkey);
pages.add(row);
}
/** 把头部页数改成与现有页 key 不符,使下一次调用必定重建。 */
void forceIncomplete(String gid) {
GidToKey header = headers.get(gid);
if (header != null)
header.setPages(header.getPages() + 99);
}
GidToKey header(String gid) {
return headers.get(gid);
}
int headerCount(String gid) {
return headers.containsKey(gid) ? 1 : 0;
}
String pageKey(String gid, int page) {
return pageSnapshot().stream()
.filter(row -> row.getGid().equals(gid) && row.getPage() == page)
.map(ImageKeyCache::getImgkey)
.findFirst().orElse(null);
}
List<Integer> pageNumbers(String gid) {
List<Integer> numbers = new ArrayList<>();
pageSnapshot().stream().filter(row -> row.getGid().equals(gid))
.forEach(row -> numbers.add(row.getPage()));
numbers.sort(Integer::compareTo);
return numbers;
}
void clear() {
headers.clear();
pages.clear();
}
/** 状态指纹:用于断言回滚后与失败前完全一致。 */
Map<String, String> stateFingerprint() {
Map<String, String> state = new LinkedHashMap<>();
synchronized (headers) {
headers.forEach((gid, header) ->
state.put("h:" + gid, header.getKey() + "/" + header.getPages()));
}
for (ImageKeyCache row : pageSnapshot())
state.put("p:" + row.getGid() + ":" + row.getPage(), String.valueOf(row.getImgkey()));
return state;
}
Map<String, Object> snapshot() {
Map<String, Object> state = new LinkedHashMap<>();
synchronized (headers) {
headers.forEach((gid, header) -> {
GidToKey copy = new GidToKey();
copy.setGid(header.getGid());
copy.setKey(header.getKey());
copy.setPages(header.getPages());
state.put("h:" + gid, copy);
});
}
state.put("pages", new ArrayList<>(pageSnapshot()));
return state;
}
@SuppressWarnings("unchecked")
void restore(Map<String, Object> state) {
headers.clear();
pages.clear();
for (Map.Entry<String, Object> entry : state.entrySet()) {
if (entry.getKey().startsWith("h:"))
headers.put(entry.getKey().substring(2), (GidToKey) entry.getValue());
}
pages.addAll((List<ImageKeyCache>) state.get("pages"));
}
/** 加锁复制一份页 key,避免在并发流操作时读到中间状态。 */
private List<ImageKeyCache> pageSnapshot() {
synchronized (pages) {
List<ImageKeyCache> copy = new ArrayList<>(pages.size());
for (ImageKeyCache row : pages)
copy.add(copyOf(row));
return copy;
}
}
private static ImageKeyCache copyOf(ImageKeyCache row) {
ImageKeyCache copy = new ImageKeyCache();
copy.setGid(row.getGid());
copy.setPage(row.getPage());
copy.setImgkey(row.getImgkey());
return copy;
}
@Override
public void insertImageKeyCache(ImageKeyCache imageKeyCache) {
noteWrite();
pages.add(imageKeyCache);
}
@Override
public void insertGidToKey(GidToKey gidToKey) {
noteWrite();
headers.put(gidToKey.getGid(), gidToKey);
}
@Override
public ImageKeyCache selectImageKeyCacheByGidAndPage(String gid, int page) {
return pageSnapshot().stream()
.filter(row -> row.getGid().equals(gid) && row.getPage() == page)
.findFirst().orElse(null);
}
@Override
public GidToKey selectKeyByGid(String gid) {
return headers.get(gid);
}
@Override
public int countImageKeyCacheByGid(String gid) {
return (int) pageSnapshot().stream().filter(row -> row.getGid().equals(gid)).count();
}
@Override
public void deleteImageKeyCacheByGid(String gid) {
noteWrite();
synchronized (pages) {
pages.removeIf(row -> row.getGid().equals(gid));
}
}
@Override
public void deleteGidToKey(String gid) {
noteWrite();
headers.remove(gid);
}
@Override
public List<Integer> selectPageNumbersByGid(String gid) {
List<Integer> numbers = new ArrayList<>();
pageSnapshot().stream().filter(row -> row.getGid().equals(gid))
.forEach(row -> numbers.add(row.getPage()));
numbers.sort(Integer::compareTo);
return numbers;
}
@Override
public int countInvalidImageKeysByGid(String gid) {
return (int) pageSnapshot().stream()
.filter(row -> row.getGid().equals(gid))
.filter(row -> row.getImgkey() == null || row.getImgkey().isBlank())
.count();
}
@Override
public int insertImageKeyCacheBatch(List<ImageKeyCache> batch) {
noteWrite();
int call = batchCalls.incrementAndGet();
if (failBatchCallAt.get() > 0 && call == failBatchCallAt.get())
throw new RuntimeException("db down");
pages.addAll(batch);
return batch.size();
}
}
}
@@ -35,7 +35,7 @@ class GalleryManageServiceTest {
remote = mock(RemoteService.class); remote = mock(RemoteService.class);
push = mock(PushService.class); push = mock(PushService.class);
service = new GalleryManageService(galleries, collectMapper, service = new GalleryManageService(galleries, collectMapper,
configurationMapper, users, mock(ShareFileMapper.class), configurationMapper, users,
mock(ImageCacheMapper.class), remote, push); mock(ImageCacheMapper.class), remote, push);
User user = new User(); User user = new User();
@@ -44,6 +44,31 @@ class GalleryManageServiceTest {
when(users.selectUserByAuthCode("code")).thenReturn(user); when(users.selectUserByAuthCode("code")).thenReturn(user);
} }
@Test
void cacheApiReturnsStructuredIndexWithoutDoubleEncoding() throws Exception {
service.galleryIndexService = mock(GalleryIndexService.class);
var index = new com.lion.lionwebsite.Domain.GidToKey();
index.setGid("123"); index.setKey("key"); index.setPages(20);
when(service.galleryIndexService.ensureIndex("link")).thenReturn(index);
var response = com.lion.lionwebsite.Util.CustomUtil.objectMapper.readTree(service.cacheImagesKey("link"));
assertEquals("success", response.get("result").asText());
assertTrue(response.get("data").isObject());
assertEquals(20, response.get("data").get("pages").asInt());
}
@Test
void cacheApiMapsPermanentAndTransientFailures() throws Exception {
service.galleryIndexService = mock(GalleryIndexService.class);
var invalid = new com.lion.lionwebsite.Exception.GalleryIndexException(
com.lion.lionwebsite.Exception.GalleryIndexException.Reason.INVALID_LINK, "invalid");
when(service.galleryIndexService.ensureIndex("bad")).thenThrow(invalid);
assertTrue(service.cacheImagesKey("bad").contains("链接错误"));
var failed = new com.lion.lionwebsite.Exception.GalleryIndexException(
com.lion.lionwebsite.Exception.GalleryIndexException.Reason.TRANSIENT, "retry");
when(service.galleryIndexService.ensureIndex("retry")).thenThrow(failed);
assertTrue(service.cacheImagesKey("retry").contains("网络波动或其他异常"));
}
// ---------- createTask 输入校验 ---------- // ---------- createTask 输入校验 ----------
/** 链接第 5 段非数字时应返回「链接错误」且不落库、不下发节点。 */ /** 链接第 5 段非数字时应返回「链接错误」且不落库、不下发节点。 */
@@ -150,6 +175,21 @@ class GalleryManageServiceTest {
} }
} }
/**
* 回归:GalleryUtil.parseGid 以前只捕 IndexOutOfBoundsException,
* 「非数字 gid」与「null」都会穿透成 500。按链接查询会直接把用户输入喂进来,
* 这里锁住真实实现(不 mock)在这些输入下都安全返回失败。
*/
@Test
void selectTaskByLinkHandlesMalformedGidWithoutThrowing() {
for (String bad : new String[]{"https://exhentai.org/g/not-a-number/key/", "garbage", null}) {
String response = assertDoesNotThrow(() -> service.selectTaskByLink(bad),
"畸形链接不应抛异常,实际输入: " + bad);
assertFalse(response.contains("\"result\":\"success\""), "实际输出: " + response);
}
verify(galleries, never()).selectGalleryByGid(anyInt());
}
@Test @Test
void selectTaskByGidReturnsFailureWhenAbsent() { void selectTaskByGidReturnsFailureWhenAbsent() {
when(galleries.selectGalleryByGid(404)).thenReturn(null); when(galleries.selectGalleryByGid(404)).thenReturn(null);
@@ -372,6 +412,72 @@ class GalleryManageServiceTest {
assertFalse(service.retryGallery(2).contains("\"result\":\"success\"")); assertFalse(service.retryGallery(2).contains("\"result\":\"success\""));
} }
// ---------- 生命周期拒绝路径 ----------
/** beginDelete 被拒(任务已在删除中/提交中)时必须回业务失败,且不得退避到裸删除。 */
@Test
void deleteRefusedByLifecycleDoesNotFallBackToRawDelete() {
Gallery gallery = new Gallery();
gallery.setGid(897);
gallery.setName("mine [897]");
gallery.setDownloader(7);
when(galleries.selectGalleryByGid(897)).thenReturn(gallery);
when(collectMapper.selectCollectorByGid(897)).thenReturn(new java.util.ArrayList<>());
service.lifecycleService = mock(GalleryTaskLifecycleService.class);
when(service.lifecycleService.beginDelete(897))
.thenReturn(GalleryTaskLifecycleService.DeleteIntent.refused("任务正在提交中,请等待提交结果明确后再删除"));
String response = service.deleteGalleryByGid(897, "code");
assertFalse(response.contains("\"result\":\"success\""), "被拒时必须回失败: " + response);
assertTrue(response.contains("提交中"), response);
verify(remote, never()).deleteGallery(any());
verify(service.lifecycleService, never()).completeDelete(anyInt(), anyString());
}
/** reset 只重投可安全下发的任务,跳过需要人工对账的 UNKNOWN 等。 */
@Test
void resetSkipsTasksThatMustNotBeDispatched() {
when(remote.isDead()).thenReturn(false);
Gallery safe = new Gallery();
safe.setGid(900);
safe.setName("safe [900]");
Gallery unknown = new Gallery();
unknown.setGid(901);
unknown.setName("unknown [901]");
when(galleries.selectUnDoneGalleries()).thenReturn(new Gallery[]{safe, unknown});
service.lifecycleService = mock(GalleryTaskLifecycleService.class);
when(service.lifecycleService.resolveDispatch(900))
.thenReturn(GalleryTaskLifecycleService.DispatchDecision.ready("dl-900"));
when(service.lifecycleService.resolveDispatch(901))
.thenReturn(GalleryTaskLifecycleService.DispatchDecision.refuse());
String response = service.resetUndone();
assertTrue(response.contains("\"result\":\"success\""), response);
verify(remote).addGalleryToQueue(safe);
verify(remote, never()).addGalleryToQueue(unknown);
}
/** 提交未知的任务重试必须被拒,提示人工对账,且不触碰节点。 */
@Test
void retryRefusesWhenLifecycleRejects() {
Gallery gallery = new Gallery();
gallery.setGid(902);
gallery.setName("unknown [902]");
gallery.setStatus("提交未知");
when(galleries.selectGalleryByGid(902)).thenReturn(gallery);
service.lifecycleService = mock(GalleryTaskLifecycleService.class);
when(service.lifecycleService.resolveDispatch(902))
.thenReturn(GalleryTaskLifecycleService.DispatchDecision.refuse());
String response = service.retryGallery(902);
assertFalse(response.contains("\"result\":\"success\""), response);
assertTrue(response.contains("人工对账"), response);
verify(remote, never()).retryGallery(any());
}
// ---------- 列表 ---------- // ---------- 列表 ----------
/** 用量查询应返回格式化后的已用量与上次重置时间。 */ /** 用量查询应返回格式化后的已用量与上次重置时间。 */
@@ -0,0 +1,492 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
import com.lion.lionwebsite.Dao.normal.*;
import com.lion.lionwebsite.Domain.Gallery;
import com.lion.lionwebsite.Domain.User;
import com.lion.lionwebsite.Util.GalleryUtil;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.util.ArrayList;
import java.util.List;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
/**
* 任务查询层与其收藏标记、未完成任务重投。
* 这些方法决定前端列表里「哪些是已收藏」「哪些任务能重试」,
* 以及重置时会不会把不该重投的任务再推给节点(会重复下载、浪费额度)。
*/
class GalleryQueryTest {
private GalleryMapper galleries;
private CollectMapper collectMapper;
private UserMapper users;
private RemoteService remote;
private GalleryManageService service;
@BeforeEach
void setUp() {
galleries = mock(GalleryMapper.class);
collectMapper = mock(CollectMapper.class);
users = mock(UserMapper.class);
remote = mock(RemoteService.class);
service = new GalleryManageService(galleries, collectMapper,
mock(CustomConfigurationMapper.class), users,
mock(ImageCacheMapper.class), remote, mock(PushService.class));
}
private static Gallery gallery(int gid, String name, String status) {
Gallery g = new Gallery();
g.setGid(gid);
g.setName(name);
g.setStatus(status);
return g;
}
private static boolean ok(String json) {
return json.contains("\"result\":\"success\"");
}
// ---------- selectAllGallery ----------
/** 无收藏时返回全部任务,且不误标 collect。 */
@Test
void selectAllGalleryWithoutCollections() {
Gallery[] all = {gallery(1, "A", "下载中"), gallery(2, "B", "下载完成")};
when(galleries.selectAllGallery()).thenReturn(all);
when(collectMapper.selectGidByCollector(7)).thenReturn(new ArrayList<>());
String json = service.selectAllGallery(7);
assertTrue(ok(json), "实际输出: " + json);
assertTrue(json.contains("A"));
assertTrue(json.contains("B"));
assertFalse(all[0].isCollect(), "无收藏时不应标记为已收藏");
}
/** 收藏过的任务必须被标记 collect=true,其余保持 false。 */
@Test
void selectAllGalleryMarksCollectedGalleries() {
Gallery[] all = {gallery(1, "A", "x"), gallery(2, "B", "x"), gallery(3, "C", "x")};
when(galleries.selectAllGallery()).thenReturn(all);
when(collectMapper.selectGidByCollector(7))
.thenReturn(new ArrayList<>(List.of(1, 3)));
String json = service.selectAllGallery(7);
assertTrue(ok(json));
assertTrue(all[0].isCollect(), "gid=1 应标记已收藏");
assertFalse(all[1].isCollect(), "gid=2 未收藏");
assertTrue(all[2].isCollect(), "gid=3 应标记已收藏");
}
/** 收藏列表非空但没有任何一项命中任务列表时,不应标记任何任务。 */
@Test
void selectAllGalleryIgnoresCollectionsForMissingGalleries() {
Gallery[] all = {gallery(1, "A", "x")};
when(galleries.selectAllGallery()).thenReturn(all);
when(collectMapper.selectGidByCollector(7))
.thenReturn(new ArrayList<>(List.of(999)));
assertTrue(ok(service.selectAllGallery(7)));
assertFalse(all[0].isCollect());
}
/**
* 收藏标记改为集合查找后的批量正确性:每个画廊只依据自己的 gid 是否在收藏集合中。
* 用「任务远多于收藏」的规模锁住「不漏标、不误标」两点。
*/
@Test
void selectAllGalleryMarksCollectionsCorrectlyAtScale() {
int total = 500;
Gallery[] all = new Gallery[total];
for (int i = 0; i < total; i++)
all[i] = gallery(i, "G" + i, "下载中");
when(galleries.selectAllGallery()).thenReturn(all);
// 收藏偶数 gid,构造一半命中、一半不命中
ArrayList<Integer> collected = new ArrayList<>();
for (int i = 0; i < total; i += 2)
collected.add(i);
when(collectMapper.selectGidByCollector(7)).thenReturn(collected);
assertTrue(ok(service.selectAllGallery(7)));
for (int i = 0; i < total; i++)
assertEquals(i % 2 == 0, all[i].isCollect(), "gid=" + i + " 的收藏标记不正确");
}
/** 查询结果为 null 时应回业务失败而不是 NPE。 */
@Test
void selectAllGalleryReportsFailureWhenNull() {
when(galleries.selectAllGallery()).thenReturn(null);
String json = service.selectAllGallery(7);
assertFalse(ok(json));
assertTrue(json.contains("没有找到图片"));
}
/** 空数组应正常返回成功(与 null 区别对待)。 */
@Test
void selectAllGalleryHandlesEmptyArray() {
when(galleries.selectAllGallery()).thenReturn(new Gallery[0]);
when(collectMapper.selectGidByCollector(7)).thenReturn(new ArrayList<>());
assertTrue(ok(service.selectAllGallery(7)));
}
// ---------- 下载人昵称与筛选(仅管理员) ----------
/** 管理员查询时每条任务都要带上下载人昵称,前端详情直接显示。 */
@Test
void selectAllGalleryFillsDownloaderNameForAdmin() {
Gallery[] all = {gallery(1, "A", "下载完成"), gallery(2, "B", "下载完成")};
all[0].setDownloader(24);
all[1].setDownloader(26);
when(galleries.selectAllGallery()).thenReturn(all);
when(collectMapper.selectGidByCollector(3)).thenReturn(new ArrayList<>());
when(users.selectAllUser()).thenReturn(new User[]{
user(3, "狮子"), user(24, "bot叔叔"), user(26, "贱笑")});
String json = service.selectAllGallery(3);
assertTrue(ok(json), "实际输出: " + json);
assertEquals("bot叔叔", all[0].getDownloaderName());
assertEquals("贱笑", all[1].getDownloaderName());
}
/** 普通用户不得拿到任何下载人昵称,避免泄露他人身份。 */
@Test
void selectAllGalleryOmitsDownloaderNameForRegularUser() {
Gallery[] all = {gallery(1, "A", "下载完成")};
all[0].setDownloader(24);
when(galleries.selectAllGallery()).thenReturn(all);
when(collectMapper.selectGidByCollector(7)).thenReturn(new ArrayList<>());
String json = service.selectAllGallery(7);
assertTrue(ok(json));
assertNull(all[0].getDownloaderName(), "普通用户不应填充下载人昵称");
assertFalse(json.contains("downloaderName"), "响应体不应出现 downloaderName: " + json);
}
private static User user(int id, String username) {
User u = new User();
u.setId(id);
u.setUsername(username);
return u;
}
// ---------- selectDownloaderByGid(管理员专用) ----------
/** 管理员按 gid 查到实际下载人昵称。 */
@Test
void selectDownloaderByGidResolvesNameForAdmin() {
Gallery g = gallery(500, "A", "下载完成");
g.setDownloader(24);
when(users.selectUserByAuthCode("admin")).thenReturn(user(3, "狮子"));
when(galleries.selectGalleryByGid(500)).thenReturn(g);
when(users.selectUserById(24)).thenReturn(user(24, "bot叔叔"));
String json = service.selectDownloaderByGid(500, "admin");
assertTrue(ok(json), "实际输出: " + json);
assertTrue(json.contains("bot叔叔"), "实际输出: " + json);
}
/** 普通用户查下载人必须被拒,不得泄露他人身份。 */
@Test
void selectDownloaderByGidRejectsRegularUser() {
when(users.selectUserByAuthCode("code")).thenReturn(user(7, "tester"));
String json = service.selectDownloaderByGid(500, "code");
assertFalse(ok(json));
assertTrue(json.contains("无权"), "实际输出: " + json);
verify(galleries, never()).selectGalleryByGid(anyInt());
}
/** 任务不存在时回业务失败而不是 NPE。 */
@Test
void selectDownloaderByGidReportsMissingTask() {
when(users.selectUserByAuthCode("admin")).thenReturn(user(3, "狮子"));
when(galleries.selectGalleryByGid(404)).thenReturn(null);
String json = service.selectDownloaderByGid(404, "admin");
assertFalse(ok(json));
assertTrue(json.contains("任务不存在"), "实际输出: " + json);
}
// ---------- selectTaskByLink / ByGid ----------
/** 库里已有该任务时直接返回,不再去外部站点解析。 */
@Test
void selectTaskByLinkReturnsStoredTaskWithoutParsing() throws Exception {
when(galleries.selectGalleryByGid(1234567)).thenReturn(gallery(1234567, "Stored", "下载中"));
try (var parser = mockStatic(GalleryUtil.class)) {
parser.when(() -> GalleryUtil.parseGid(anyString())).thenReturn(1234567);
String json = service.selectTaskByLink("https://exhentai.org/g/1234567/abc/");
assertTrue(ok(json));
assertTrue(json.contains("Stored"));
parser.verify(() -> GalleryUtil.parse(anyString(), anyBoolean(), any()), never());
}
}
/** 库里没有时回落到在线解析。 */
@Test
void selectTaskByLinkFallsBackToOnlineParse() throws Exception {
when(galleries.selectGalleryByGid(1234567)).thenReturn(null);
try (var parser = mockStatic(GalleryUtil.class)) {
parser.when(() -> GalleryUtil.parseGid(anyString())).thenReturn(1234567);
parser.when(() -> GalleryUtil.parse(anyString(), anyBoolean(), any()))
.thenReturn(gallery(1234567, "FromWeb", "等待确认下载"));
String json = service.selectTaskByLink("https://exhentai.org/g/1234567/abc/");
assertTrue(ok(json));
assertTrue(json.contains("FromWeb"));
}
}
/** 在线解析返回 null 时提示查询失败。 */
@Test
void selectTaskByLinkReportsFailureWhenParseReturnsNull() throws Exception {
when(galleries.selectGalleryByGid(1234567)).thenReturn(null);
try (var parser = mockStatic(GalleryUtil.class)) {
parser.when(() -> GalleryUtil.parseGid(anyString())).thenReturn(1234567);
parser.when(() -> GalleryUtil.parse(anyString(), anyBoolean(), any())).thenReturn(null);
String json = service.selectTaskByLink("https://exhentai.org/g/1234567/abc/");
assertFalse(ok(json));
assertTrue(json.contains("查询失败"));
}
}
/** 解析抛异常同样要转成业务失败。 */
@Test
void selectTaskByLinkReportsFailureWhenParseThrows() throws Exception {
when(galleries.selectGalleryByGid(1234567)).thenReturn(null);
try (var parser = mockStatic(GalleryUtil.class)) {
parser.when(() -> GalleryUtil.parseGid(anyString())).thenReturn(1234567);
parser.when(() -> GalleryUtil.parse(anyString(), anyBoolean(), any()))
.thenThrow(new java.io.IOException("upstream down"));
String json = service.selectTaskByLink("https://exhentai.org/g/1234567/abc/");
assertFalse(ok(json));
assertTrue(json.contains("查询失败"));
}
}
/** gid 解析不出来时直接回「链接错误」,不查库。 */
@Test
void selectTaskByLinkRejectsUnparseableLink() throws Exception {
try (var parser = mockStatic(GalleryUtil.class)) {
parser.when(() -> GalleryUtil.parseGid(anyString())).thenReturn(null);
String json = service.selectTaskByLink("https://example.com/nope");
assertFalse(ok(json));
assertTrue(json.contains("链接错误"));
verify(galleries, never()).selectGalleryByGid(anyInt());
}
}
@Test
void selectTaskByGidHandlesFoundAndMissing() {
when(galleries.selectGalleryByGid(5)).thenReturn(gallery(5, "Found", "x"));
assertTrue(ok(service.selectTaskByGid(5)));
when(galleries.selectGalleryByGid(6)).thenReturn(null);
String json = service.selectTaskByGid(6);
assertFalse(ok(json));
assertTrue(json.contains("未找到该图片"));
}
// ---------- selectGalleryByName / ByDownloader ----------
/** 按名字查应把参数包成 LIKE 模式,前后都要有 %。 */
@Test
void selectGalleryByNameWrapsPattern() {
when(galleries.selectGalleryByName("%sakura%")).thenReturn(gallery(1, "Sakura", "x"));
assertTrue(ok(service.selectGalleryByName("sakura")));
verify(galleries).selectGalleryByName("%sakura%");
}
@Test
void selectGalleryByNameReportsFailureWhenMissing() {
when(galleries.selectGalleryByName(anyString())).thenReturn(null);
String json = service.selectGalleryByName("nope");
assertFalse(ok(json));
assertTrue(json.contains("没有找到该名字的图片"));
}
/** 按下载者查:授权码要换成 userId 再查。 */
@Test
void selectGalleryByDownloaderResolvesUserId() {
User u = new User();
u.setId(7);
when(users.selectUserByAuthCode("code")).thenReturn(u);
when(galleries.selectGalleryByDownloader(7)).thenReturn(new Gallery[]{gallery(1, "Mine", "x")});
assertTrue(ok(service.selectGalleryByDownloader("code")));
verify(galleries).selectGalleryByDownloader(7);
}
@Test
void selectGalleryByDownloaderReportsFailureWhenEmpty() {
User u = new User();
u.setId(7);
when(users.selectUserByAuthCode("code")).thenReturn(u);
when(galleries.selectGalleryByDownloader(7)).thenReturn(new Gallery[0]);
String json = service.selectGalleryByDownloader("code");
assertFalse(ok(json));
assertTrue(json.contains("您未下载图片"));
}
// ---------- resetUndone ----------
/** 节点在线且存在未完成任务时,应逐个重投并报告数量。 */
@Test
void resetUndoneResendsEveryUnfinishedTask() {
when(remote.isDead()).thenReturn(false);
Gallery[] undone = {gallery(1, "A", "下载中"), gallery(2, "B", "等待压缩")};
when(galleries.selectUnDoneGalleries()).thenReturn(undone);
String json = service.resetUndone();
assertTrue(ok(json), "实际输出: " + json);
assertTrue(json.contains("2本"), "应带上重投数量: " + json);
verify(remote).addGalleryToQueue(undone[0]);
verify(remote).addGalleryToQueue(undone[1]);
}
/** 节点离线时不得重投(会丢消息),直接提示失败。 */
@Test
void resetUndoneRefusesWhenNodeOffline() {
when(remote.isDead()).thenReturn(true);
String json = service.resetUndone();
assertFalse(ok(json));
assertTrue(json.contains("节点不在线"));
verify(remote, never()).addGalleryToQueue(any());
verify(galleries, never()).selectUnDoneGalleries();
}
/** 没有未完成任务时不重投任何东西。 */
@Test
void resetUndoneReportsWhenNothingToDo() {
when(remote.isDead()).thenReturn(false);
when(galleries.selectUnDoneGalleries()).thenReturn(new Gallery[0]);
String json = service.resetUndone();
assertFalse(ok(json));
assertTrue(json.contains("当前没有未下载完成的图片"));
verify(remote, never()).addGalleryToQueue(any());
}
@Test
void resetUndoneHandlesNullArray() {
when(remote.isDead()).thenReturn(false);
when(galleries.selectUnDoneGalleries()).thenReturn(null);
String json = service.resetUndone();
assertFalse(ok(json));
verify(remote, never()).addGalleryToQueue(any());
}
// ---------- retryGallery ----------
/** 不存在的任务直接失败,且不碰节点。 */
@Test
void retryRejectsMissingTask() {
when(galleries.selectGalleryByGid(404)).thenReturn(null);
String json = service.retryGallery(404);
assertFalse(ok(json));
assertTrue(json.contains("任务不存在"), "实际输出: " + json);
verify(remote, never()).retryGallery(any());
}
/** 已完成的任务无需重试,直接按成功返回(幂等语义)。 */
@Test
void retryIsNoopForCompletedTask() {
when(galleries.selectGalleryByGid(1)).thenReturn(gallery(1, "Done", "下载完成"));
String json = service.retryGallery(1);
assertTrue(ok(json), "实际输出: " + json);
assertTrue(json.contains("下载完成"));
verify(remote, never()).retryGallery(any());
}
@Test
void retryRefusesWhenNodeOffline() {
when(galleries.selectGalleryByGid(1)).thenReturn(gallery(1, "G", "提交失败"));
when(remote.isDead()).thenReturn(true);
String json = service.retryGallery(1);
assertFalse(ok(json));
assertTrue(json.contains("节点不在线"));
verify(remote, never()).retryGallery(any());
}
/** 节点接受重试时透传其返回状态;被拒时透传失败原因。 */
@Test
void retryPropagatesNodeOutcome() {
Gallery g = gallery(1, "G", "提交失败");
when(galleries.selectGalleryByGid(1)).thenReturn(g);
when(remote.isDead()).thenReturn(false);
when(remote.retryGallery(g)).thenReturn(new RemoteService.RetryResult(true, "下载中"));
String okJson = service.retryGallery(1);
assertTrue(ok(okJson), "实际输出: " + okJson);
assertTrue(okJson.contains("下载中"));
when(remote.retryGallery(g)).thenReturn(new RemoteService.RetryResult(false, "节点未接受重试请求"));
String failJson = service.retryGallery(1);
assertFalse(ok(failJson));
assertTrue(failJson.contains("节点未接受重试请求"));
}
// ---------- reconnect ----------
/** 重连结果码到用户可见文案的映射。 */
@Test
void reconnectMapsResultCodesToMessages() {
when(remote.reconnect()).thenReturn((byte) 0);
assertTrue(ok(service.reconnect()), "0 应为成功");
when(remote.reconnect()).thenReturn((byte) -1);
String fail = service.reconnect();
assertFalse(ok(fail));
assertTrue(fail.contains("重连失败"));
when(remote.reconnect()).thenReturn((byte) -2);
assertTrue(service.reconnect().contains("当前未连接"));
when(remote.reconnect()).thenReturn((byte) 42);
assertTrue(service.reconnect().contains("未知错误"));
}
}
@@ -26,7 +26,7 @@ class GallerySubmissionTest {
RemoteService remote = mock(RemoteService.class); RemoteService remote = mock(RemoteService.class);
CustomConfigurationMapper configuration = mock(CustomConfigurationMapper.class); CustomConfigurationMapper configuration = mock(CustomConfigurationMapper.class);
GalleryManageService service = new GalleryManageService(galleries, mock(CollectMapper.class), configuration, GalleryManageService service = new GalleryManageService(galleries, mock(CollectMapper.class), configuration,
users, mock(ShareFileMapper.class), mock(ImageCacheMapper.class), remote, mock(PushService.class)); users, mock(ImageCacheMapper.class), remote, mock(PushService.class));
User user = new User(); User user = new User();
user.setId(7); user.setId(7);
user.setUsername("test"); user.setUsername("test");
@@ -0,0 +1,209 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
import com.lion.lionwebsite.Dao.normal.GalleryMapper;
import com.lion.lionwebsite.Domain.CustomConfiguration;
import com.lion.lionwebsite.Util.CustomUtil;
import com.lion.lionwebsite.Util.GalleryUtil;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
/**
* 定时任务与订阅更新的主逻辑。
* 这些方法由调度器在无人值守时触发,失败只会体现在日志里,因此每条分支
* 都必须有断言锁定:该重连的重连、该告警的告警、不该写库的绝不写库。
*/
class LocalServiceTest {
private CustomConfigurationMapper configurationMapper;
private GalleryMapper galleryMapper;
private PushService pushService;
private RemoteService remoteService;
private SubscriptionRefreshScheduler refreshScheduler;
private LocalService service;
@BeforeEach
void setUp() {
configurationMapper = mock(CustomConfigurationMapper.class);
galleryMapper = mock(GalleryMapper.class);
pushService = mock(PushService.class);
remoteService = mock(RemoteService.class);
refreshScheduler = mock(SubscriptionRefreshScheduler.class);
service = new LocalService(configurationMapper, galleryMapper,
pushService, remoteService, refreshScheduler);
}
// ---------- CheckConnectionAvailability ----------
/** 连接已死:直接重连并发告警,不应再去探测可用性。 */
@Test
void deadConnectionTriggersReconnectAndAlert() {
when(remoteService.isDead()).thenReturn(true);
service.CheckConnectionAvailability();
verify(remoteService).initChannel();
verify(pushService).sendToMe(contains("自动进行重连"));
verify(remoteService, never()).checkAvailability();
}
/** 连接活着且探测有响应:什么都不做。 */
@Test
void healthyConnectionDoesNothing() {
when(remoteService.isDead()).thenReturn(false);
when(remoteService.checkAvailability()).thenReturn((byte) 0);
service.CheckConnectionAvailability();
verify(remoteService, never()).reconnect();
verifyNoInteractions(pushService);
}
/** 探测无响应(-1)时按 reconnect 的返回值给出不同告警文案。 */
@Test
void probeTimeoutReconnectsAndReportsOutcome() {
when(remoteService.isDead()).thenReturn(false);
when(remoteService.checkAvailability()).thenReturn((byte) -1);
when(remoteService.reconnect()).thenReturn((byte) 0);
service.CheckConnectionAvailability();
verify(pushService).sendToMe(contains("重连成功"));
when(remoteService.reconnect()).thenReturn((byte) -1);
service.CheckConnectionAvailability();
verify(pushService).sendToMe(contains("重连失败"));
when(remoteService.reconnect()).thenReturn((byte) -2);
service.CheckConnectionAvailability();
verify(pushService).sendToMe(contains("当前未连接,不进行重连"));
when(remoteService.reconnect()).thenReturn((byte) 9);
service.CheckConnectionAvailability();
verify(pushService).sendToMe(contains("未知错误"));
}
// ---------- reset ----------
/** 额度重置必须同时把用量清零并记录重置时间。 */
@Test
void resetZeroesQuotaAndStampsTime() {
service.reset();
verify(configurationMapper).updateConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, "0");
verify(configurationMapper).updateConfiguration(eq(CustomConfiguration.LAST_RESET_AMOUNT_TIME), anyString());
}
// ---------- verifyCookie ----------
/** Cookie 失效(内容为空)时应告警。 */
@Test
void verifyCookieAlertsOnEmptyContent() throws Exception {
try (var requests = mockStatic(GalleryUtil.class)) {
requests.when(() -> GalleryUtil.requests(anyString(), anyString(), any(), any()))
.thenReturn(" ");
service.verifyCookie();
verify(pushService).sendToMe("cookie过期");
}
}
/** Cookie 有效时不应产生告警。 */
@Test
void verifyCookieSilentWhenValid() throws Exception {
try (var requests = mockStatic(GalleryUtil.class)) {
requests.when(() -> GalleryUtil.requests(anyString(), anyString(), any(), any()))
.thenReturn("<html>ok</html>");
service.verifyCookie();
verifyNoInteractions(pushService);
}
}
/** 请求异常要带原因告警,不能静默失败。 */
@Test
void verifyCookieAlertsOnNetworkFailure() throws Exception {
try (var requests = mockStatic(GalleryUtil.class)) {
requests.when(() -> GalleryUtil.requests(anyString(), anyString(), any(), any()))
.thenThrow(new IOException("connection reset"));
service.verifyCookie();
verify(pushService).sendToMe(contains("connection reset"));
}
}
// ---------- updateSub ----------
/** 全部子账号刷新成功:同步给节点并记录更新时间。 */
@Test
void updateSubStampsTimeWhenAllAccountsSucceed() throws Exception {
when(refreshScheduler.refreshAllNow()).thenReturn(true);
assertTrue(service.updateSub());
verify(remoteService).requestSubscriptionSync();
verify(configurationMapper).updateConfiguration(
eq(CustomConfiguration.LAST_UPDATE_SUB_TIME), anyString());
}
/** 刷新失败时仍要通知节点,但不更新「上次更新时间」,避免掩盖故障。 */
@Test
void updateSubDoesNotStampTimeWhenRefreshFails() throws Exception {
when(refreshScheduler.refreshAllNow()).thenReturn(false);
assertFalse(service.updateSub());
verify(remoteService).requestSubscriptionSync();
verify(configurationMapper, never()).updateConfiguration(anyString(), anyString());
}
/**
* 定时刷新入口必须已经移除。
* 旧的 {@code updateSubScheduler} 会一次性刷新全部账号且没有 initialDelay,
* 于是每次重启都触发一遍全量刷新——这正是本次要消除的行为。
*/
@Test
void scheduledFullRefreshEntryPointIsGone() {
assertThrows(NoSuchMethodException.class,
() -> LocalService.class.getMethod("updateSubScheduler"));
assertThrows(NoSuchMethodException.class,
() -> LocalService.class.getMethod("updateSub", boolean.class),
"不应再保留 isManual 参数:定时路径已不存在");
}
// ---------- 定时作业的调度声明 ----------
/**
* 各定时任务的 cron 属于运维契约(额度重置必须是周一 4 点等),
* 这里用反射锁定,避免被无意改动后无人察觉。
*/
@Test
void scheduledCronExpressionsMatchOperationalContract() throws Exception {
assertCron("CheckConnectionAvailability", "0 0/30 * * * *");
assertCron("reset", "0 0 4 * * MON");
assertCron("verifyCookie", "0 0 0 * * *");
assertCron("clearThumbnailCache", "0 0 4 1 * *");
// 订阅刷新已改为 SubscriptionRefreshScheduler 的分散 tick,
// 其周期与初始延迟由 SubscriptionRefreshSchedulerTest 锁定。
}
private static void assertCron(String method, String expected) throws Exception {
var annotation = LocalService.class.getMethod(method)
.getAnnotation(org.springframework.scheduling.annotation.Scheduled.class);
assertNotNull(annotation, method + " 应带 @Scheduled");
assertEquals(expected, annotation.cron(), method + " 的 cron 与运维约定不一致");
}
/** now() 只用于断言「时间被写入」,这里确认它确实是格式化的当前时间。 */
@Test
void nowIsFormattedTimestamp() {
assertTrue(CustomUtil.now().matches("\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}"));
}
}
@@ -1,27 +0,0 @@
package com.lion.lionwebsite.Service;
import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import java.nio.file.*;
import java.util.HashMap;
import static org.junit.jupiter.api.Assertions.*;
class PersonalArchiveTest {
@Test void archiveContainsCompleteFilesAndCanBeOpenedImmediately(@TempDir Path root) throws Exception {
Path source = Files.createDirectory(root.resolve("source"));
Files.createDirectory(source.resolve("nested"));
for (int i = 0; i < 100; i++) Files.writeString(source.resolve("nested/" + i + ".txt"), "content-" + i);
Path archive = root.resolve("result.tar");
PersonalService.writeTar(source, archive);
var contents = new HashMap<String, String>();
try (var input = new TarArchiveInputStream(Files.newInputStream(archive))) {
org.apache.commons.compress.archivers.tar.TarArchiveEntry entry;
while ((entry = input.getNextTarEntry()) != null) {
if (!entry.isDirectory()) contents.put(entry.getName(), new String(input.readAllBytes(), java.nio.charset.StandardCharsets.UTF_8));
}
}
assertEquals(100, contents.size());
for (int i = 0; i < 100; i++) assertEquals("content-" + i, contents.get("nested/" + i + ".txt"));
}
}
@@ -0,0 +1,76 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
import com.lion.lionwebsite.Domain.CustomConfiguration;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
/**
* 个人管理服务在移除文件管理后的剩余契约:订阅时间、家里 IP 与留言转发。
* 文件浏览/上传/分享/打包/删除相关行为已随接口一并下线。
*/
class PersonalServiceTest {
private CustomConfigurationMapper configurationMapper;
private PushService pushService;
private PersonalService service;
@BeforeEach
void setUp() {
configurationMapper = mock(CustomConfigurationMapper.class);
pushService = mock(PushService.class);
service = new PersonalService(configurationMapper, pushService);
}
/** CustomConfiguration 只有 @Data,没有全参构造,测试里用 setter 组装。 */
private static CustomConfiguration config(String parameter, String value) {
CustomConfiguration c = new CustomConfiguration();
c.setParameter(parameter);
c.setValue(value);
return c;
}
private static boolean ok(String json) {
return json.contains("\"result\":\"success\"");
}
// ---------- lastUpdate / getIp ----------
@Test
void lastUpdateReturnsConfigurationValue() {
when(configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME))
.thenReturn(config(CustomConfiguration.LAST_UPDATE_SUB_TIME, "2026-09-15 10:00:00"));
String json = service.lastUpdate();
assertTrue(ok(json));
assertTrue(json.contains("2026-09-15 10:00:00"));
}
@Test
void getIpReturnsAddressAndUpdateTime() {
when(configurationMapper.selectConfiguration(CustomConfiguration.CURRENT_IP_ADDRESS))
.thenReturn(config(CustomConfiguration.CURRENT_IP_ADDRESS, "203.0.113.7"));
when(configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME))
.thenReturn(config(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME, "2026-09-15 10:00:00"));
String json = service.getIp();
assertTrue(ok(json), "实际输出: " + json);
assertTrue(json.contains("203.0.113.7"));
}
// ---------- message2me ----------
/** 留言必须原样转发到 Telegram,且返回成功。 */
@Test
void message2meForwardsToPushService() {
String json = service.message2me("hello from user");
assertTrue(ok(json));
verify(pushService).sendToMe("hello from user");
}
}
@@ -0,0 +1,70 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
import com.lion.lionwebsite.Dao.normal.UserMapper;
import com.lion.lionwebsite.Domain.CustomConfiguration;
import com.lion.lionwebsite.Domain.User;
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
/**
* 公开服务在移除分享取件后的剩余契约:家里 IP 记录、改授权码与身份查询。
* 分享码换文件的 GetFile 已随文件分享功能下线。
*/
class PublicServiceTest {
private CustomConfigurationMapper configurationMapper;
private UserMapper userMapper;
private TaskHandlerInterceptor interceptor;
private PublicService service;
@BeforeEach
void setUp() {
configurationMapper = mock(CustomConfigurationMapper.class);
userMapper = mock(UserMapper.class);
interceptor = mock(TaskHandlerInterceptor.class);
service = new PublicService(configurationMapper, userMapper, interceptor);
}
private static boolean ok(String json) {
return json.contains("\"result\":\"success\"");
}
// ---------- logIpAddress ----------
/** 记录 IP 时同时写地址与观测时间,两处都要落库。 */
@Test
void logIpAddressWritesBothAddressAndTime() {
service.logIpAddress("203.0.113.7");
verify(configurationMapper).updateConfiguration(CustomConfiguration.CURRENT_IP_ADDRESS, "203.0.113.7");
verify(configurationMapper)
.updateConfiguration(eq(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME), anyString());
}
// ---------- alterAuthCode ----------
/** 改码成功后必须刷新拦截器里的授权码缓存,否则新码要等重启才生效。 */
@Test
void alterAuthCodeUpdatesCache() {
String json = service.alterAuthCode("old", "new");
verify(userMapper).updateAuthCode("old", "new");
verify(interceptor).updateAuthCodes();
assertTrue(ok(json), "实际输出: " + json);
}
// ---------- getUserId ----------
@Test
void getUserIdDelegatesToMapper() {
User expected = new User(7, "code", "alice", null, true);
when(userMapper.selectUserByAuthCode("code")).thenReturn(expected);
assertSame(expected, service.getUserId("code"));
}
}
@@ -0,0 +1,95 @@
package com.lion.lionwebsite.Service;
import com.lion.lionwebsite.Domain.Gallery;
import com.lion.lionwebsite.Util.Response;
import com.pengrad.telegrambot.TelegramBot;
import com.pengrad.telegrambot.request.SendMessage;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
/**
* Telegram 告警文案。任务是无人值守跑一整夜的,这些消息是运维唯一的可见性,
* 因此重点断言「什么情况下会发」以及「消息里是否带上了定位所需的字段」。
*/
class PushServiceTest {
private TelegramBot bot;
private PushService service;
@BeforeEach
void setUp() {
bot = mock(TelegramBot.class);
service = new PushService(bot);
}
private String sentText() {
var captor = org.mockito.ArgumentCaptor.forClass(SendMessage.class);
verify(bot).execute(captor.capture());
return String.valueOf(captor.getValue().getParameters().get("text"));
}
/** 成功提交任务时只报「谁提交了什么」,不带失败原因。 */
@Test
void taskCreateReportOnSuccess() {
Response response = Response.generateResponse().success();
service.taskCreateReport("alice", "My Gallery", response);
String text = sentText();
assertTrue(text.contains("alice"));
assertTrue(text.contains("My Gallery"));
assertFalse(text.contains("下载失败"), "成功时不应出现失败文案: " + text);
}
/** 失败提交必须带上失败原因,方便直接定位。 */
@Test
void taskCreateReportOnFailureIncludesReason() {
Response response = Response.generateResponse().failure("链接错误");
service.taskCreateReport("alice", "My Gallery", response);
String text = sentText();
assertTrue(text.contains("下载失败"), "实际推送: " + text);
assertTrue(text.contains("链接错误"), "应带上失败原因: " + text);
}
/** 下载完成通知要含任务名与完成时间(时间按东八区格式化)。 */
@Test
void downloadCompleteIncludesNameAndTimestamp() {
Gallery gallery = new Gallery();
gallery.setName("Nightly Gallery");
service.downloadComplete(gallery);
String text = sentText();
assertTrue(text.contains("Nightly Gallery"));
assertTrue(text.contains("完成时间"));
assertTrue(text.matches("(?s).*\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}.*"),
"应包含 yyyy-MM-dd HH:mm:ss 形式的时间: " + text);
}
@Test
void storageNodeUpDownProduceDistinctMessages() {
service.storageNodeOnline();
assertTrue(sentText().contains("上线"));
clearInvocations(bot);
service.storageNodeOffline();
assertTrue(sentText().contains("掉线"));
}
/** sendToMe 是统一出口,必须真的调用 bot.execute 而不是只打日志。 */
@Test
void sendToMeExecutesTelegramRequest() {
service.sendToMe("hello");
var captor = org.mockito.ArgumentCaptor.forClass(SendMessage.class);
verify(bot).execute(captor.capture());
assertEquals("hello", captor.getValue().getParameters().get("text"));
}
}

Some files were not shown because too many files have changed in this diff Show More