Compare commits
93
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b9358d9817 | ||
|
|
a72d99ff0a | ||
|
|
788fd1e7e2 | ||
|
|
d6eb25a0f6 | ||
|
|
9faae09851 | ||
|
|
4c1a2d22e2 | ||
|
|
e6c7dab12e | ||
|
|
776eb8cf53 | ||
|
|
e03a2e007b | ||
|
|
5f1066cdab | ||
|
|
e3d4220997 | ||
|
|
42921ba97b | ||
|
|
584db277a6 | ||
|
|
628f3ad0a2 | ||
|
|
dfb647bcd3 | ||
|
|
00e27152fe | ||
|
|
654181d2ea | ||
|
|
bee2f11fd6 | ||
|
|
305036ddfe | ||
|
|
d949415e4d | ||
|
|
3adb42c538 | ||
|
|
a0959618ca | ||
|
|
a288846552 | ||
|
|
850819ed86 | ||
|
|
576092578d | ||
|
|
213ed4e7f6 | ||
|
|
d5b97b82a0 | ||
|
|
4623fca8b0 | ||
|
|
f5ef8ca487 | ||
|
|
50e513b8cc | ||
|
|
d3b18f90fa | ||
|
|
a9ba631847 | ||
|
|
60facae9b5 | ||
|
|
b15eeaf45e | ||
|
|
1e6e3a1557 | ||
|
|
7f823b6150 | ||
|
|
f65c5ad860 | ||
|
|
3c8be3e7c9 | ||
|
|
fb327ba00c | ||
|
|
10e6d44570 | ||
|
|
44995afe78 | ||
|
|
bc4ee64a83 | ||
|
|
14dd12d9ad | ||
|
|
fc8548f90f | ||
|
|
9475237731 | ||
|
|
ccc05fda19 | ||
|
|
72f8ab597a | ||
|
|
5dd09e003b | ||
|
|
7ca12d7999 | ||
|
|
2f10d7a868 | ||
|
|
29f1532dea | ||
|
|
b500691008 | ||
|
|
bcb7a7a6dd | ||
|
|
8b99608193 | ||
|
|
3bdaddaead | ||
|
|
d7ba01be2d | ||
|
|
c1ff2938bf | ||
|
|
2df627eab4 | ||
|
|
00187100f2 | ||
|
|
e8cc4b1097 | ||
|
|
0838434d64 | ||
|
|
325b9b321f | ||
|
|
25cd6d73b6 | ||
|
|
0cc583b910 | ||
|
|
cca2980efa | ||
|
|
27ceed8f4a | ||
|
|
78747c60eb | ||
|
|
c31b3ff5f0 | ||
|
|
7057b28ff0 | ||
|
|
fd016c7809 | ||
|
|
6f0049442f | ||
|
|
1516226192 | ||
|
|
54b1dd47a6 | ||
|
|
07b5b1c376 | ||
|
|
2c0ed4be58 | ||
|
|
793ae12348 | ||
|
|
97c1e91f44 | ||
|
|
bb6a247381 | ||
|
|
5cb2b7005b | ||
|
|
1db8cdf264 | ||
|
|
f451aaaf9c | ||
|
|
9df2b7994b | ||
|
|
e1afc3a6c5 | ||
|
|
2c08552986 | ||
|
|
fd04d61885 | ||
|
|
34f8cb73b2 | ||
|
|
a7bfe1df29 | ||
|
|
b78939528a | ||
|
|
a3b79fc0ea | ||
|
|
39e4e606b0 | ||
|
|
8742e59ebe | ||
|
|
e2fce692bd | ||
|
|
32d98b3d84 |
+46
@@ -0,0 +1,46 @@
|
|||||||
|
target/
|
||||||
|
!.mvn/wrapper/maven-wrapper.jar
|
||||||
|
!**/src/main/**/target/
|
||||||
|
!**/src/test/**/target/
|
||||||
|
|
||||||
|
# 本地运行与测试产生的日志
|
||||||
|
run.out
|
||||||
|
*.log
|
||||||
|
|
||||||
|
### IntelliJ IDEA ###
|
||||||
|
.idea/modules.xml
|
||||||
|
.idea/jarRepositories.xml
|
||||||
|
.idea/compiler.xml
|
||||||
|
.idea/libraries/
|
||||||
|
*.iws
|
||||||
|
*.iml
|
||||||
|
*.ipr
|
||||||
|
|
||||||
|
### Eclipse ###
|
||||||
|
.apt_generated
|
||||||
|
.classpath
|
||||||
|
.factorypath
|
||||||
|
.project
|
||||||
|
.settings
|
||||||
|
.springBeans
|
||||||
|
.sts4-cache
|
||||||
|
|
||||||
|
### NetBeans ###
|
||||||
|
/nbproject/private/
|
||||||
|
/nbbuild/
|
||||||
|
/dist/
|
||||||
|
/nbdist/
|
||||||
|
/.nb-gradle/
|
||||||
|
build/
|
||||||
|
!**/src/main/**/build/
|
||||||
|
!**/src/test/**/build/
|
||||||
|
|
||||||
|
### VS Code ###
|
||||||
|
.vscode/
|
||||||
|
|
||||||
|
### Mac OS ###
|
||||||
|
.DS_Store
|
||||||
|
/.idea/encodings.xml
|
||||||
|
|
||||||
|
# Migration script bytecode
|
||||||
|
scripts/__pycache__/
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
# 图片索引缓存(GalleryIndex)— 重写模块说明
|
||||||
|
|
||||||
|
## 问题
|
||||||
|
|
||||||
|
在线阅读的索引是「gid → 每一页 imgkey」,存在 `cache.db` 的两张表里:
|
||||||
|
|
||||||
|
| 表 | 列 | 含义 |
|
||||||
|
|---|---|---|
|
||||||
|
| `gidToKey` | `gid`, `key`, `pages` | 画廊头部:gid、画廊 key、总页数 |
|
||||||
|
| `ImageKeyCache` | `gid`, `imgkey`, `page` | 每页的 imgkey |
|
||||||
|
|
||||||
|
旧实现(`GalleryManageService.cacheImagesKey`)依次做四件事:查缓存、判断是否需要重建、
|
||||||
|
外呼抓 mpv 页、逐页 `INSERT`。由此产生的实际问题:
|
||||||
|
|
||||||
|
1. **没有唯一约束**。实测生产缓存库有 4 组重复 gid、97 组重复 `(gid,page)`。
|
||||||
|
2. **整组替换不是原子的**。先写头部再逐页写,中途失败会留下「头部命中、页 key 缺失」的
|
||||||
|
半截索引;后续请求直接判定为已缓存,永远取不到图。
|
||||||
|
3. **用行数判断完整性**。`count(*) == pages` 无法发现「页码 1、3 而 pages=2」这种跳号。
|
||||||
|
4. **没有并发合并**。同一本画廊被并发打开时,两个构建者会互相删除对方刚写入的行。
|
||||||
|
5. **逐页写入放大写事务**,抓取与写入之间没有事务边界。
|
||||||
|
|
||||||
|
## 交付物与职责
|
||||||
|
|
||||||
|
| 文件 | 职责 |
|
||||||
|
|---|---|
|
||||||
|
| `Service/GalleryIndexService.java` | 索引的唯一读写入口:同 gid 合并、抓取在事务外、校验后整组短事务替换 |
|
||||||
|
| `Exception/GalleryIndexException.java` | 分类失败原因:INVALID_LINK / UNAVAILABLE / MALFORMED / TRANSIENT |
|
||||||
|
| `Configuration/GalleryIndexConfiguration.java` | 缓存库的事务管理器与读写模板(独立文件,未改 SqlConfiguration) |
|
||||||
|
| `Dao/cache/ImageCacheMapper.java` | 新增页码查询、脏行计数、批量写入(原有方法全部保留) |
|
||||||
|
| `Maintenance/GalleryIndexMaintenance.java` | 离线体检与修复工具(只删不健康 gid 的整组索引) |
|
||||||
|
| `scripts/migrate_image_index_cache.{sh,sql}` | 迁移:默认 dry-run、备份、清理、建唯一索引 |
|
||||||
|
| `Service/GalleryIndexServiceTest.java` | 服务编排的行为约束(无 Spring、无网络) |
|
||||||
|
| `Maintenance/GalleryIndexMaintenanceTest.java` | 工具在真实临时 SQLite 上的判定与删除范围 |
|
||||||
|
| `Dao/cache/ImageCacheMapperSqlTest.java` | 注解 SQL 在真实 MyBatis + SQLite 上的验证 |
|
||||||
|
|
||||||
|
## 接口
|
||||||
|
|
||||||
|
```java
|
||||||
|
GidToKey ensureIndex(String url) throws GalleryIndexException;
|
||||||
|
```
|
||||||
|
|
||||||
|
返回该画廊的 `gid` / `key` / `pages`。三条不变量:
|
||||||
|
|
||||||
|
- **抓取在事务外**:`upstream.fetchPageKeys(url)` 先于任何写事务执行;写事务只在
|
||||||
|
`publish(...)` 里开启,删除旧索引与批量写入在同一个事务内整体生效。
|
||||||
|
- **校验先于写入**:行必须满足 gid 与请求一致、页码恰好为 1..N、不重复、imgkey 非空,
|
||||||
|
否则抛 MALFORMED,此时尚未开启写事务,库中旧索引原样保留。
|
||||||
|
- **读在快照内**:`lookup` 在同一个读事务里读头部与页码,避免读到整组替换的中间状态。
|
||||||
|
|
||||||
|
失败原因:
|
||||||
|
|
||||||
|
| Reason | 触发条件 | isPermanent() | 建议处置 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| INVALID_LINK | 链接里取不到 gid 或 key | true | 直接回「链接错误」 |
|
||||||
|
| UNAVAILABLE | 上游返回空页列表(下架/删除) | true | 回「该图片已下架或已被删除」 |
|
||||||
|
| MALFORMED | 上游页 key 不自洽 | false | 可重试,回「网络波动或其他异常」 |
|
||||||
|
| TRANSIENT | 上游异常或写库失败 | false | 可重试,回「网络波动或其他异常」 |
|
||||||
|
|
||||||
|
## 接入方式(由主模型完成 GalleryManageService 侧改动)
|
||||||
|
|
||||||
|
构造函数注入:
|
||||||
|
|
||||||
|
```java
|
||||||
|
final GalleryIndexService galleryIndexService;
|
||||||
|
```
|
||||||
|
|
||||||
|
`cacheImagesKey` 改为:
|
||||||
|
|
||||||
|
```java
|
||||||
|
public String cacheImagesKey(String url) {
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
try {
|
||||||
|
GidToKey gidToKey = galleryIndexService.ensureIndex(url);
|
||||||
|
return response.success(objectMapper.valueToTree(gidToKey)).toJSONString();
|
||||||
|
} catch (GalleryIndexException e) {
|
||||||
|
response.failure(e.getReason() == GalleryIndexException.Reason.INVALID_LINK
|
||||||
|
? "链接错误"
|
||||||
|
: e.isPermanent() ? "该图片已下架或已被删除" : "网络波动或其他异常");
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Spring 装配无需额外配置:`GalleryIndexConfiguration` 已把 `GalleryIndexService` 注册为 bean,
|
||||||
|
依赖 `ImageCacheMapper`(cache 数据源)与两个 TransactionTemplate。原来的
|
||||||
|
`isIndexComplete` / `rollbackImageIndex` 私有方法与 `ImageCacheMapper` 的
|
||||||
|
`countImageKeyCacheByGid` / `delete*ByGid` 仍保留,供旧调用方与非索引路径使用。
|
||||||
|
|
||||||
|
读取路径 `getCachedImage` 不需要改动:它按 `(gid,page)` 查 ImageKeyCache,现在会走新加的
|
||||||
|
`ux_ImageKeyCache_gid_page` 唯一索引。
|
||||||
|
|
||||||
|
## 迁移
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. dry-run(只读,不带 --apply 绝不写入)
|
||||||
|
scripts/migrate_image_index_cache.sh --db /path/to/copy.db
|
||||||
|
|
||||||
|
# 2. 先在 SQLite 备份副本上演练
|
||||||
|
sqlite3 -readonly /home/lionwebsite/cache.db ".backup '/tmp/cache-copy.db'"
|
||||||
|
|
||||||
|
# 3. 确认后执行(自动做 .backup 备份并校验)
|
||||||
|
scripts/migrate_image_index_cache.sh --db /tmp/cache-copy.db --apply
|
||||||
|
```
|
||||||
|
|
||||||
|
迁移只做两类写入:删除不健康 gid 的整组索引(重复头部、半截、跳号、空 imgkey)、
|
||||||
|
删除孤儿页 key;随后建立 `ux_gidToKey_gid` 与 `ux_ImageKeyCache_gid_page`。
|
||||||
|
若清理后仍有重复,建唯一索引会失败并整体回滚。真实的 `/home/lionwebsite/cache.db` 未被本次工作修改。
|
||||||
|
|
||||||
|
离线修复工具:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
java -cp sqlite-jdbc.jar Maintenance/GalleryIndexMaintenance.java --db /tmp/cache-copy.db # dry-run
|
||||||
|
java -cp sqlite-jdbc.jar Maintenance/GalleryIndexMaintenance.java --db /tmp/cache-copy.db --apply # 执行
|
||||||
|
```
|
||||||
|
|
||||||
|
对 `/home/lionwebsite` 下的库运行 `--apply` 会被拒绝,除非显式加 `--allow-production`。
|
||||||
|
|
||||||
|
## 测试
|
||||||
|
|
||||||
|
在 `/tmp` 的独立源码副本中运行(未复制任何生产 runtime config、未改生产库):
|
||||||
|
|
||||||
|
```
|
||||||
|
mvn -o -Djacoco.skip=true \
|
||||||
|
-Dtest='GalleryIndexServiceTest,GalleryIndexMaintenanceTest,ImageCacheMapperSqlTest' \
|
||||||
|
-DfailIfNoSpecifiedTests=false test
|
||||||
|
|
||||||
|
Tests run: 9, Failures: 0, Errors: 0 -- GalleryIndexMaintenanceTest
|
||||||
|
Tests run: 21, Failures: 0, Errors: 0 -- GalleryIndexServiceTest
|
||||||
|
Tests run: 7, Failures: 0, Errors: 0 -- ImageCacheMapperSqlTest
|
||||||
|
Tests run: 37, Failures: 0, Errors: 0
|
||||||
|
BUILD SUCCESS
|
||||||
|
```
|
||||||
|
|
||||||
|
覆盖的关键场景:
|
||||||
|
|
||||||
|
- 同 gid 24 线程并发只抓取一次、只发布一次;失败结果也被等待者共享;失败后可重试。
|
||||||
|
- 抓取发生在事务之外;所有删除与写入都在事务内;整组替换只有一个写事务。
|
||||||
|
- 行数相同但页码跳号、空 imgkey、空 key、pages<=0 均判定为不完整并重建。
|
||||||
|
- 写入失败整体回滚,旧索引与失败前逐字段一致。
|
||||||
|
- 上游畸形数据(gid 不符、页码重复/跳号/越界、imgkey 为空)在开启写事务之前被拒绝。
|
||||||
|
- 大规模索引按 BATCH_SIZE 分片写入;重复发布替换旧行而非累积。
|
||||||
|
- 注解 SQL 在真实 SQLite 上验证,含唯一约束确实拒绝重复写入。
|
||||||
|
|
||||||
|
## 已知边界
|
||||||
|
|
||||||
|
- 迁移后 gidToKey 433 条、ImageKeyCache 43704 条(原 443 / 43918)。
|
||||||
|
被删除的是 4 个冲突或不完整 gid 的整组索引与 214 行关联页 key,由下一次在线阅读重抓。
|
||||||
|
- 未做性能压测,不承诺加索引后的具体延迟收益,只确认查询计划已由全表扫描变为索引查找。
|
||||||
|
- lookup 的读事务只是为了拿到一致快照;实测 sqlite-jdbc 不允许在已建立的连接上设置
|
||||||
|
readOnly,因此读模板只设超时、不设只读标志。
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# 2026-09-21 低内存调优记录(jar 运行期)
|
||||||
|
|
||||||
|
编译机未上线期间,两个后端都以 jar 方式运行。本机为 2 核 / 2.9G,活跃堆很小但 RSS 偏高,
|
||||||
|
做了一轮只改 JVM 参数与依赖声明的低内存调优。全程未改业务代码。
|
||||||
|
|
||||||
|
## 实测前后对照
|
||||||
|
|
||||||
|
| 服务 | 项目 | 调优前 | 调优后 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| 主站 lionwebsite | RSS | 291 MB | 约 227 MB |
|
||||||
|
| 主站 lionwebsite | jar 体积 | 68.1 MB | 52.2 MB |
|
||||||
|
| 主站 lionwebsite | netty 模块 | 50 个 | 7 个 |
|
||||||
|
| 主站 lionwebsite | 线程数 | 40 | 37 |
|
||||||
|
| 存储节点 storageNode | RSS | 171 MB | 约 155-162 MB |
|
||||||
|
| 存储节点 storageNode | netty 模块 | 35 个 | 5 个 |
|
||||||
|
| 存储节点 storageNode | 线程数 | 24 | 20-22 |
|
||||||
|
|
||||||
|
调优前后堆使用都在 40-52 MB 量级,RSS 的主要成分是元空间、代码缓存与各类映射,
|
||||||
|
因此收益主要来自收敛预留和减少类加载面,而不是堆。
|
||||||
|
|
||||||
|
## 一、JVM 参数(systemd drop-in,不在版本控制内)
|
||||||
|
|
||||||
|
主站 `/etc/systemd/system/lionwebsite.service.d/jar-run.conf`:
|
||||||
|
|
||||||
|
/usr/local/jdk-25/bin/java -Xmx256m -Xss512k -XX:MaxMetaspaceSize=160m \
|
||||||
|
-XX:+UseSerialGC -XX:+UseCompactObjectHeaders \
|
||||||
|
-XX:ReservedCodeCacheSize=96m -XX:+ExitOnOutOfMemoryError \
|
||||||
|
-jar /home/lionwebsite/lionwebsite.jar
|
||||||
|
|
||||||
|
存储节点 `/etc/systemd/system/storageNode.service.d/jar-run.conf`:
|
||||||
|
|
||||||
|
/usr/local/jdk-25/bin/java -Xmx192m -Xss512k -XX:MaxMetaspaceSize=96m \
|
||||||
|
-XX:+UseSerialGC -XX:+UseCompactObjectHeaders \
|
||||||
|
-XX:ReservedCodeCacheSize=96m -XX:+ExitOnOutOfMemoryError \
|
||||||
|
-cp /root/gallery/storageNode/storageNode-jar.jar:/root/gallery/storageNode/lib/* lion.Main
|
||||||
|
|
||||||
|
取值依据:
|
||||||
|
|
||||||
|
- `UseSerialGC`:小堆、低吞吐场景,省掉 G1 的分区元数据与并发标记线程。存储节点原本就是默认 SerialGC,主站原本是默认 G1,本次对齐。
|
||||||
|
- `UseCompactObjectHeaders`:对象头 12 到 8 字节,堆内以 String 与 map 节点为主时收益直接。
|
||||||
|
- `ReservedCodeCacheSize=96m`:原先 240m 预留,实测主站只用约 14 MB、节点约 6 MB。
|
||||||
|
- `-Xmx` 收敛:主站由 384m 到 256m,节点由 256m 到 192m,相对活跃堆仍留 4-6 倍余量。
|
||||||
|
|
||||||
|
回滚:恢复同目录 `jar-run.conf.bak.20260921T090605Z`(主站)或
|
||||||
|
`jar-run.conf.bak.20260921T090746Z`(节点)后 `daemon-reload` 加 `restart`。
|
||||||
|
|
||||||
|
## 二、依赖精简(已入库)
|
||||||
|
|
||||||
|
`netty-all` 是聚合 pom,会拖进全部模块。两个后端只用到 NIO 传输与基础编解码,
|
||||||
|
因此改为按需声明:
|
||||||
|
|
||||||
|
- 主站(netty 4.2.17):`netty-transport` 加 `netty-codec-base` 加 `netty-handler`。
|
||||||
|
`handler` 用于 `RemoteService` 的 `LoggingHandler` 协议调试日志。
|
||||||
|
- 存储节点(netty 4.1.138):`netty-transport` 加 `netty-codec`。
|
||||||
|
节点侧未使用 `handler` 模块的类,故不声明。4.1 无独立 `codec-base`,编解码基类在 `netty-codec`。
|
||||||
|
|
||||||
|
剔除的内容包括 5 个平台的 native-quic、aarch64 与 riscv64 与 osx 的 epoll 与 kqueue 与 io_uring 传输、
|
||||||
|
以及 codec-http3 与 mqtt 与 redis 与 smtp 与 stomp 与 xml 与 protobuf 等。
|
||||||
|
|
||||||
|
`hutool-all` 未调整。
|
||||||
|
|
||||||
|
## 三、已评估但未采用:AOT 缓存
|
||||||
|
|
||||||
|
JDK 25 的 `-XX:AOTCache` 可免原生编译生成类缓存。实测交替 A/B 各两轮:
|
||||||
|
|
||||||
|
| | 启动耗时 | RSS |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 无 AOT | 6.37s / 6.74s | 220.9 MB / 223.1 MB |
|
||||||
|
| 带 AOT | 6.27s / 6.36s | 221.4 MB / 222.7 MB |
|
||||||
|
|
||||||
|
启动约快 0.2s(噪声量级),RSS 与元空间均无改善,代价是 80 MB 缓存文件且每次发版需重训。
|
||||||
|
按降低内存的目标不值得,故未在生产启用。
|
||||||
|
|
||||||
|
## 四、遗留与后续
|
||||||
|
|
||||||
|
- 原生编译机恢复后应重新评估原生镜像:内存会显著低于 jar,届时上述 JVM 参数失去意义。
|
||||||
|
- 存储节点目录内仍保留 `lib.bak.20260921T091617Z`(47 jar)与 `storageNode.jar.bak.*`,为本次回滚路径。
|
||||||
|
- 两个 `jar-run.conf` 的 JVM 参数只存在于服务器 drop-in,未纳入版本控制;重新部署时需按本文恢复。
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
# 2026-09-08 审查修复与编译交接
|
||||||
|
|
||||||
|
本批源码已在本机按问题分别提交,尚未推送、后端编译或部署。编译机恢复后,需要先取得下列仓库的提交,再按现有发布手册构建、验证与上线。此次没有调整鉴权或数据库表结构。
|
||||||
|
|
||||||
|
## 提交清单
|
||||||
|
|
||||||
|
| 仓库 | 提交 | 内容 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| storageNode | cbfd634 | 压缩临时文件、ZIP 完整性校验、失败保留源目录并恢复可重试状态 |
|
||||||
|
| lionwebsite-backend | f65c5ad | 下发节点前持久化任务,超时保留记录,原子累加用量 |
|
||||||
|
| lionwebsite-backend | 7f823b6 | 修复 Range 解析、读取长度、空文件和越界处理 |
|
||||||
|
| lionwebsite-backend | 1e6e3a1 | 上游订阅下载移出全局锁,设置超时,拒绝旧配置/旧请求的刷新结果 |
|
||||||
|
| lionwebsite-backend | b15eeaf | 复用连接线程组,登记等待对象后再发送,统一超时/失败/关闭清理 |
|
||||||
|
| lionwebsite-backend | 60facae | 关闭图片和 TAR 文件流,合并同图请求,完成后发布缓存,检查转换结果 |
|
||||||
|
| lionwebsite-frontend-desktop | 287dc09 | 图片失败继续加载,单图重试,忽略旧页面图片事件 |
|
||||||
|
| lionwebsite-frontend-desktop | b0f832c | 退避重连、连接状态提示、唤醒后重连、重连与提交后的全量刷新 |
|
||||||
|
| lionwebsite-frontend-desktop | d0eb2f0 | 链接 GID 类型匹配,补充任务提交回归测试 |
|
||||||
|
| lionwebsite-backend | a9ba631 | 连接反复断开时重新安排监听,验证缓存失败后可重试 |
|
||||||
|
| lionwebsite-backend | d3b18f9 | 修复图片 Key 缓存递归更新与 null 写入,异常时关闭 HTTP 响应 |
|
||||||
|
|
||||||
|
各仓库分别保留了原有历史。前端原先未提交的 `index.html`、`vite.config.js`、`llm_readme.md` 以及未跟踪的 `dist/`、`node_modules/` 未纳入本批提交。
|
||||||
|
|
||||||
|
## 已验证与待验证
|
||||||
|
|
||||||
|
已完成:
|
||||||
|
|
||||||
|
- 前端 `node --test tests/*.test.mjs`:9 项通过。
|
||||||
|
- 前端生产构建:成功,产物输出到 `/tmp/lionwebsite-review-build`,没有覆盖现有 `dist/` 或线上资源。构建提示主包体积超过 500 kB,本批未做依赖拆包。
|
||||||
|
- 各次提交的 `git diff --check` 与最终源码调用链检查。
|
||||||
|
|
||||||
|
尚未执行:
|
||||||
|
|
||||||
|
- 主站、存储节点的 Java 编译及 JUnit 测试。
|
||||||
|
- GraalVM 原生构建、真实节点断连、真实上游下载和生产端到端验证。
|
||||||
|
|
||||||
|
**以下 Java 测试仅在编译机的独立工作副本执行,不在生产主站或存储机执行。两个仓库依次运行 `mvn test`,通过后再按现有部署手册构建原生程序。** 新增测试均使用临时目录、Mock 或内存通道,不需要生产数据库和真实上游。
|
||||||
|
|
||||||
|
主站新增测试:`GallerySubmissionTest`、`FileDownloadTest`、`SubscriptionRefreshServiceTest`、`RemoteServiceTest`、`PersonalArchiveTest`、`ImageFileCacheTest`、`GalleryKeyCacheTest`。
|
||||||
|
|
||||||
|
存储节点新增测试:`DownloadCheckServiceTest`。两个仓库已有的订阅快照测试也应一起运行。
|
||||||
|
|
||||||
|
## 行为与接口说明
|
||||||
|
|
||||||
|
HTTP 路由、参数名和节点消息格式保持兼容,任务状态继续使用 `已提交`、`下载中`、`等待压缩`、`压缩中`、`下载完成`。
|
||||||
|
|
||||||
|
- `POST /GalleryManage`:先保存任务再下发。节点未确认时返回业务 failure,但任务记录仍在,前端刷新后可以重试;不会把已经收到的完成状态写回已提交。新增任务时即计入本周用量,重试不重复计量。
|
||||||
|
- 存储节点压缩失败:保留源文件,回到等待压缩状态,30 秒后可自动重试;手动重试会取消等待。仅完整校验通过的 ZIP 才视为完成。
|
||||||
|
- 使用 `FileDownload.export` 的文件接口:支持单个普通范围、后缀范围和开放结尾范围;非法或不可满足范围返回 416,`Content-Range: bytes */<size>`;多个范围回退为完整 200 响应;HEAD 不输出正文;普通完整响应不再带 Content-Range。
|
||||||
|
- 图片缓存:同一图片的并发请求共享下载,临时文件关闭且转换成功后才发布。下载连接超时 5 秒、读取超时 15 秒,转换超时 60 秒。图片缺失或下载失败仍返回 404,中断可返回 503。
|
||||||
|
- 订阅刷新:连接和连接池等待超时 5 秒,读取超时 15 秒;网络操作不占用订阅状态锁。上游 Key、过滤选项或启用状态改变后会使旧缓存失效,旧刷新结果不能重新发布。仅名称修改且刷新失败时保留原有缓存。
|
||||||
|
- WebSocket:连接失败按 1/2/4/8/16/30 秒退避,建立连接超时 10 秒;重连后重新读取任务列表和用量。页面恢复可见或网络恢复时重新连接。
|
||||||
|
|
||||||
|
## 上线验证重点
|
||||||
|
|
||||||
|
按现有发布手册安排存储节点、主站和前端上线,避开正在运行的生产任务。源码修复本身不代表生产已经更新。
|
||||||
|
|
||||||
|
1. 新建任务,模拟节点已有归档或立即返回状态,确认数据库与页面均保留完成状态且没有重复行。
|
||||||
|
2. 使用临时测试目录制造一次压缩失败,确认源文件保留,恢复目录可写后能重试,损坏 ZIP 不会标记完成。
|
||||||
|
3. 下载完整文件和小范围/尾部范围,确认响应字节数;验证空文件和越界请求。
|
||||||
|
4. 在测试上游阻塞刷新时修改其他账号/读取快照,确认操作不被网络等待阻塞;旧请求结束后不能覆盖新配置。
|
||||||
|
5. 阅读器人为令一张图片失败,确认后续图片继续加载、单图重试有效。
|
||||||
|
6. 浏览器断网、恢复网络、切后台再返回,确认连接提示和进度能够恢复。
|
||||||
|
|
||||||
|
根目录 `/home/lionwebsite/API_DOCUMENTATION.md` 已追加待发布说明;本文件是这部分接口行为变更的仓库内交接记录。
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# 订阅缓存分代化 — 重写模块说明
|
||||||
|
|
||||||
|
## 问题
|
||||||
|
|
||||||
|
一个订阅账号对应两份内容:`v2ray.txt`(Base64 节点)与 `clash.yaml`(Clash 配置)。
|
||||||
|
旧实现先 `atomicWrite` 写 v2ray,再 `atomicWrite` 写 clash,两次替换彼此独立:
|
||||||
|
|
||||||
|
1. **两份文件可以不同版本**。第二次写入失败(磁盘满、进程被杀)时,用户会拿到
|
||||||
|
「新 v2ray + 旧 clash」的混合版本,且没有任何状态能表明它已经撕裂。
|
||||||
|
2. **读取方无法保证配对**。分发路径与备机快照各自 `Files.readAllBytes` 两个路径,
|
||||||
|
在这两次读之间完成一次刷新,就会读到跨版本组合。
|
||||||
|
3. **过期刷新可能复活**。在途刷新返回时只比较上游 key 与过滤开关;一次
|
||||||
|
`invalidateCache` 之后,迟到的响应仍会把内容写回被清空的路径。
|
||||||
|
4. **失败留半成品**。第一次写成功、第二次失败时,磁盘上就是只有一份文件的账号。
|
||||||
|
|
||||||
|
## 交付物与职责
|
||||||
|
|
||||||
|
| 文件 | 职责 |
|
||||||
|
|---|---|
|
||||||
|
| `Util/SubscriptionCacheStore.java` | 分代存储:两份文件同一分代写入 → 校验 → 指针原子发布;读取始终来自指针指向的同一分代 |
|
||||||
|
| `Service/SubscriptionRefreshService.java` | 抓取与解析在协调器锁外进行,发布短持锁;`invalidateCache` 递增版本使在途刷新失效 |
|
||||||
|
| `Service/SubscriptionStandbySnapshotService.java` | 快照构建在读锁内通过同一 store 解析成对内容,保证同版本 |
|
||||||
|
| `Util/ClashSubscriptionParser.java` | 按 YAML 语义过滤高倍节点并清理 `proxy-groups` 引用(由主模型提供) |
|
||||||
|
|
||||||
|
## 目录布局
|
||||||
|
|
||||||
|
```
|
||||||
|
<cacheRoot>/<accountId>/current 指针(Properties 文本,原子替换)
|
||||||
|
<cacheRoot>/<accountId>/generations/<generation>/v2ray.txt
|
||||||
|
<cacheRoot>/<accountId>/generations/<generation>/clash.yaml
|
||||||
|
<cacheRoot>/<accountId>/v2ray.txt + clash.yaml 旧结构,仅在两份都在时按 legacy 读
|
||||||
|
```
|
||||||
|
|
||||||
|
`current` 记录分代名与两份文件的 SHA-256。分代名是定宽 `时间戳-序号`,
|
||||||
|
因此字典序即时间序;默认只保留最近两个分代(当前与上一个)。
|
||||||
|
|
||||||
|
## 不变量
|
||||||
|
|
||||||
|
- **发布最后一步才切指针**:先写两份文件,读回校验长度与内容,再原子替换 `current`。
|
||||||
|
抓取失败、解析失败、任一份写入失败、指针写入失败,都会回滚新分代目录并保留上一个完整版本。
|
||||||
|
- **读取配对**:`read` / `resolve` / `hasComplete` 走同一份 `load` 契约,每个调用只读一次
|
||||||
|
指针快照,再按该快照校验双文件哈希。只有两份齐全且匹配才算一个有效版本;
|
||||||
|
指针缺失才回退 legacy(两份齐全时),指针存在但读不出完整版本(半写、篡改、缺字段)
|
||||||
|
时按无可用缓存处理,不回退到更旧的半份内容。
|
||||||
|
- **分发不会发出半份缓存**:`cachedPath` 在无有效版本时返回保证不存在的哨兵路径
|
||||||
|
(`.unavailable/`),使 `SubService` 判定为 503,而不会被单份 legacy 文件骗过。
|
||||||
|
`SubService.updateSub` 在读锁内完成 resolve 与 `FileDownload.export`,与 publish/invalidate 互斥。
|
||||||
|
- **失效不可复活**:`invalidateCache` 在写锁内删除指针/分代/旧结构文件,并移除该账号的
|
||||||
|
刷新版本标记;在途响应返回时因版本标记对不上而被丢弃,不会重新发布。
|
||||||
|
- **兼容旧结构**:只有当账号根下 `v2ray.txt` 与 `clash.yaml` 都在时才读为 legacy 版本;
|
||||||
|
新分代发布成功后旧结构文件被移除,避免两份来源并存。
|
||||||
|
|
||||||
|
## 测试
|
||||||
|
|
||||||
|
| 测试 | 锁定行为 |
|
||||||
|
|---|---|
|
||||||
|
| `Util/SubscriptionCacheStoreTest` | 发布后两份同分代;第二份写入失败 / 指针发布失败均保留旧版本;legacy 双文件才有效;半份 legacy、缺一份分代、内容被篡改、坏指针、指针指向缺失分代均判定不可用;每次查询只读一次指针;哨兵路径保证不存在 |
|
||||||
|
| `Service/SubscriptionCacheGenerationTest` | 在途旧刷新在失效与新版发布后被丢弃;被丢弃的刷新不留任何缓存 |
|
||||||
|
| `Service/SubscriptionFilteringTest` | 过滤规则;失败保留上次完整版本;`cachedPath` / `hasCompleteCache` 新契约;legacy 路径定位 |
|
||||||
|
| `Service/SubscriptionStandbySnapshotServiceTest` | 快照从单一分代读成对内容;不完整 legacy 账号不入快照 |
|
||||||
|
| `Util/ClashSubscriptionParserTest` | YAML 语义过滤(内联节点与引用)与畸形正文拒绝 |
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# 备用订阅更新事件
|
||||||
|
|
||||||
|
主站 `/sub/{client}/{key}` 接受网关的签名事件;`GET` 在客户端类型与缓存路径有效后,同一个主库事务写更新记录和长期收据。签名有效的 `HEAD` 不写记录。普通请求以及无效签名仍走原来的更新记录逻辑。
|
||||||
|
|
||||||
|
`POST /internal/subscription-events` 有界读取原始请求(最多 256 KiB),批次最多 100 条。签名算法为 HMAC-SHA256,同步密钥来自 `SUBSCRIPTION_SYNC_SECRET`;签名输入是 `v1\n` + epoch 秒时间戳 + `\n` + 原始请求体 SHA256 十六进制。时间偏差最多 300 秒。请求头为 `X-Subscription-Timestamp` 和 `X-Subscription-Signature`。主站 GET 将单个事件的原始 JSON 使用无 padding 的 base64url 编码放入 `X-Subscription-Event`,同样对原始事件 JSON 签名。
|
||||||
|
|
||||||
|
事件字段:`eventId`(规范小写 UUID)、`keyHash`(公开 key 的小写 SHA256)、`client`(v2 或 cat)、`clientIp`(IPv4/IPv6 字面值)、`userAgent`(最多 2048 字符)、`occurredAt`(epoch 毫秒)。允许历史事件,事件时间不可超过未来 5 分钟。签名中的 IP 为权威来源,不能由未经签名的 XFF 覆盖。
|
||||||
|
|
||||||
|
响应 `acknowledged` 只在事务已提交后返回,包括已存在的相同事件。冲突事件 ID 返回 400;签名无效返回 403;数据库异常返回 5xx,无确认列表。未知 hash 不确认;历史映射中用户已删除则写终态收据并确认,不猜测用户归属。可见记录按事件时间和 ID 倒序保留每用户最近 10 条,收据独立长期保留。批次任意数据库写入失败会全部回滚。
|
||||||
|
|
||||||
|
## 显式迁移
|
||||||
|
|
||||||
|
运行前先执行稳定 user_id 关联迁移。新表不会在启动时创建。
|
||||||
|
|
||||||
|
1. 先在数据库副本上演练:`python3 scripts/migrate_subscription_events.py /path/to/copy.db`。
|
||||||
|
2. 停止生产主站服务,确认没有其他写入者。
|
||||||
|
3. 执行 `python3 scripts/migrate_subscription_events.py /path/to/main.db --apply --service-stopped`。
|
||||||
|
4. 检查输出 integrity 为 ok,保留输出中的 SQLite 备份,然后启动新程序。
|
||||||
|
|
||||||
|
脚本在写入前使用 SQLite backup API 生成权限 0600 的备份,在单个事务中建表并初始化历史 hash,重复执行保留既有收据和历史。不将公开 key 写入历史表或输出;原数据库备份含既有敏感数据,须保留原有保护措施。
|
||||||
|
|
||||||
|
快照构建记住所有现有绑定;重置或删除绑定前也保存历史 hash。用户改名时回传事件按 stable user_id 读取当前名字。合法旧订阅标签的 user_id 可以为 NULL,历史映射保留标签并按原标签写回,不创建用户,也不猜测同名新用户归属。可见日志删除不删除历史映射或收据。
|
||||||
|
|
||||||
|
Java 编译与测试必须在部署手册指定的 192.168.5.30 执行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mvn -Dtest=SubscriptionEventIntegrationTest,SubServiceTest,SubscriptionStandbySnapshotServiceTest,MyBatisNativeConfigurationTest test
|
||||||
|
```
|
||||||
|
|
||||||
|
Python 迁移验证可本机执行:`python3 -m unittest discover -s scripts -p 'test_subscription_events.py' -v`。
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
# 任务生命周期与协议 v2
|
||||||
|
|
||||||
|
范围:主站下载任务的持久化生命周期、与存储节点的可靠上报协议。对应评估报告第 2、6 节。
|
||||||
|
|
||||||
|
## 为什么重写
|
||||||
|
|
||||||
|
原实现里创建先`POST`上游、后落库,删除先删库、后等节点,节点上报异步写完后立刻清队列:上下游任一侧失败都会留下「主站与节点说法不一致」的状态,且没有任何东西能把它们对回来。
|
||||||
|
|
||||||
|
复现过的具体后果:模拟状态消息发送失败后,完成任务已从节点队列移除、消息却未发出,于是「归档已存在、主站仍显示未完成」。
|
||||||
|
|
||||||
|
## 不变量
|
||||||
|
|
||||||
|
1. `gid` 只有一个有效生命周期意图。创建先落唯一 `CREATE` 意图(并写一条可见的占位任务行),靠 `gallery(gid)` 主键唯一把并发创建收敛成一个赢家;输家直接得到业务失败。
|
||||||
|
2. 上游副作用只在意图持久化之后发生。顺序固定为:登记意图 → 只读解析(`prepare`,无 POST)→ 持久化 `SUBMITTING` → 真正 `submit`。
|
||||||
|
3. `SUBMITTING` 之后绝不自动重发。上游不支持幂等,因此超时/断连/落库失败/重启中断一律落 `UNKNOWN` 等待人工对账处理,而不是再 `POST` 一次。
|
||||||
|
4. 任务行与周用量在同一短事务里写入;重复落库不重复计量。数据库失败时任务行与意图保持一致(要么都在,要么明确为 `UNKNOWN`),不允许出现「有任务没计量」。
|
||||||
|
5. 删除失败保留记录。先落 `DELETE_PENDING` 与稳定 `deleteOperationId`,只有节点确认(或文件已不存在)后才在同一事务里清收藏、删任务行并留 `DELETED` 墓碑。
|
||||||
|
6. 上报按 `revision` 单调推进,按 `operationId` 归属。迟到的旧世代上报被丢弃且不改变当前任务;墓碑(`DELETED`)永久忽略所有上报,已删除任务只能由显式 `CREATE` 重新建立新世代。
|
||||||
|
7. 回执只在提交之后发送。数据库失败不回 ACK,节点保留快照重投;重复上报也回 ACK 以停止重放,但不会重复通知。
|
||||||
|
8. 通知与 WebSocket 推送在事务提交之后、由独立执行器发送;慢通知不阻塞状态提交。
|
||||||
|
|
||||||
|
9. 只有确认已提交(`SUBMITTED`)或迁移前确实提交过的旧任务才允许下发给节点。`UNKNOWN`、`RESERVED`、`SUBMITTING`、`DELETE_PENDING`、`DELETED` 一律不下发:上游不支持幂等,把未确认/未计量的任务重投会在节点侧变成「下载完成却 0 额度」。
|
||||||
|
10. 启动时先绑定上报处理器与判定解析器、执行恢复(`SUBMITTING → UNKNOWN`、释放 `RESERVED`),完成之后才允许连接与自动重投;恢复失败时只保持连接,禁止自动派发。
|
||||||
|
|
||||||
|
## 状态表
|
||||||
|
|
||||||
|
| 层 | 取值 |
|
||||||
|
| --- | --- |
|
||||||
|
| `task_operation.state` | `RESERVED` `SUBMITTING` `SUBMITTED` `UNKNOWN` `DELETE_PENDING` `DELETED` |
|
||||||
|
| 任务展示状态 | `提交中` `已提交` `下载中` `等待压缩` `压缩中` `下载完成` `提交未知` `待删除` `删除失败` |
|
||||||
|
| 节点上报状态码 | `1` 下载中 `2` 等待压缩 `3` 压缩中 `4` 下载完成(沿用旧码) |
|
||||||
|
|
||||||
|
状态码到中文文案只在 `TaskStatus` 一处映射。
|
||||||
|
|
||||||
|
## 线协议
|
||||||
|
|
||||||
|
帧格式不变:`[messageType(1B)][length(4B, 大端)][JSON]`。旧类型 `0..9` 的字段一个都不动(旧节点 Jackson 会拒绝未知字段)。
|
||||||
|
|
||||||
|
| 类型 | 消息 | 关键字段 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 10 | `ProtocolHelloMessage` | `protocolVersion=2`, `features=["reliable-task-v2"]` |
|
||||||
|
| 11 | `TaskCommandMessage` | `operationId`, `action=DOWNLOAD\|DELETE`, `gid`, `name` |
|
||||||
|
| 12 | `TaskReportMessage` | `reportId`, `revision`, `galleryTasks[]`, `operationIds{gid→op}` |
|
||||||
|
| 13 | `TaskReportAckMessage` | `reportId`, `result` |
|
||||||
|
|
||||||
|
流程:主站先发旧 `Identity`,再发类型 10;节点回类型 10 声明 v2 后,才允许使用 `11..13`。未协商成功时自动退化为旧的 `DownloadPostMessage`/`DeleteGalleryMessage`(此时不承诺可靠 ACK,属已知兼容边界)。
|
||||||
|
|
||||||
|
## 部署与迁移
|
||||||
|
|
||||||
|
1. 用 `scripts/migrate_task_operation_v2.sql` 建表。脚本只有独立语句、不含 `BEGIN/COMMIT`,由外层 wrapper 包事务并在执行前备份。
|
||||||
|
2. 该脚本还会在 `gallery(gid)` 上建唯一索引。历史库若已有重复 gid,这条语句会失败:先人工确认重复来源再处理,不要直接删行。
|
||||||
|
3. 主库连接需要 busy 等待(`SqlConfiguration` 已为主数据源设置 `PRAGMA busy_timeout=5000`)。单文件 SQLite 并发开短事务时,没有等待窗口会直接抛出 `SQLITE_BUSY`。生产库建议同时启用 WAL。
|
||||||
|
4. 旧任务行没有操作记录,首次收到上报时会按显式策略补建一次「世代」(优先采纳上报里的 `operationId`),不会改动其已有状态。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
`mvn -o test`:549 项,0 失败/错误/跳过。
|
||||||
|
|
||||||
|
新增关键用例(`TaskLifecycleIntegrationTest`,真实 SQLite 文件 + 真实事务/触发器/唯一约束,落在 `/tmp` 临时目录):
|
||||||
|
|
||||||
|
- 意图与占位行先于上游 `POST`;`SUBMITTING` 先于 `submit`。
|
||||||
|
- 同 `gid` 6 线程并发创建:上游只被提交一次、任务行不超过一条。
|
||||||
|
- 上游超时后落 `UNKNOWN`;再次点击创建也不会第二次 `POST`。
|
||||||
|
- 重启恢复把中断的 `SUBMITTING` 判为 `UNKNOWN`(任务行同步显示「提交未知」),安全释放 `RESERVED` 占位,且不改写 `operationId`、不重发。
|
||||||
|
- 额度写入失败时落库事务回滚,任务保持可对账的 `提交未知` + `UNKNOWN`。
|
||||||
|
- 重投/重试复用同一 `operationId`。
|
||||||
|
- 判定为拒绝的任务(`UNKNOWN` 等)绝不下发,也不退化为无 `operationId` 的旧命令。
|
||||||
|
- 删除失败保留任务行与意图,重试复用同一删除 ID;成功后清收藏、删行、留墓碑。删除意图 ID 不匹配时不得完成删除。
|
||||||
|
- 正在提交(`RESERVED`/`SUBMITTING`)的任务拒绝删除;释放预占只清匹配世代的记录与占位行。
|
||||||
|
- 落库严格匹配本次创建世代(expectedOperationId),被删除/换代后不得复活已删 gid。
|
||||||
|
- 墓碑永久忽略所有上报(含同一 `operationId`),只允许显式 `CREATE` 用新 `operationId` 重建。
|
||||||
|
- 旧节点不带 `operationId` 的连续上报保持可接受;新 v2 已知世代缺归属的上报仍被拒。
|
||||||
|
- 过期 `revision`、缺 `operationId`、待删除任务的上报都被丢弃;未知 gid 忽略但回执。
|
||||||
|
- 重复上报不重复通知;回执写入失败时 `apply` 返回不可回执结果。
|
||||||
|
|
||||||
|
`TaskProtocolV2CodecTest` 锁定类型 10..13 的帧头与字段名,并断言旧 `DownloadPostMessage` 没有混入新字段。
|
||||||
|
|
||||||
|
## 已知限制
|
||||||
|
|
||||||
|
未做双机真实故障演练。`UNKNOWN` 目前只提供状态与持久化记录:任务行会显示「提交未知」并拒绝自动重投/重试,对账与人工解决的接口由后续批次补齐。
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
# LionWebsite 一键登录(/login):实现说明
|
||||||
|
|
||||||
|
状态:**已全部完成并上线**。主站侧提交 `00e2715`(文档 `dfb647b`、`628f3ad`),
|
||||||
|
桌面端 `9b5bc94`;机器人侧由 `debian-qq-chatgpt` 于 host-vm103-debian-qq 完成
|
||||||
|
(见共享盘 `tasks/2026-09-21-lionwebsite-login-bot-command/`)。
|
||||||
|
关联待办:`td-20260921-a1b2c3`(已归档为
|
||||||
|
`todos/archive/done-td-20260921-a1b2c3-lionwebsite-login-tg.md`)。
|
||||||
|
|
||||||
|
用户已明确选择**接法 B:共享密钥自签票据**,理由是机器人到主站的网络调用不可靠。
|
||||||
|
因此主站**不提供**任何票据签发端点,也不需要机器人侧发起 HTTP 请求——机器人本地
|
||||||
|
用共享密钥算出票据即可。
|
||||||
|
|
||||||
|
## 1. 已经上线的东西(主站侧)
|
||||||
|
|
||||||
|
### 1.1 票据格式
|
||||||
|
|
||||||
|
```
|
||||||
|
v1.<签发时间的 Unix 秒级时间戳>.<HMAC-SHA256 十六进制小写>
|
||||||
|
```
|
||||||
|
|
||||||
|
签名原文为 `v1.<时间戳>`(固定前缀 `v1.` 加时间戳本身),密钥为共享密钥的 UTF-8 字节。
|
||||||
|
等价于 `hex(HMAC_SHA256(key, "v1." + timestamp))`。
|
||||||
|
|
||||||
|
### 1.2 校验规则
|
||||||
|
|
||||||
|
- 必须恰好三段,第一段必须是 `v1`;
|
||||||
|
- 签名比较用常量时间(`MessageDigest.isEqual`);
|
||||||
|
- 时间戳只接受 `[now - 300s, now]`,**未来时间戳一律拒绝**(防伪造者用远期时间换长期有效);
|
||||||
|
- 窗口内**允许重放**(用户明确接受),过期即失效;
|
||||||
|
- 默认 300 秒有效期,可用 `personal.login.ticket-ttl-seconds` 调整,下限 30 秒。
|
||||||
|
|
||||||
|
### 1.3 端点
|
||||||
|
|
||||||
|
| 端点 | 作用 |
|
||||||
|
| --- | --- |
|
||||||
|
| `GET /login?t=<票据>` | 校验票据 → 作废旧会话(防会话固定)→ 建新会话 → 302 跳 `/index`;失败 302 跳 `/denied` |
|
||||||
|
| `GET /login/logout` | 销毁会话 → 302 跳 `/denied` |
|
||||||
|
| `GET /denied` | 静态提示页:「请在机器人里发送 `/login`」 |
|
||||||
|
|
||||||
|
nginx 无需改动:`location /` 会把 `/login` 改写成后端 `/personal/login`。
|
||||||
|
|
||||||
|
### 1.4 会话与鉴权
|
||||||
|
|
||||||
|
- 登录成功下发 `JSESSIONID`,参数为 `Path=/`、`HttpOnly`、`SameSite=Lax`,14 天滑动过期。
|
||||||
|
**`Path=/` 是必须的**:nginx 会把 `/user` 改写成后端 `/personal/user`,若沿用容器按
|
||||||
|
请求路径推导的 `/personal`,浏览器判定 `/user` 不匹配就不会带会话,面板会一直 401。
|
||||||
|
- `PersonalInterceptor` 放行「有效会话 **或** `AuthCode=alone`」,两者都拒绝时返回 **401**。
|
||||||
|
`alone` 是留给下载器前端与存储节点 `/message2me` 推送的,本次**没有**退役它。
|
||||||
|
|
||||||
|
### 1.5 密钥
|
||||||
|
|
||||||
|
- 主站从环境变量 `PERSONAL_LOGIN_SECRET` 读取(`personal.login.secret`),
|
||||||
|
由 systemd drop-in `/etc/systemd/system/lionwebsite.service.d/login-secret.conf`
|
||||||
|
加载 `/etc/lionwebsite/login-secret.env`(`600` 权限,不在仓库里)。
|
||||||
|
- 密钥为空时**一律拒绝**票据,不会退化成放行。
|
||||||
|
- 轮换方式:改两侧配置并重启。轮换后旧票据在 300 秒内自然失效。
|
||||||
|
|
||||||
|
## 2. 机器人侧待实施(host-vm103-debian-qq)
|
||||||
|
|
||||||
|
### 2.1 生成票据
|
||||||
|
|
||||||
|
密钥必须与主站一致。**已由主人批准写入 PersonalHub 保险库**,引用为
|
||||||
|
`secret://services/service-lionwebsite/panel-login-secret`;机器人侧按既有秘密读取流程
|
||||||
|
申请取值,不要写进仓库或聊天记录。Python 侧计算方式:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import hashlib, hmac, time
|
||||||
|
|
||||||
|
def lionwebsite_login_link(secret: str, base: str = "https://personal.lionwebsite.xyz") -> str:
|
||||||
|
stamp = str(int(time.time()))
|
||||||
|
signed = f"v1.{stamp}"
|
||||||
|
digest = hmac.new(secret.encode(), signed.encode(), hashlib.sha256).hexdigest()
|
||||||
|
return f"{base}/login?t=v1.{stamp}.{digest}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Shell 等价写法(`openssl`)供人工验证用:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
SIG=$(printf 'v1.%s' "$STAMP" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $NF}')
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.2 命令接入(已实施:复用既有 `/login`)
|
||||||
|
|
||||||
|
**复用既有 `/login`,不新建命令名;一条回复里同时给出两个登录链接。**
|
||||||
|
|
||||||
|
实现方式(vm103 实际做法,比原计划更稳):**机器人进程不持有密钥**。
|
||||||
|
签发走保险库固定动作 `CollabSecretService.use_secret`,值只交给进程内签名函数,
|
||||||
|
不返回、不落盘、不进日志;审计只记引用与动作名 `lionwebsite-login-link`。
|
||||||
|
QQ 侧 `plugins/login.py` 只经内部令牌 POST `/internal/panel-login`,
|
||||||
|
由 PersonalService 统一产出文案,因此 QQ 与 Telegram 不会漂移。
|
||||||
|
|
||||||
|
- Telegram:`telegram_panel_login()` 的返回文案改为同时给出 PersonalHub 面板链接与
|
||||||
|
LionWebsite 面板链接,`TELEGRAM_COMMANDS` 里 `login` 的描述同步更新。
|
||||||
|
- QQ(LionQQBot):既有 `/login`(别名 `/登录`)返回同样两条链接,共用同一套生成逻辑。
|
||||||
|
- PersonalHub 未完整配置(缺 password / session_secret / TOTP)时,现有逻辑会回一句
|
||||||
|
配置提示;此时**仍要发出 LionWebsite 链接**,不要让一条链接的失败带掉另一条。
|
||||||
|
- 帮助菜单(Telegram `/help` 与 QQ 菜单)中 `/login` 的描述同步更新。
|
||||||
|
- 文案不要写成「两条都只能用一次」:PersonalHub 链接是一次性的,
|
||||||
|
LionWebsite 票据是 300 秒窗口内可重放。
|
||||||
|
|
||||||
|
示例文案:
|
||||||
|
|
||||||
|
```
|
||||||
|
已生成登录链接:
|
||||||
|
|
||||||
|
· Personal Hub:[打开面板](< PersonalHub 链接 >)
|
||||||
|
· LionWebsite:[打开面板](< LionWebsite 链接 >)
|
||||||
|
|
||||||
|
两条链接 5 分钟内有效,请勿转发。
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 与 PersonalHub 现有 /login 的关系
|
||||||
|
|
||||||
|
PersonalHub 的 `/login` 原本只签发**它自己面板**的票据
|
||||||
|
(`OneTimePanelLoginTickets` + `/panel/telegram-login`)。改造时保留这条,
|
||||||
|
再在同一回复里附加 LionWebsite 链接;两者的票据机制与有效期互不影响。
|
||||||
|
|
||||||
|
## 3. 交付与复验
|
||||||
|
|
||||||
|
- 机器人侧提交状态:**两个机器人仓库的改动尚未提交 Git**(`v103` 侧待主人确认后
|
||||||
|
按项目约定提交,注意 PersonalHub 工作区另有前一轮未提交改动,勿混提)。
|
||||||
|
- `us9929main-chatgpt` 的独立复验(2026-09-21):
|
||||||
|
- 票据边界实测:`now` 与 `now-290s` 放行,`now-301s`、`now-400s`、`now+60s` 均落 `/denied`;
|
||||||
|
篡改签名与畸形串同样落 `/denied`。
|
||||||
|
- 密钥明文扫描:抓取两个任务目录全部正文与线程消息共 17 KiB,未出现密钥明文,
|
||||||
|
只有 4 处 `secret://` 引用。
|
||||||
|
- 内部接口鉴权:`POST /internal/panel-login` 无令牌与错令牌均返回 403。
|
||||||
|
- PersonalHub `/health` 返回 200;共享盘 waiting 指针已消费,队列已清空。
|
||||||
|
|
||||||
|
## 4. 可选收尾(未做,需要时另开任务)
|
||||||
|
|
||||||
|
- 移动端 `PrivateMainForMobile` 仍写死 `authCode: "alone"`。后端两种方式都接受,
|
||||||
|
所以它能用;要一并切到会话,照搬 `PrivateMain/src/store/index.js` 的改法即可。
|
||||||
|
- `sourcecode/storageNode` 有两处硬编码 `alone`(`CustomUtil.java` 的 `/message2me`
|
||||||
|
与 `MultiThreadedHTTPServer.java` 的本机鉴权),属另一仓库,需单独发布后才能退役字面量。
|
||||||
|
- 概览页的「本机订阅」链接 `https://personal.lionwebsite.xyz/sub/self` 目前是 404,
|
||||||
|
后端没有对应映射,属历史遗留,与本次登录改造无关。
|
||||||
+160
@@ -0,0 +1,160 @@
|
|||||||
|
================================================================================
|
||||||
|
LionWebsite 项目总结
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
一、项目概况
|
||||||
|
──────────────────────────────────────────────────────────────────────────────
|
||||||
|
名称: LionWebsite
|
||||||
|
技术栈: Spring Boot 4.1.1, Java 21 字节码(生产 JDK 25), Maven, SQLite, MyBatis 4, Netty 4.2
|
||||||
|
定位: 个人/私有 Web 应用,兼具 E-Hentai 画廊下载管理、代理订阅
|
||||||
|
管理等功能。
|
||||||
|
运行端口: 8888
|
||||||
|
数据库: 双 SQLite 数据库 — LionWebsite.db (主库) + cache.db (缓存库)
|
||||||
|
构建目标: 当前以 JVM/JAR 运行;保留 GraalVM Native Image 配置但尚未在 JDK 25 完成原生验证
|
||||||
|
|
||||||
|
二、项目结构
|
||||||
|
──────────────────────────────────────────────────────────────────────────────
|
||||||
|
src/main/java/com/lion/lionwebsite/
|
||||||
|
|
||||||
|
├── LionWebsiteApplication.java 主启动类 (@EnableScheduling, 双数据源 MapperScan)
|
||||||
|
│
|
||||||
|
├── Configuration/
|
||||||
|
│ ├── SqlConfiguration.java 双数据源 (main + cache) SQLite 配置
|
||||||
|
│ ├── MyBatisNativeConfiguration.java GraalVM Native 适配 (AOT hints)
|
||||||
|
│ ├── WebsocketConfiguration.java WebSocket 注册 (/ws/)
|
||||||
|
│ ├── InterceptorConfiguration.java 拦截器链注册
|
||||||
|
│ ├── CorsConfig.java CORS 全开放
|
||||||
|
│ └── CustomBean.java Telegram Bot Bean + Native 反射注册
|
||||||
|
│
|
||||||
|
├── Controller/
|
||||||
|
│ ├── GalleryManageController.java 画廊任务 CRUD、收藏、图片在线缓存 /GalleryManage
|
||||||
|
│ ├── QueryController.java E-Hentai 搜索代理 /query
|
||||||
|
│ ├── PublicController.java 根路由、IP、订阅、验证 /、/ip、/sub/、/validate
|
||||||
|
│ ├── PersonalController.java 个人管理 (订阅更新/最后更新时间/IP/留言) /personal/
|
||||||
|
│ ├── SubController.java 订阅绑定管理 /personal/subBind/
|
||||||
|
│ └── UserController.java 用户管理 /personal/user
|
||||||
|
│
|
||||||
|
├── Service/
|
||||||
|
│ ├── GalleryManageService.java 核心画廊管理 (任务创建/查询/删除/图片缓存/在线图片)
|
||||||
|
│ ├── RemoteService.java Netty TCP 客户端连接远程存储节点 (5.255.110.45:26321+)
|
||||||
|
│ ├── WebSocketService.java WebSocket 推送下载进度给前端
|
||||||
|
│ ├── PushService.java Telegram Bot 通知 (admin 告警)
|
||||||
|
│ ├── QueryService.java E-Hentai 搜索 + 缩略图代理缓存 (转 AVIF)
|
||||||
|
│ ├── LocalServiceImpl.java 定时任务 (连接检测/额度重置/Cookie验证/订阅更新/缩略图清理)
|
||||||
|
│ ├── PublicServiceImpl.java IP 记录、授权码修改、身份查询
|
||||||
|
│ ├── PersonalServiceImpl.java 订阅最后更新时间、家里 IP、留言转发
|
||||||
|
│ ├── SubService.java 代理订阅绑定/重置/查询/更新记录
|
||||||
|
│ ├── CollectService.java 画廊收藏/取消收藏
|
||||||
|
│ └── UserServiceImpl.java 用户 CRUD + 授权码管理
|
||||||
|
│
|
||||||
|
├── Dao/
|
||||||
|
│ ├── normal/ 主库 Mapper
|
||||||
|
│ │ ├── GalleryMapper.java gallery 表 CRUD
|
||||||
|
│ │ ├── UserMapper.java user 表 CRUD
|
||||||
|
│ │ ├── CollectMapper.java collect 表 CRUD
|
||||||
|
│ │ ├── CustomConfigurationMapper.java 配置键值对读写
|
||||||
|
│ │ └── SubMapper.java 订阅绑定 & 更新记录
|
||||||
|
│ └── cache/ 缓存库 Mapper
|
||||||
|
│ └── ImageCacheMapper.java 图片 key 缓存 (gidToKey + ImageKeyCache)
|
||||||
|
│
|
||||||
|
├── Domain/ 实体类 (Lombok @Data)
|
||||||
|
│ ├── Gallery.java 画廊 (gid, name, link, pages, status, resolution, ...)
|
||||||
|
│ ├── GalleryForQuery.java 搜索结果的画廊精简信息
|
||||||
|
│ ├── GalleryTask.java 下载任务状态 (下载中/下载完成/压缩中/压缩完成)
|
||||||
|
│ ├── User.java 用户 (id, AuthCode, username, isEnable)
|
||||||
|
│ ├── GidToKey.java 画廊 GID → MPV Key 映射
|
||||||
|
│ ├── ImageKeyCache.java 图片 key 缓存 (gid, page, imgkey)
|
||||||
|
│ ├── CustomConfiguration.java 配置键常量定义
|
||||||
|
│ ├── SubBind.java 订阅绑定 (key, user)
|
||||||
|
│ ├── SubUpdateRecord.java 订阅更新记录 (ip, UA, time, location)
|
||||||
|
│ └── PageNameCache.java 页面名缓存 (gid, page, pageName)
|
||||||
|
│
|
||||||
|
├── Message/ 自定义 TCP 消息协议 (Netty)
|
||||||
|
│ ├── AbstractMessage.java 消息基类 (定义了 7 种消息类型常量)
|
||||||
|
│ ├── MessageCodec.java Netty ByteToMessageCodec 编解码器
|
||||||
|
│ ├── DownloadPostMessage.java 下发下载任务
|
||||||
|
│ ├── DownloadStatusMessage.java 下载进度状态上报
|
||||||
|
│ ├── ResponseMessage.java 通用响应
|
||||||
|
│ ├── DeleteGalleryMessage.java 删除画廊指令
|
||||||
|
│ ├── IdentityMessage.java 身份认证
|
||||||
|
│ ├── MaintainMessage.java 维护/心跳消息
|
||||||
|
│ └── AvailableCheckMessage.java 可用性检测
|
||||||
|
│
|
||||||
|
├── Interceptor/
|
||||||
|
│ ├── TaskHandlerInterceptor.java 验证 AuthCode 是否有效 (用于 /GalleryManage, /validate)
|
||||||
|
│ ├── PersonalInterceptor.java 限制 /personal/**, /remote/** 仅 AuthCode="alone"
|
||||||
|
│ └── HumanInterceptor.java 拦截无 User-Agent 的请求 (机器人防护)
|
||||||
|
│
|
||||||
|
├── Filter/
|
||||||
|
│ ├── AdaptorFilter.java UA 检测: 移动端重定向到 /mobile, 桌面端放行; 日志记录
|
||||||
|
│ └── AccessFilter.java 更新用户最后访问时间 (/validate 接口)
|
||||||
|
│
|
||||||
|
├── Util/
|
||||||
|
│ ├── Response.java 通用 JSON 响应封装 ({result, data})
|
||||||
|
│ ├── CustomUtil.java 工具类 (文件大小格式化/时间/空闲端口/404)
|
||||||
|
│ ├── GalleryUtil.java E-Hentai 网页解析/图片下载/MPV key 管理/图片格式转换
|
||||||
|
│ └── FileDownload.java 支持断点续传的文件下载工具 (Range)
|
||||||
|
│
|
||||||
|
├── Error/
|
||||||
|
│ └── ErrorCode.java 错误码常量 (IO_ERROR=1, FILE_NOT_FOUND=2, COMPRESS_ERROR=3)
|
||||||
|
│
|
||||||
|
└── Exception/
|
||||||
|
└── ResolutionNotMatchException.java 分辨率不匹配异常
|
||||||
|
|
||||||
|
三、核心功能模块
|
||||||
|
──────────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
1. E-Hentai 画廊下载管理
|
||||||
|
- 用户通过 AuthCode 提交 E-Hentai 画廊链接,指定目标分辨率
|
||||||
|
- GalleryUtil 解析页面 (Jsoup) 获取: 名称/语言/页数/文件大小/可选分辨率
|
||||||
|
- 通过 Netty TCP 将下载任务发往远程存储节点 (RemoteService)
|
||||||
|
- RemoteService 维护与存储节点的长连接 (自动重连+端口探测)
|
||||||
|
- 存储节点实时回传下载进度 (DownloadStatusMessage),通过 WebSocket 推送给前端
|
||||||
|
- 支持图片在线预览: 缓存 MPV key → 按需下载单页 → 转为 AVIF 格式
|
||||||
|
- 画廊收藏/取消收藏
|
||||||
|
|
||||||
|
2. 个人管理服务
|
||||||
|
- 手动触发订阅更新,返回最近更新时间与已记录的家庭 IP
|
||||||
|
- 管理员留言经 PushService 转发到 Telegram
|
||||||
|
|
||||||
|
(文件浏览/上传/下载/分享/压缩/删除已在 td-20260921-a1b2c3 中整体下线)
|
||||||
|
|
||||||
|
3. E-Hentai 搜索代理
|
||||||
|
- 代理搜索 exhentai.org,返回格式化结果 (含缩略图 URL)
|
||||||
|
- 缩略图代理: 下载 → ImageMagick 转 AVIF → 本地缓存 → 返回
|
||||||
|
|
||||||
|
4. 代理订阅管理
|
||||||
|
- 定时从外部链接拉取 V2Ray/Clash 订阅配置
|
||||||
|
- 过滤高倍率节点 (流量倍率 > 2)
|
||||||
|
- 为每个用户生成唯一订阅 Key,记录更新 IP/UA/时间/位置
|
||||||
|
|
||||||
|
5. Telegram 通知
|
||||||
|
- 通过 Telegram Bot 向 admin 推送: 任务提交/完成/失败、存储节点上下线、
|
||||||
|
Cookie 过期、订阅异常等
|
||||||
|
|
||||||
|
四、定时任务 (@Scheduled)
|
||||||
|
──────────────────────────────────────────────────────────────────────────────
|
||||||
|
- 每 30 分钟: 检测存储节点连接,断开则自动重连
|
||||||
|
- 每周一 4:00: 重置每周下载额度
|
||||||
|
- 每天 0:00: 验证 E-Hentai Cookie 有效性
|
||||||
|
- 每 24 小时: 更新代理订阅配置文件
|
||||||
|
- 每周一 4:00: 清理缩略图缓存 (保留最近 10000 个)
|
||||||
|
|
||||||
|
五、安全机制
|
||||||
|
──────────────────────────────────────────────────────────────────────────────
|
||||||
|
- 所有管理接口需 AuthCode 参数 (TaskHandlerInterceptor 校验)
|
||||||
|
- /personal 和 /remote 路径限 AuthCode="alone" 用户
|
||||||
|
- HumanInterceptor 拒绝无 User-Agent 请求
|
||||||
|
- AdaptorFilter 记录所有请求日志 (IP/路径/UA/时间)
|
||||||
|
|
||||||
|
六、依赖
|
||||||
|
──────────────────────────────────────────────────────────────────────────────
|
||||||
|
spring-boot-starter-webmvc, spring-boot-starter-websocket, mybatis-spring-boot-starter 4.1
|
||||||
|
jsoup (HTML 解析), hutool-all (工具集), sqlite-jdbc (数据库)
|
||||||
|
httpclient5 (HTTP 请求)
|
||||||
|
commons-io, commons-lang3, netty-all (TCP 通信)
|
||||||
|
java-telegram-bot-api (Telegram Bot), graalvm native-maven-plugin (AOT)
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
End of Summary
|
||||||
|
================================================================================
|
||||||
@@ -5,7 +5,7 @@
|
|||||||
<parent>
|
<parent>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-starter-parent</artifactId>
|
<artifactId>spring-boot-starter-parent</artifactId>
|
||||||
<version>3.1.1</version>
|
<version>4.1.1</version>
|
||||||
<relativePath/> <!-- lookup parent from repository -->
|
<relativePath/> <!-- lookup parent from repository -->
|
||||||
</parent>
|
</parent>
|
||||||
<groupId>com.lion</groupId>
|
<groupId>com.lion</groupId>
|
||||||
@@ -14,26 +14,30 @@
|
|||||||
<name>LionWebsite</name>
|
<name>LionWebsite</name>
|
||||||
<description>LionWebsite</description>
|
<description>LionWebsite</description>
|
||||||
<properties>
|
<properties>
|
||||||
<java.version>17</java.version>
|
<java.version>21</java.version>
|
||||||
|
<tomcat.version>11.0.27</tomcat.version>
|
||||||
|
<jackson-bom.version>3.1.7</jackson-bom.version>
|
||||||
</properties>
|
</properties>
|
||||||
<dependencies>
|
<dependencies>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.yaml</groupId>
|
||||||
<artifactId>spring-boot-starter-thymeleaf</artifactId>
|
<artifactId>snakeyaml</artifactId>
|
||||||
|
<version>2.6</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-starter-web</artifactId>
|
<artifactId>spring-boot-starter-webmvc</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.mybatis.spring.boot</groupId>
|
<groupId>org.mybatis.spring.boot</groupId>
|
||||||
<artifactId>mybatis-spring-boot-starter</artifactId>
|
<artifactId>mybatis-spring-boot-starter</artifactId>
|
||||||
<version>3.0.2</version>
|
<version>4.1.0</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.projectlombok</groupId>
|
<groupId>org.projectlombok</groupId>
|
||||||
<artifactId>lombok</artifactId>
|
<artifactId>lombok</artifactId>
|
||||||
|
<version>1.18.46</version>
|
||||||
<optional>true</optional>
|
<optional>true</optional>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
@@ -44,20 +48,20 @@
|
|||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.mybatis.spring.boot</groupId>
|
<groupId>org.mybatis.spring.boot</groupId>
|
||||||
<artifactId>mybatis-spring-boot-starter-test</artifactId>
|
<artifactId>mybatis-spring-boot-starter-test</artifactId>
|
||||||
<version>3.0.2</version>
|
<version>4.1.0</version>
|
||||||
<scope>test</scope>
|
<scope>test</scope>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.jsoup</groupId>
|
<groupId>org.jsoup</groupId>
|
||||||
<artifactId>jsoup</artifactId>
|
<artifactId>jsoup</artifactId>
|
||||||
<version>1.15.3</version>
|
<version>1.23.2</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>cn.hutool</groupId>
|
<groupId>cn.hutool</groupId>
|
||||||
<artifactId>hutool-all</artifactId>
|
<artifactId>hutool-all</artifactId>
|
||||||
<version>5.8.11</version>
|
<version>5.8.47</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
@@ -66,22 +70,8 @@
|
|||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.httpcomponents</groupId>
|
<groupId>org.apache.httpcomponents.client5</groupId>
|
||||||
<artifactId>httpclient</artifactId>
|
<artifactId>httpclient5</artifactId>
|
||||||
<version>4.5.14</version>
|
|
||||||
</dependency>
|
|
||||||
|
|
||||||
|
|
||||||
<dependency>
|
|
||||||
<groupId>org.apache.httpcomponents</groupId>
|
|
||||||
<artifactId>httpmime</artifactId>
|
|
||||||
<version>4.5.14</version>
|
|
||||||
</dependency>
|
|
||||||
|
|
||||||
<dependency>
|
|
||||||
<groupId>org.im4java</groupId>
|
|
||||||
<artifactId>im4java</artifactId>
|
|
||||||
<version>1.4.0</version>
|
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
@@ -92,24 +82,103 @@
|
|||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.apache.commons</groupId>
|
<groupId>org.apache.commons</groupId>
|
||||||
<artifactId>commons-compress</artifactId>
|
<artifactId>commons-compress</artifactId>
|
||||||
<version>1.21</version>
|
<version>1.28.0</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>commons-io</groupId>
|
<groupId>commons-io</groupId>
|
||||||
<artifactId>commons-io</artifactId>
|
<artifactId>commons-io</artifactId>
|
||||||
<version>2.11.0</version>
|
<version>2.22.0</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>io.netty</groupId>
|
<groupId>io.netty</groupId>
|
||||||
<artifactId>netty-all</artifactId>
|
<!--
|
||||||
<version>4.1.86.Final</version>
|
只保留实际用到的模块。原先的 netty-all 是聚合 pom,会拖进 50 个模块
|
||||||
|
(含 5 个平台的 native-quic、aarch64/riscv64/osx 的 epoll/kqueue/io_uring、
|
||||||
|
codec-http3/mqtt/redis/smtp/stomp/xml/protobuf 等),
|
||||||
|
而本项目只用 Bootstrap/NioEventLoopGroup/NioSocketChannel/ByteBuf/
|
||||||
|
ByteToMessageCodec/LengthFieldBasedFrameDecoder/LoggingHandler/Promise。
|
||||||
|
transport 提供 Bootstrap/EventLoop/Channel,codec-base 提供
|
||||||
|
ByteToMessageCodec 与 LengthFieldBasedFrameDecoder,handler 提供
|
||||||
|
LoggingHandler(RemoteService 的协议调试日志用)。
|
||||||
|
buffer/common/resolver 由它们传递引入。
|
||||||
|
-->
|
||||||
|
<artifactId>netty-transport</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>io.netty</groupId>
|
||||||
|
<artifactId>netty-codec-base</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>io.netty</groupId>
|
||||||
|
<artifactId>netty-handler</artifactId>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>com.github.pengrad</groupId>
|
||||||
|
<artifactId>java-telegram-bot-api</artifactId>
|
||||||
|
<version>10.1.0</version>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-websocket</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
</dependencies>
|
</dependencies>
|
||||||
|
|
||||||
<build>
|
<build>
|
||||||
<plugins>
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-compiler-plugin</artifactId>
|
||||||
|
<configuration>
|
||||||
|
<annotationProcessorPaths>
|
||||||
|
<path>
|
||||||
|
<groupId>org.projectlombok</groupId>
|
||||||
|
<artifactId>lombok</artifactId>
|
||||||
|
<version>1.18.46</version>
|
||||||
|
</path>
|
||||||
|
</annotationProcessorPaths>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.jacoco</groupId>
|
||||||
|
<artifactId>jacoco-maven-plugin</artifactId>
|
||||||
|
<version>0.8.13</version>
|
||||||
|
<executions>
|
||||||
|
<execution>
|
||||||
|
<id>prepare-agent</id>
|
||||||
|
<goals>
|
||||||
|
<goal>prepare-agent</goal>
|
||||||
|
</goals>
|
||||||
|
</execution>
|
||||||
|
<execution>
|
||||||
|
<id>report</id>
|
||||||
|
<phase>test</phase>
|
||||||
|
<goals>
|
||||||
|
<goal>report</goal>
|
||||||
|
</goals>
|
||||||
|
</execution>
|
||||||
|
</executions>
|
||||||
|
</plugin>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.graalvm.buildtools</groupId>
|
||||||
|
<artifactId>native-maven-plugin</artifactId>
|
||||||
|
<version>1.1.8</version>
|
||||||
|
<configuration>
|
||||||
|
<imageName>lionwebsite</imageName>
|
||||||
|
<buildArgs>
|
||||||
|
<arg>-H:+ReportExceptionStackTraces</arg>
|
||||||
|
<arg>--initialize-at-build-time=org.apache.commons.logging.LogFactory,org.apache.commons.logging.LogFactoryService,org.apache.commons.logging.impl.WeakHashtable,org.sqlite.util.ProcessRunner</arg>
|
||||||
|
</buildArgs>
|
||||||
|
<metadataRepository>
|
||||||
|
<enabled>true</enabled>
|
||||||
|
</metadataRepository>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
<plugin>
|
<plugin>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||||
@@ -122,14 +191,6 @@
|
|||||||
</excludes>
|
</excludes>
|
||||||
</configuration>
|
</configuration>
|
||||||
</plugin>
|
</plugin>
|
||||||
<plugin>
|
|
||||||
<groupId>org.apache.maven.plugins</groupId>
|
|
||||||
<artifactId>maven-compiler-plugin</artifactId>
|
|
||||||
<configuration>
|
|
||||||
<source>17</source>
|
|
||||||
<target>17</target>
|
|
||||||
</configuration>
|
|
||||||
</plugin>
|
|
||||||
</plugins>
|
</plugins>
|
||||||
</build>
|
</build>
|
||||||
|
|
||||||
|
|||||||
Executable
+151
@@ -0,0 +1,151 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# 图片索引(cache.db)迁移:为缓存表补唯一约束与查询索引。
|
||||||
|
#
|
||||||
|
# 设计约束:
|
||||||
|
# * 默认 dry-run,只读检查;必须显式 --apply 才执行写入。
|
||||||
|
# * 必须显式 --db 指定数据库文件,不读取任何运行时配置,也不推断生产路径。
|
||||||
|
# * 写库用 sqlite3 -cmd '.bail on',任一语句出错立即以非零码退出,事务不提交。
|
||||||
|
# * 只以读写模式打开目标库(文件必须已存在,不会新建);dry-run 全用 -readonly。
|
||||||
|
# * 备份用 SQLite 自身的 .backup(online backup API),而不是 cp:
|
||||||
|
# cp 会漏掉 WAL 里尚未 checkpoint 的已提交内容。
|
||||||
|
# * 备份后立即校验备份可读且含两张缓存表,失败则中止并删除无效备份。
|
||||||
|
# * 不健康(不完整/冲突)的 gid 整组索引会被删除,由下一次在线阅读重新抓取。
|
||||||
|
# * 唯一约束建在唯一索引上:若清理后仍有重复,建索引会失败并整体回滚。
|
||||||
|
#
|
||||||
|
# 用法:
|
||||||
|
# scripts/migrate_image_index_cache.sh --db /tmp/cache-copy.db # dry-run(只读)
|
||||||
|
# scripts/migrate_image_index_cache.sh --db /tmp/cache-copy.db --apply # 执行迁移
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
SQL_FILE="$SCRIPT_DIR/migrate_image_index_cache.sql"
|
||||||
|
|
||||||
|
db_path=""
|
||||||
|
apply="no"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
sed -n '2,30p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--db) db_path="${2:-}"; shift 2 ;;
|
||||||
|
--apply) apply="yes"; shift ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*)
|
||||||
|
echo "未知参数: $1" >&2
|
||||||
|
exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ -z "$db_path" ]]; then
|
||||||
|
echo "必须显式指定缓存库: --db <cache.db>;本脚本不会自动推断数据库位置。" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$db_path" ]]; then
|
||||||
|
echo "数据库不存在: $db_path" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "$SQL_FILE" ]]; then
|
||||||
|
echo "缺少 SQL: $SQL_FILE" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v sqlite3 >/dev/null 2>&1; then
|
||||||
|
echo "缺少 sqlite3 命令行工具" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$(sqlite3 -readonly "$db_path" \
|
||||||
|
"select count(*) from sqlite_master where type='table' and name in ('gidToKey','ImageKeyCache');")" != "2" ]]; then
|
||||||
|
echo "缓存表缺失:$db_path 中找不到 gidToKey 与 ImageKeyCache。" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 体检统计,与 SQL 里实际清理的范围一致:
|
||||||
|
# bad_gids 将被整组删除的 gid 个数(去重),覆盖:
|
||||||
|
# 重复头部 / 行数不符 / 页码去重后不足 / 页码区间异常 /
|
||||||
|
# 空 imgkey / 空 key / 非法 pages
|
||||||
|
# bad_gid_rows 这些 gid 占用的头部行数(重复头部的 gid 会多于 1 行)
|
||||||
|
# duplicate_pages 重复的 (gid,page) 组数,仅用于展示,会随所属 gid 一并清理
|
||||||
|
# orphan_gids 有页 key 但无头部记录的 gid 个数
|
||||||
|
read_stats() {
|
||||||
|
sqlite3 -readonly "$db_path" "
|
||||||
|
select 'header_rows', count(*) from gidToKey;
|
||||||
|
select 'page_rows', count(*) from ImageKeyCache;
|
||||||
|
select 'duplicate_gids', count(*) from (select gid from gidToKey group by gid having count(*) > 1);
|
||||||
|
select 'duplicate_pages', count(*) from (select gid,page from ImageKeyCache group by gid,page having count(*) > 1);
|
||||||
|
select 'bad_gids', count(distinct g.gid) from gidToKey g
|
||||||
|
where g.pages is null or g.pages <= 0
|
||||||
|
or g.key is null or trim(g.key) = ''
|
||||||
|
or (select count(*) from ImageKeyCache i where i.gid = g.gid) <> g.pages
|
||||||
|
or (select count(distinct i.page) from ImageKeyCache i where i.gid = g.gid) <> g.pages
|
||||||
|
or (select min(i.page) from ImageKeyCache i where i.gid = g.gid) <> 1
|
||||||
|
or (select max(i.page) from ImageKeyCache i where i.gid = g.gid) <> g.pages
|
||||||
|
or exists (select 1 from ImageKeyCache i
|
||||||
|
where i.gid = g.gid and (i.imgkey is null or trim(i.imgkey) = ''));
|
||||||
|
select 'bad_gid_rows', coalesce(sum(c), 0) from (
|
||||||
|
select count(*) as c from gidToKey g where g.gid in (
|
||||||
|
select g2.gid from gidToKey g2 where g2.pages is null or g2.pages <= 0
|
||||||
|
or g2.key is null or trim(g2.key) = ''
|
||||||
|
or (select count(*) from ImageKeyCache i where i.gid = g2.gid) <> g2.pages
|
||||||
|
or (select count(distinct i.page) from ImageKeyCache i where i.gid = g2.gid) <> g2.pages
|
||||||
|
or (select min(i.page) from ImageKeyCache i where i.gid = g2.gid) <> 1
|
||||||
|
or (select max(i.page) from ImageKeyCache i where i.gid = g2.gid) <> g2.pages
|
||||||
|
or exists (select 1 from ImageKeyCache i
|
||||||
|
where i.gid = g2.gid and (i.imgkey is null or trim(i.imgkey) = '')))
|
||||||
|
group by g.gid);
|
||||||
|
select 'orphan_gids', count(distinct gid) from ImageKeyCache
|
||||||
|
where gid not in (select gid from gidToKey);
|
||||||
|
select 'strict_index', count(*) from sqlite_master
|
||||||
|
where type='index' and name='ux_ImageKeyCache_gid_page';
|
||||||
|
"
|
||||||
|
}
|
||||||
|
|
||||||
|
mode_label="dry-run(只读)"
|
||||||
|
if [[ "$apply" == "yes" ]]; then mode_label="apply(读写)"; fi
|
||||||
|
|
||||||
|
echo "=== 缓存库图片索引迁移($mode_label) ==="
|
||||||
|
echo "数据库: $db_path"
|
||||||
|
echo "--- 迁移前体检 ---"
|
||||||
|
read_stats | while IFS='|' read -r name value; do
|
||||||
|
printf ' %-16s %s\n' "$name" "$value"
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ "$apply" != "yes" ]]; then
|
||||||
|
echo
|
||||||
|
echo "dry-run:未做任何写入。确认无误后加 --apply 执行迁移。"
|
||||||
|
echo "提示:请先在 SQLite 副本上演练,副本应这样生成(含 WAL 内容):"
|
||||||
|
echo " sqlite3 -readonly <生产库> \".backup '<副本路径>'\""
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ---- 备份:SQLite online backup,包含 WAL 中已提交但未 checkpoint 的内容 ----
|
||||||
|
backup="$db_path.before-image-index.$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
if ! sqlite3 "$db_path" ".backup '$backup'"; then
|
||||||
|
echo "备份失败,未做任何修改: $backup" >&2
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
if [[ ! -s "$backup" ]]; then
|
||||||
|
echo "备份文件为空,未做任何修改: $backup" >&2
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
if [[ "$(sqlite3 -readonly "$backup" \
|
||||||
|
"select count(*) from sqlite_master where type='table' and name in ('gidToKey','ImageKeyCache');")" != "2" ]]; then
|
||||||
|
echo "备份校验失败(缺少缓存表),未做任何修改,已删除无效备份。" >&2
|
||||||
|
rm -f -- "$backup"
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
echo "已备份(SQLite .backup): $backup"
|
||||||
|
|
||||||
|
# ---- 执行迁移:.bail on 保证任一语句失败即退出,事务不提交 ----
|
||||||
|
sqlite3 -cmd '.bail on' "$db_path" < "$SQL_FILE"
|
||||||
|
|
||||||
|
echo "--- 迁移后体检 ---"
|
||||||
|
read_stats | while IFS='|' read -r name value; do
|
||||||
|
printf ' %-16s %s\n' "$name" "$value"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "图片索引迁移完成: $db_path"
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
-- 图片索引缓存迁移:为 cache.db 的两张索引表补唯一约束与查询索引。
|
||||||
|
--
|
||||||
|
-- 目标(对应本次重写的三条不变量):
|
||||||
|
-- 1. gid 唯一 —— 同一本画廊只能有一条 gidToKey 记录;
|
||||||
|
-- 2. (gid,page) 唯一 —— 同一页只能有一个 imgkey,同时天然满足按页查询的索引需求;
|
||||||
|
-- 3. 只保留「完整」的索引 —— 半截或冲突的 gid 整组删除,让下一次在线阅读重新抓取。
|
||||||
|
--
|
||||||
|
-- 为什么用唯一索引而不是重建表:
|
||||||
|
-- * 唯一索引对 SQLite 与唯一约束等价,都会拒绝重复写入;
|
||||||
|
-- * 不需要搬表、改列,旧版本二进制仍可读写同一份数据;
|
||||||
|
-- * 回滚只需 drop index,最坏情况也只需回到脚本自动生成的备份文件。
|
||||||
|
--
|
||||||
|
-- 幂等:所有重建索引都带 IF NOT EXISTS;清理语句只按集合删除,重复执行结果相同。
|
||||||
|
-- 安全:本脚本只做 DELETE 与 CREATE INDEX,不建表、不改列、不碰 gidToKey/ImageKeyCache
|
||||||
|
-- 之外的任何对象。必须在事务中整体提交,调用方使用 sqlite3 -bail 保证任一语句
|
||||||
|
-- 失败即终止、事务回滚。
|
||||||
|
--
|
||||||
|
-- 执行前请务必先备份数据库文件(迁移脚本 migrate_image_index_cache.sh --apply 会自动备份)。
|
||||||
|
|
||||||
|
BEGIN IMMEDIATE;
|
||||||
|
|
||||||
|
-- ---------- 1. 冲突 gid:同一 gid 有多条头部记录 ----------
|
||||||
|
-- 无法判断哪一条正确,整组移除,由下次访问重抓。
|
||||||
|
CREATE TEMP TABLE __conflict_gid AS
|
||||||
|
SELECT gid FROM gidToKey GROUP BY gid HAVING COUNT(*) > 1;
|
||||||
|
|
||||||
|
DELETE FROM ImageKeyCache WHERE gid IN (SELECT gid FROM __conflict_gid);
|
||||||
|
DELETE FROM gidToKey WHERE gid IN (SELECT gid FROM __conflict_gid);
|
||||||
|
|
||||||
|
-- ---------- 2. 半截或非法 gid:头部与页 key 不一致 ----------
|
||||||
|
-- 判定同时覆盖:页码数量不符、页码不连续(跳号/重复)、区间不是 1..pages、
|
||||||
|
-- 头部字段为空、以及存在空的 imgkey 行。
|
||||||
|
CREATE TEMP TABLE __broken_gid AS
|
||||||
|
SELECT g.gid
|
||||||
|
FROM gidToKey g
|
||||||
|
WHERE g.pages IS NULL OR g.pages <= 0
|
||||||
|
OR g.key IS NULL OR trim(g.key) = ''
|
||||||
|
OR (SELECT COUNT(*) FROM ImageKeyCache i WHERE i.gid = g.gid) <> g.pages
|
||||||
|
OR (SELECT COUNT(DISTINCT i.page) FROM ImageKeyCache i WHERE i.gid = g.gid) <> g.pages
|
||||||
|
OR (SELECT MIN(i.page) FROM ImageKeyCache i WHERE i.gid = g.gid) <> 1
|
||||||
|
OR (SELECT MAX(i.page) FROM ImageKeyCache i WHERE i.gid = g.gid) <> g.pages
|
||||||
|
OR EXISTS (SELECT 1 FROM ImageKeyCache i
|
||||||
|
WHERE i.gid = g.gid AND (i.imgkey IS NULL OR trim(i.imgkey) = ''));
|
||||||
|
|
||||||
|
DELETE FROM ImageKeyCache WHERE gid IN (SELECT gid FROM __broken_gid);
|
||||||
|
DELETE FROM gidToKey WHERE gid IN (SELECT gid FROM __broken_gid);
|
||||||
|
|
||||||
|
-- ---------- 3. 孤儿页 key:没有对应头部记录 ----------
|
||||||
|
DELETE FROM ImageKeyCache WHERE gid NOT IN (SELECT gid FROM gidToKey);
|
||||||
|
|
||||||
|
-- ---------- 4. 唯一约束 ----------
|
||||||
|
-- 到这里若仍有重复,说明上面的判定有遗漏;CREATE UNIQUE INDEX 会失败,
|
||||||
|
-- 配合 -bail 让整个事务回滚,不会留下「一半约束」的库。
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS ux_gidToKey_gid
|
||||||
|
ON gidToKey (gid);
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS ux_ImageKeyCache_gid_page
|
||||||
|
ON ImageKeyCache (gid, page);
|
||||||
|
|
||||||
|
-- 按 gid 统计页数会用到 (gid,page) 前缀,按页取 key 也走同一索引,
|
||||||
|
-- 因此不再单独建 page 索引,避免重复占用写放大。
|
||||||
|
|
||||||
|
DROP TABLE __conflict_gid;
|
||||||
|
DROP TABLE __broken_gid;
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
Executable
+18
@@ -0,0 +1,18 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
db_path="${1:-LionWebsite.db}"
|
||||||
|
legacy_key="${2:-}"
|
||||||
|
|
||||||
|
if [[ -z "$legacy_key" || ! "$legacy_key" =~ ^[A-Za-z0-9._~-]+$ ]]; then
|
||||||
|
echo "用法: $0 <LionWebsite.db> <现有共享订阅 upstream key>" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
if [[ ! -f "$db_path" ]]; then
|
||||||
|
echo "数据库不存在: $db_path" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
sqlite3 "$db_path" -cmd ".parameter init" -cmd ".parameter set :legacy_key '$legacy_key'" \
|
||||||
|
< "$(dirname "$0")/migrate_subscription_accounts.sql"
|
||||||
|
echo "订阅子账号迁移完成: $db_path"
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
BEGIN;
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS subscription_account (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
name VARCHAR(100) NOT NULL UNIQUE,
|
||||||
|
upstream_key VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
filter_high_multiplier INTEGER NOT NULL DEFAULT 1,
|
||||||
|
enabled INTEGER NOT NULL DEFAULT 1,
|
||||||
|
last_success_at DATETIME,
|
||||||
|
last_error TEXT,
|
||||||
|
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO subscription_account (name, upstream_key, filter_high_multiplier, enabled)
|
||||||
|
SELECT '旧共享订阅', :legacy_key, 1, 1
|
||||||
|
WHERE NOT EXISTS (SELECT 1 FROM subscription_account WHERE name = '旧共享订阅')
|
||||||
|
AND NOT EXISTS (SELECT 1 FROM subscription_account WHERE upstream_key = :legacy_key);
|
||||||
|
|
||||||
|
CREATE TABLE sub_bind_new (
|
||||||
|
key VARCHAR(255) NOT NULL PRIMARY KEY,
|
||||||
|
user VARCHAR(255) NOT NULL UNIQUE,
|
||||||
|
subscription_account_id INTEGER NOT NULL,
|
||||||
|
FOREIGN KEY (subscription_account_id) REFERENCES subscription_account(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO sub_bind_new (key, user, subscription_account_id)
|
||||||
|
SELECT sb.key, sb.user, sa.id
|
||||||
|
FROM sub_bind sb
|
||||||
|
JOIN (SELECT id FROM subscription_account
|
||||||
|
WHERE name = '旧共享订阅' OR upstream_key = :legacy_key
|
||||||
|
ORDER BY id LIMIT 1) sa;
|
||||||
|
|
||||||
|
DROP TABLE sub_bind;
|
||||||
|
ALTER TABLE sub_bind_new RENAME TO sub_bind;
|
||||||
|
CREATE INDEX idx_sub_bind_account ON sub_bind(subscription_account_id);
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Create durable subscription event receipts/history, with a SQLite backup.
|
||||||
|
|
||||||
|
Stop the production backend before --apply and pass --service-stopped to confirm.
|
||||||
|
No automatic startup DDL exists. Dry run is read-only; only hashes enter history.
|
||||||
|
Receipt retention is independent of the most recent ten visible update records.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import datetime
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
DDL = (
|
||||||
|
"""CREATE TABLE IF NOT EXISTS subscription_event_receipt (
|
||||||
|
event_id TEXT PRIMARY KEY NOT NULL, key_hash TEXT NOT NULL,
|
||||||
|
client TEXT NOT NULL, occurred_at INTEGER NOT NULL, source TEXT NOT NULL,
|
||||||
|
ingested_at INTEGER NOT NULL, fingerprint TEXT NOT NULL, outcome TEXT NOT NULL
|
||||||
|
)""",
|
||||||
|
"""CREATE TABLE IF NOT EXISTS subscription_event_binding (
|
||||||
|
key_hash TEXT PRIMARY KEY NOT NULL, user_id INTEGER,
|
||||||
|
user_name TEXT NOT NULL, first_seen_at INTEGER NOT NULL, last_seen_at INTEGER NOT NULL
|
||||||
|
)""",
|
||||||
|
"CREATE INDEX IF NOT EXISTS idx_subscription_event_binding_user ON subscription_event_binding(user_id)",
|
||||||
|
"CREATE INDEX IF NOT EXISTS idx_sub_record_user_time ON sub_update_record(user_id,time DESC,id DESC)",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def migrate(database: Path, apply=False, service_stopped=False):
|
||||||
|
if not database.is_file():
|
||||||
|
raise ValueError("database must already exist")
|
||||||
|
if apply and not service_stopped:
|
||||||
|
raise ValueError("stop backend and supply --service-stopped before applying")
|
||||||
|
uri = database.resolve().as_uri() + ("?mode=rw" if apply else "?mode=ro")
|
||||||
|
with sqlite3.connect(uri, uri=True) as db:
|
||||||
|
for table in ("User", "sub_bind", "sub_update_record"):
|
||||||
|
columns = {row[1] for row in db.execute(f'PRAGMA table_info("{table}")')}
|
||||||
|
if not columns or (table != "User" and "user_id" not in columns):
|
||||||
|
raise ValueError("stable user links migration is required first")
|
||||||
|
unresolved = db.execute("SELECT count(*) FROM sub_bind sb LEFT JOIN User u ON u.id=sb.user_id WHERE sb.user_id IS NOT NULL AND u.id IS NULL").fetchone()[0]
|
||||||
|
if unresolved:
|
||||||
|
raise ValueError("resolve non-null dangling subscription user IDs before migration")
|
||||||
|
tables = {row[0] for row in db.execute("SELECT name FROM sqlite_master WHERE type='table'")}
|
||||||
|
result = {"apply": apply, "tables_to_create": sorted({"subscription_event_receipt", "subscription_event_binding"} - tables)}
|
||||||
|
if not apply:
|
||||||
|
return result
|
||||||
|
stamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
|
||||||
|
backup = database.with_name(database.name + ".before-subscription-events." + stamp)
|
||||||
|
# Create mode 600 before any sensitive database bytes are written.
|
||||||
|
backup.touch(mode=0o600, exist_ok=False)
|
||||||
|
with sqlite3.connect(backup) as snapshot:
|
||||||
|
db.backup(snapshot)
|
||||||
|
try:
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
|
for statement in DDL:
|
||||||
|
db.execute(statement)
|
||||||
|
now = int(datetime.datetime.now(datetime.timezone.utc).timestamp() * 1000)
|
||||||
|
for key, user_id, name in db.execute("SELECT sb.key,sb.user_id,coalesce(u.username,sb.user) FROM sub_bind sb LEFT JOIN User u ON u.id=sb.user_id"):
|
||||||
|
key_hash = hashlib.sha256(key.encode("utf-8")).hexdigest()
|
||||||
|
db.execute("""INSERT INTO subscription_event_binding VALUES(?,?,?,?,?)
|
||||||
|
ON CONFLICT(key_hash) DO UPDATE SET user_name=excluded.user_name,last_seen_at=excluded.last_seen_at
|
||||||
|
WHERE subscription_event_binding.user_id IS excluded.user_id""", (key_hash, user_id, name, now, now))
|
||||||
|
db.commit()
|
||||||
|
except BaseException:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
result["backup"] = str(backup)
|
||||||
|
result["history_bindings"] = db.execute("SELECT count(*) FROM subscription_event_binding").fetchone()[0]
|
||||||
|
result["integrity"] = db.execute("PRAGMA quick_check").fetchone()[0]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("database", type=Path)
|
||||||
|
parser.add_argument("--apply", action="store_true")
|
||||||
|
parser.add_argument("--service-stopped", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
print(json.dumps(migrate(args.database, args.apply, args.service_stopped), ensure_ascii=False))
|
||||||
Executable
+29
@@ -0,0 +1,29 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
db_path="${1:-LionWebsite.db}"
|
||||||
|
|
||||||
|
if [[ ! -f "$db_path" ]]; then
|
||||||
|
echo "数据库不存在: $db_path" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if command -v systemctl >/dev/null 2>&1 && systemctl is-active --quiet lionwebsite 2>/dev/null; then
|
||||||
|
echo "检测到 lionwebsite.service 仍在运行;请先停止服务再迁移,避免写冲突。" >&2
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
|
||||||
|
if pgrep -f 'java .*lionwebsite\.jar' >/dev/null 2>&1; then
|
||||||
|
echo "检测到主站进程仍在运行;请先停止服务再迁移,避免写冲突。" >&2
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
|
||||||
|
backup="${db_path}.before-refresh-schedule.$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
cp "$db_path" "$backup"
|
||||||
|
echo "已备份: $backup"
|
||||||
|
|
||||||
|
sqlite3 "$db_path" < "$(dirname "$0")/migrate_subscription_refresh.sql"
|
||||||
|
|
||||||
|
echo "--- 校验 ---"
|
||||||
|
sqlite3 "$db_path" "select count(*) as accounts, sum(next_refresh_at is null) as missing_schedule from subscription_account;"
|
||||||
|
echo "订阅刷新计划迁移完成: $db_path"
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
-- 为每账号分散刷新增加计划字段。
|
||||||
|
-- 纯新增列:旧版本二进制忽略这两列,因此回滚程序时不必回滚数据库。
|
||||||
|
BEGIN;
|
||||||
|
|
||||||
|
-- next_refresh_at:该子账号下一次应刷新的时刻,Epoch 毫秒。
|
||||||
|
-- 用整数存时刻,避免 SQLite 文本时间戳被按本地时区解释(实证存在 8 小时偏差)。
|
||||||
|
-- last_success_epoch:最近一次成功刷新的真实时刻,Epoch 毫秒。
|
||||||
|
-- 既有的 last_success_at 保持原样,仅供页面展示,不参与调度判断。
|
||||||
|
ALTER TABLE subscription_account ADD COLUMN next_refresh_at INTEGER;
|
||||||
|
ALTER TABLE subscription_account ADD COLUMN last_success_epoch INTEGER;
|
||||||
|
|
||||||
|
-- 回填 last_success_epoch:SQLite 的 strftime 把文本时间按 UTC 解释,而
|
||||||
|
-- CURRENT_TIMESTAMP 写入的正是 UTC,因此这里能得到正确时刻(JDBC 读取则会偏 8 小时)。
|
||||||
|
-- 必须回填:否则所有既有账号都会被当成「从未成功过」,迁移后立刻集中补刷一遍,
|
||||||
|
-- 正好复现本次改造要消除的爆发。
|
||||||
|
UPDATE subscription_account
|
||||||
|
SET last_success_epoch = CAST(strftime('%s', last_success_at) AS INTEGER) * 1000
|
||||||
|
WHERE last_success_epoch IS NULL AND last_success_at IS NOT NULL;
|
||||||
|
|
||||||
|
-- 已有账号在窗口内按 id 错开,避免迁移后同一 tick 集中开火。
|
||||||
|
-- 以迁移时刻为基准按 37 分钟步长铺开(12 个账号约 7.4 小时排完)。
|
||||||
|
UPDATE subscription_account
|
||||||
|
SET next_refresh_at = (CAST(strftime('%s', 'now') AS INTEGER) * 1000) + (id * 37 * 60 * 1000)
|
||||||
|
WHERE next_refresh_at IS NULL;
|
||||||
|
|
||||||
|
COMMIT;
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
-- 任务操作记录表(协议 v2:先持久化意图,再产生外部副作用)
|
||||||
|
--
|
||||||
|
-- 这是显式迁移脚本,不会被应用自动执行,也不对生产库运行。
|
||||||
|
-- 执行前请备份:sqlite3 LionWebsite.db ".backup LionWebsite.db.before-task-operation"
|
||||||
|
--
|
||||||
|
-- 幂等:可重复执行。
|
||||||
|
|
||||||
|
create table if not exists task_operation
|
||||||
|
(
|
||||||
|
gid integer not null
|
||||||
|
constraint task_operation_pk
|
||||||
|
primary key,
|
||||||
|
operationId varchar(64),
|
||||||
|
deleteOperationId varchar(64),
|
||||||
|
state varchar(20) not null,
|
||||||
|
lastRevision integer not null default 0,
|
||||||
|
lastReportId varchar(64),
|
||||||
|
updateTime long not null default 0
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 节点上报消费只按 gid 命中主键,这里额外为「按状态找待处理意图」提供查询入口。
|
||||||
|
create index if not exists idx_task_operation_state on task_operation (state);
|
||||||
|
|
||||||
|
-- gallery.gid 目前只有 table 级主键约束(部分历史库甚至是自增主键的衍生约束),
|
||||||
|
-- 这里显式建立唯一索引,保证同一 gid 不会出现两条任务行:
|
||||||
|
-- 「同一 gid 只有一个有效意图」是 v2 生命周期成立的前提。
|
||||||
|
-- 若该语句因历史重复数据失败,请先人工确认重复行来源再处理,不要直接删除。
|
||||||
|
create unique index if not exists idx_gallery_gid_unique on gallery (gid);
|
||||||
|
|
||||||
|
-- 旧任务行没有操作记录:启动/查询时按 gid 惰性补建为 SUBMITTED(已在上游提交过),
|
||||||
|
-- 不需要在迁移里批量写入,避免把正在进行的任务判成 UNKNOWN。
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Explicit task-v2 schema migration. Default is read-only; --apply backs up first."""
|
||||||
|
import argparse
|
||||||
|
import datetime
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
|
||||||
|
def migrate(database: Path, apply: bool = False) -> dict:
|
||||||
|
if not database.is_file():
|
||||||
|
raise ValueError("database must already exist")
|
||||||
|
with sqlite3.connect(database.resolve().as_uri() + ("?mode=rw" if apply else "?mode=ro"), uri=True) as db:
|
||||||
|
if not db.execute("pragma table_info(gallery)").fetchall():
|
||||||
|
raise ValueError("required table missing: gallery")
|
||||||
|
duplicates = db.execute("select count(*) from (select gid from gallery group by gid having count(*)>1)").fetchone()[0]
|
||||||
|
if duplicates:
|
||||||
|
raise ValueError("duplicate gallery gid; reconcile before migration")
|
||||||
|
present = bool(db.execute("pragma table_info(task_operation)").fetchall())
|
||||||
|
result = {"apply": apply, "create_operation_table": not present}
|
||||||
|
if not apply:
|
||||||
|
return result
|
||||||
|
stamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
|
||||||
|
backup = database.with_name(database.name + ".before-task-v2." + stamp)
|
||||||
|
with sqlite3.connect(backup) as copy:
|
||||||
|
db.backup(copy)
|
||||||
|
backup.chmod(0o600)
|
||||||
|
sql = Path(__file__).with_name("migrate_task_operation_v2.sql").read_text()
|
||||||
|
try:
|
||||||
|
db.executescript("BEGIN IMMEDIATE;\n" + sql + "\nCOMMIT;")
|
||||||
|
except BaseException:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
result["backup"] = str(backup)
|
||||||
|
result["operation_rows"] = db.execute("select count(*) from task_operation").fetchone()[0]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("database", type=Path)
|
||||||
|
parser.add_argument("--apply", action="store_true")
|
||||||
|
arguments = parser.parse_args()
|
||||||
|
print(json.dumps(migrate(arguments.database, arguments.apply), ensure_ascii=False))
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Add stable user identity to subscription bindings and access records.
|
||||||
|
|
||||||
|
The default invocation only reports schema changes. --apply creates a consistent
|
||||||
|
SQLite backup before the transaction. Run against an isolated database first.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import datetime
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
|
||||||
|
def migrate(db_path: Path, apply: bool = False) -> dict:
|
||||||
|
if not db_path.is_file():
|
||||||
|
raise ValueError("database must already exist")
|
||||||
|
uri = db_path.resolve().as_uri() + ("?mode=rw" if apply else "?mode=ro")
|
||||||
|
with sqlite3.connect(uri, uri=True) as db:
|
||||||
|
changes = []
|
||||||
|
for table in ("sub_bind", "sub_update_record"):
|
||||||
|
columns = {row[1] for row in db.execute(f'PRAGMA table_info("{table}")')}
|
||||||
|
if not columns:
|
||||||
|
raise ValueError(f"required table missing: {table}")
|
||||||
|
if "user_id" not in columns:
|
||||||
|
changes.append(f"add {table}.user_id")
|
||||||
|
duplicates = db.execute(
|
||||||
|
"select count(*) from (select username from User group by username having count(*) > 1)"
|
||||||
|
).fetchone()[0]
|
||||||
|
if duplicates:
|
||||||
|
raise ValueError("ambiguous duplicate usernames; resolve identities before migration")
|
||||||
|
result = {"apply": apply, "schema_changes": changes}
|
||||||
|
if not apply:
|
||||||
|
return result
|
||||||
|
stamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
|
||||||
|
backup = db_path.with_name(db_path.name + ".before-user-links." + stamp)
|
||||||
|
with sqlite3.connect(backup) as snapshot:
|
||||||
|
db.backup(snapshot)
|
||||||
|
backup.chmod(0o600)
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
for change in changes:
|
||||||
|
table = change.split()[1].split(".")[0]
|
||||||
|
db.execute(f'ALTER TABLE "{table}" ADD COLUMN user_id INTEGER REFERENCES User(id)')
|
||||||
|
for table in ("sub_bind", "sub_update_record"):
|
||||||
|
db.execute(f'UPDATE "{table}" SET user_id=(SELECT id FROM User WHERE username="{table}".user) WHERE user_id IS NULL')
|
||||||
|
db.execute("CREATE UNIQUE INDEX IF NOT EXISTS User_username_uindex ON User(username)")
|
||||||
|
db.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_sub_bind_user_id ON sub_bind(user_id) WHERE user_id IS NOT NULL")
|
||||||
|
db.execute("CREATE INDEX IF NOT EXISTS idx_sub_record_user_id ON sub_update_record(user_id, id)")
|
||||||
|
db.commit()
|
||||||
|
except BaseException:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
result["backup"] = str(backup)
|
||||||
|
result["unresolved_bindings"] = db.execute("select count(*) from sub_bind where user_id is null").fetchone()[0]
|
||||||
|
result["unresolved_records"] = db.execute("select count(*) from sub_update_record where user_id is null").fetchone()[0]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("database", type=Path)
|
||||||
|
parser.add_argument("--apply", action="store_true")
|
||||||
|
args = parser.parse_args()
|
||||||
|
print(json.dumps(migrate(args.database, args.apply), ensure_ascii=False))
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
import hashlib
|
||||||
|
|
||||||
|
spec = importlib.util.spec_from_file_location("migration", Path(__file__).with_name("migrate_subscription_events.py"))
|
||||||
|
migration = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(migration)
|
||||||
|
|
||||||
|
|
||||||
|
class SubscriptionEventsMigrationTest(unittest.TestCase):
|
||||||
|
def test_explicit_backup_atomic_and_idempotent(self):
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
path = Path(directory) / "main.db"
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.executescript("""
|
||||||
|
CREATE TABLE User(id INTEGER PRIMARY KEY,username TEXT);
|
||||||
|
CREATE TABLE sub_bind(key TEXT,user TEXT,user_id INTEGER);
|
||||||
|
CREATE TABLE sub_update_record(id INTEGER PRIMARY KEY,user_id INTEGER,time INTEGER);
|
||||||
|
INSERT INTO User VALUES(3,'alice');
|
||||||
|
INSERT INTO sub_bind VALUES('public-test-key','alice',3);
|
||||||
|
""")
|
||||||
|
dry = migration.migrate(path)
|
||||||
|
self.assertEqual(2, len(dry["tables_to_create"]))
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
migration.migrate(path, True)
|
||||||
|
first = migration.migrate(path, True, True)
|
||||||
|
self.assertEqual("ok", first["integrity"])
|
||||||
|
self.assertEqual(0o600, Path(first["backup"]).stat().st_mode & 0o777)
|
||||||
|
with sqlite3.connect(first["backup"]) as db:
|
||||||
|
self.assertEqual(0, db.execute("SELECT count(*) FROM sqlite_master WHERE name='subscription_event_binding'").fetchone()[0])
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
row = db.execute("SELECT key_hash,user_id,user_name FROM subscription_event_binding").fetchone()
|
||||||
|
self.assertEqual((hashlib.sha256(b"public-test-key").hexdigest(), 3, "alice"), row)
|
||||||
|
db.execute("INSERT INTO subscription_event_receipt VALUES('id','hash','v2',0,'primary',1,'fingerprint','recorded')")
|
||||||
|
second = migration.migrate(path, True, True)
|
||||||
|
self.assertEqual([], second["tables_to_create"])
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
self.assertEqual(1, db.execute("SELECT count(*) FROM subscription_event_receipt").fetchone()[0])
|
||||||
|
self.assertEqual(1, db.execute("SELECT count(*) FROM subscription_event_binding").fetchone()[0])
|
||||||
|
|
||||||
|
def test_legacy_labels_without_user_ids_remain_null(self):
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
path = Path(directory) / "main.db"
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.executescript("""
|
||||||
|
CREATE TABLE User(id INTEGER PRIMARY KEY,username TEXT);
|
||||||
|
CREATE TABLE sub_bind(key TEXT,user TEXT,user_id INTEGER);
|
||||||
|
CREATE TABLE sub_update_record(id INTEGER PRIMARY KEY,user_id INTEGER,time INTEGER);
|
||||||
|
INSERT INTO sub_bind VALUES('legacy-key','legacy-label',NULL);
|
||||||
|
""")
|
||||||
|
result = migration.migrate(path, True, True)
|
||||||
|
self.assertEqual(1, result["history_bindings"])
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
row = db.execute("SELECT key_hash,user_id,user_name FROM subscription_event_binding").fetchone()
|
||||||
|
self.assertEqual((hashlib.sha256(b"legacy-key").hexdigest(), None, "legacy-label"), row)
|
||||||
|
self.assertIsNone(db.execute("SELECT user_id FROM sub_bind").fetchone()[0])
|
||||||
|
self.assertEqual(0, db.execute("SELECT count(*) FROM User").fetchone()[0])
|
||||||
|
migration.migrate(path, True, True)
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
self.assertEqual(1, db.execute("SELECT count(*) FROM subscription_event_binding").fetchone()[0])
|
||||||
|
|
||||||
|
def test_orphan_binding_aborts_without_ddl(self):
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
path = Path(directory) / "main.db"
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.executescript("""
|
||||||
|
CREATE TABLE User(id INTEGER PRIMARY KEY,username TEXT);
|
||||||
|
CREATE TABLE sub_bind(key TEXT,user TEXT,user_id INTEGER);
|
||||||
|
CREATE TABLE sub_update_record(id INTEGER PRIMARY KEY,user_id INTEGER,time INTEGER);
|
||||||
|
INSERT INTO sub_bind VALUES('test','deleted',9);
|
||||||
|
""")
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
migration.migrate(path, True, True)
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
self.assertEqual(0, db.execute("SELECT count(*) FROM sqlite_master WHERE name LIKE 'subscription_event_%'").fetchone()[0])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
import sqlite3
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
spec = importlib.util.spec_from_file_location("migration", Path(__file__).with_name("migrate_user_links.py"))
|
||||||
|
migration = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(migration)
|
||||||
|
|
||||||
|
|
||||||
|
class MigrationTest(unittest.TestCase):
|
||||||
|
def test_dry_run_idempotent_migration_and_rename(self):
|
||||||
|
with tempfile.TemporaryDirectory() as temporary:
|
||||||
|
path = Path(temporary) / "test.db"
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.executescript("""
|
||||||
|
CREATE TABLE User(id INTEGER PRIMARY KEY, username TEXT);
|
||||||
|
CREATE TABLE sub_bind(key TEXT PRIMARY KEY, user TEXT, subscription_account_id INTEGER);
|
||||||
|
CREATE TABLE sub_update_record(id INTEGER PRIMARY KEY, user TEXT, location TEXT);
|
||||||
|
INSERT INTO User VALUES(1,'old-name');
|
||||||
|
INSERT INTO sub_bind VALUES('example-key','old-name',2);
|
||||||
|
INSERT INTO sub_update_record VALUES(10,'old-name','unknown');
|
||||||
|
INSERT INTO sub_update_record VALUES(11,'old-name','unknown');
|
||||||
|
""")
|
||||||
|
before = path.read_bytes()
|
||||||
|
self.assertEqual(2, len(migration.migrate(path)["schema_changes"]))
|
||||||
|
self.assertEqual(before, path.read_bytes())
|
||||||
|
self.assertEqual(0, migration.migrate(path, True)["unresolved_bindings"])
|
||||||
|
self.assertEqual([], migration.migrate(path, True)["schema_changes"])
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("update User set username='new-name' where id=1")
|
||||||
|
self.assertEqual((1,), db.execute("select user_id from sub_bind").fetchone())
|
||||||
|
# A slow first lookup must not overwrite the second request's row.
|
||||||
|
db.execute("update sub_update_record set location='first-location' where id=10")
|
||||||
|
self.assertEqual(("unknown",), db.execute("select location from sub_update_record where id=11").fetchone())
|
||||||
|
|
||||||
|
def test_ambiguous_names_fail_without_modifying_database(self):
|
||||||
|
with tempfile.TemporaryDirectory() as temporary:
|
||||||
|
path = Path(temporary) / "test.db"
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.executescript("""
|
||||||
|
CREATE TABLE User(id INTEGER PRIMARY KEY, username TEXT);
|
||||||
|
CREATE TABLE sub_bind(user TEXT);
|
||||||
|
CREATE TABLE sub_update_record(user TEXT);
|
||||||
|
INSERT INTO User VALUES(1,'same'),(2,'same');
|
||||||
|
""")
|
||||||
|
before = path.read_bytes()
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
migration.migrate(path, True)
|
||||||
|
self.assertEqual(before, path.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.*;
|
||||||
|
import com.lion.lionwebsite.Message.*;
|
||||||
|
import com.lion.lionwebsite.Service.SubscriptionRefreshPlanner;
|
||||||
|
import com.lion.lionwebsite.Util.GalleryUtil;
|
||||||
|
import com.pengrad.telegrambot.TelegramBot;
|
||||||
|
import com.pengrad.telegrambot.model.*;
|
||||||
|
import com.pengrad.telegrambot.model.User;
|
||||||
|
import com.pengrad.telegrambot.response.SendResponse;
|
||||||
|
import com.zaxxer.hikari.HikariConfig;
|
||||||
|
import jakarta.annotation.PostConstruct;
|
||||||
|
import org.springframework.aot.hint.annotation.RegisterReflectionForBinding;
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
|
||||||
|
import java.time.Clock;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.Random;
|
||||||
|
|
||||||
|
@Configuration
|
||||||
|
@RegisterReflectionForBinding(classes = {CustomConfiguration.class, GidToKey.class, ImageKeyCache.class,
|
||||||
|
GalleryForQuery.class, Gallery.class, GalleryTask.class, HikariConfig.class,
|
||||||
|
PageNameCache.class, User.class,
|
||||||
|
SendResponse.class, Message.class, com.pengrad.telegrambot.model.User.class,
|
||||||
|
Chat.class, MessageEntity.class,
|
||||||
|
AbstractMethodError.class, DeleteGalleryMessage.class, DownloadPostMessage.class, DownloadStatusMessage.class,
|
||||||
|
IdentityMessage.class, MaintainMessage.class, ResponseMessage.class, AvailableCheckMessage.class,
|
||||||
|
// 协议 v2 的消息与任务操作实体:原生镜像下 Jackson/MyBatis 都要反射构造它们。
|
||||||
|
ProtocolHelloMessage.class, TaskCommandMessage.class, TaskReportMessage.class, TaskReportAckMessage.class,
|
||||||
|
TaskOperation.class,
|
||||||
|
SubscriptionSnapshotMessage.class, SubscriptionSnapshotPayload.class, SubscriptionAccountSnapshot.class,
|
||||||
|
SubscriptionBindingSnapshot.class, LinkPreviewOptions.class})
|
||||||
|
public class CustomBean {
|
||||||
|
|
||||||
|
@Value("${bot.token:5222939329:AAHa6l9ZuVVdNSDLPI_H-c8O_VgeOEw5plA}")
|
||||||
|
private String botToken;
|
||||||
|
|
||||||
|
@Value("${gallery.cookie:ipb_session_id=af2b2b1a795b39550711134d7bdcbf7f; ipb_member_id=5774855; ipb_pass_hash=4b061c3abe25289568b5a8e0123fb3b9; sk=oye107wk02gtomb56x65dmv4qzbn; nw=1}")
|
||||||
|
private String ehentaiCookie;
|
||||||
|
|
||||||
|
@PostConstruct
|
||||||
|
void initGalleryCookie() {
|
||||||
|
GalleryUtil.setEhentaiCookie(ehentaiCookie);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
public TelegramBot getTelegramBot(){
|
||||||
|
return new TelegramBot(botToken);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 订阅分散刷新的排程器。
|
||||||
|
*
|
||||||
|
* <p>放在这里是为了让窗口、最小间隔、tick 周期与重试间隔集中由配置注入,
|
||||||
|
* 同时保持 {@link SubscriptionRefreshPlanner} 本身是可直接构造的纯对象
|
||||||
|
* (便于用固定时钟与固定随机种子做确定性单元测试)。
|
||||||
|
*/
|
||||||
|
@Bean
|
||||||
|
public SubscriptionRefreshPlanner subscriptionRefreshPlanner(
|
||||||
|
@Value("${subscription.refresh.window-hours:24}") long windowHours,
|
||||||
|
@Value("${subscription.refresh.min-gap-minutes:60}") long minGapMinutes,
|
||||||
|
@Value("${subscription.refresh.tick-interval-ms:300000}") long tickIntervalMs,
|
||||||
|
@Value("${subscription.refresh.retry-delay-minutes:60}") long retryDelayMinutes) {
|
||||||
|
return new SubscriptionRefreshPlanner(Clock.systemDefaultZone(), new Random(),
|
||||||
|
Duration.ofHours(windowHours), Duration.ofMinutes(minGapMinutes),
|
||||||
|
Duration.ofMillis(tickIntervalMs), Duration.ofMinutes(retryDelayMinutes));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
|
||||||
|
import com.lion.lionwebsite.Domain.ImageKeyCache;
|
||||||
|
import com.lion.lionwebsite.Service.GalleryIndexService;
|
||||||
|
import com.lion.lionwebsite.Util.GalleryUtil;
|
||||||
|
import org.springframework.beans.factory.annotation.Qualifier;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
|
||||||
|
import org.springframework.transaction.PlatformTransactionManager;
|
||||||
|
import org.springframework.transaction.support.TransactionTemplate;
|
||||||
|
|
||||||
|
import javax.sql.DataSource;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 图片索引服务的事务装配。
|
||||||
|
*
|
||||||
|
* <p>单独成文件是为了不触碰既有的 SqlConfiguration:那里的两个 SqlSessionFactory
|
||||||
|
* 保持原样,这里只额外声明缓存库的事务管理器与模板。
|
||||||
|
*
|
||||||
|
* <p>事务管理器必须绑定 datasource-cache —— 索引只写缓存库,
|
||||||
|
* 若误用主库事务管理器,整组替换就失去原子性(删除与写入落在不同连接上)。
|
||||||
|
*/
|
||||||
|
@Configuration
|
||||||
|
public class GalleryIndexConfiguration {
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
public PlatformTransactionManager cacheTransactionManager(
|
||||||
|
@Qualifier("datasource-cache") DataSource cacheDataSource) {
|
||||||
|
return new DataSourceTransactionManager(cacheDataSource);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 只用于图片索引的短事务发布,超时留给上层请求超时控制,这里显式设为 30 秒。 */
|
||||||
|
@Bean
|
||||||
|
public TransactionTemplate cacheTransactionTemplate(
|
||||||
|
@Qualifier("cacheTransactionManager") PlatformTransactionManager cacheTransactionManager) {
|
||||||
|
TransactionTemplate template = new TransactionTemplate(cacheTransactionManager);
|
||||||
|
template.setTimeout(30);
|
||||||
|
return template;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 只读快照事务,用于一次性读取头部与页码。
|
||||||
|
*
|
||||||
|
* <p>读事务不写库,因此只读连接可以与写入并发;它的作用是让「读头部 + 读页码」
|
||||||
|
* 落在同一个快照上,避免恰好读到整组替换的中间状态。
|
||||||
|
*/
|
||||||
|
@Bean
|
||||||
|
public TransactionTemplate cacheReadTransactionTemplate(
|
||||||
|
@Qualifier("cacheTransactionManager") PlatformTransactionManager cacheTransactionManager) {
|
||||||
|
TransactionTemplate template = new TransactionTemplate(cacheTransactionManager);
|
||||||
|
template.setTimeout(10);
|
||||||
|
return template;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 这里显式不设置 {@code readOnly}:sqlite-jdbc 不允许在已建立的连接上改只读标志
|
||||||
|
* (实测抛 SQLException: Cannot change read-only flag after establishing a connection),
|
||||||
|
* 而事务模板拿到的是连接池里的既有连接,设置只读会让每次读取都直接失败。
|
||||||
|
* 快照一致性由「同一个事务」保证,不需要额外的只读标志。
|
||||||
|
*/
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
public GalleryIndexService galleryIndexService(
|
||||||
|
ImageCacheMapper imageCacheMapper,
|
||||||
|
@Qualifier("cacheTransactionTemplate") TransactionTemplate cacheTransactionTemplate,
|
||||||
|
@Qualifier("cacheReadTransactionTemplate") TransactionTemplate cacheReadTransactionTemplate) {
|
||||||
|
return new GalleryIndexService(imageCacheMapper, cacheTransactionTemplate,
|
||||||
|
cacheReadTransactionTemplate,
|
||||||
|
GalleryIndexConfiguration::fetchPageKeys);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 生产环境的上游实现:沿用 GalleryUtil.parseImageKeys(HTTP + JS 解析)。
|
||||||
|
*
|
||||||
|
* <p>放在这里而不是服务内部,是为了让服务在测试中可以脱离静态工具类。
|
||||||
|
*/
|
||||||
|
static List<ImageKeyCache> fetchPageKeys(String url) throws Exception {
|
||||||
|
return GalleryUtil.parseImageKeys(url);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,11 +1,9 @@
|
|||||||
package com.lion.lionwebsite.Configuration;
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.MaskDomain;
|
|
||||||
import com.lion.lionwebsite.Interceptor.HumanInterceptor;
|
import com.lion.lionwebsite.Interceptor.HumanInterceptor;
|
||||||
import com.lion.lionwebsite.Interceptor.PersonalInterceptor;
|
import com.lion.lionwebsite.Interceptor.PersonalInterceptor;
|
||||||
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
|
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
import lombok.RequiredArgsConstructor;
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import org.springframework.context.annotation.Bean;
|
import org.springframework.context.annotation.Bean;
|
||||||
import org.springframework.context.annotation.Configuration;
|
import org.springframework.context.annotation.Configuration;
|
||||||
import org.springframework.web.servlet.HandlerInterceptor;
|
import org.springframework.web.servlet.HandlerInterceptor;
|
||||||
@@ -13,14 +11,19 @@ import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
|
|||||||
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
|
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
|
||||||
|
|
||||||
@Configuration
|
@Configuration
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class InterceptorConfiguration implements WebMvcConfigurer {
|
public class InterceptorConfiguration implements WebMvcConfigurer {
|
||||||
@Resource
|
final TaskHandlerInterceptor taskHandlerInterceptor;
|
||||||
TaskHandlerInterceptor taskHandlerInterceptor;
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void addInterceptors(InterceptorRegistry registry) {
|
public void addInterceptors(InterceptorRegistry registry) {
|
||||||
registry.addInterceptor(getPersonalInterceptor()).addPathPatterns("/personal/**", "/remote/**");
|
// 登录端点必须排除在闸门之外,否则校验票据的入口会被自己拦住。
|
||||||
registry.addInterceptor(taskHandlerInterceptor).addPathPatterns("/GalleryManage", "/validate");
|
// /personal/ 只是跳到静态入口 /index,也一并排除:未登录时让前端自己去提示,
|
||||||
|
// 否则用户直接访问根域名只会看到 401。
|
||||||
|
registry.addInterceptor(getPersonalInterceptor())
|
||||||
|
.addPathPatterns("/personal/**", "/remote/**")
|
||||||
|
.excludePathPatterns("/personal/", "/personal/login", "/personal/login/logout", "/personal/denied");
|
||||||
|
registry.addInterceptor(taskHandlerInterceptor).addPathPatterns("/GalleryManage", "/GalleryManage/**", "/validate");
|
||||||
registry.addInterceptor(getHumanInterceptor()).addPathPatterns("/", "/mobile");
|
registry.addInterceptor(getHumanInterceptor()).addPathPatterns("/", "/mobile");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -33,14 +36,4 @@ public class InterceptorConfiguration implements WebMvcConfigurer {
|
|||||||
public HandlerInterceptor getHumanInterceptor(){
|
public HandlerInterceptor getHumanInterceptor(){
|
||||||
return new HumanInterceptor();
|
return new HumanInterceptor();
|
||||||
}
|
}
|
||||||
|
|
||||||
@Bean
|
|
||||||
public CustomUtil getParameterUtil(){
|
|
||||||
CustomUtil customUtil = new CustomUtil();
|
|
||||||
MaskDomain[] maskDomains = new MaskDomain[2];
|
|
||||||
maskDomains[0] = new MaskDomain("exhentai.org", "element-plus.org");
|
|
||||||
maskDomains[1] = new MaskDomain("e-hentai.org", "element.org");
|
|
||||||
customUtil.setMaskDomains(maskDomains);
|
|
||||||
return customUtil;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,268 @@
|
|||||||
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
|
import org.apache.commons.logging.LogFactory;
|
||||||
|
import org.apache.ibatis.annotations.DeleteProvider;
|
||||||
|
import org.apache.ibatis.annotations.InsertProvider;
|
||||||
|
import org.apache.ibatis.annotations.SelectProvider;
|
||||||
|
import org.apache.ibatis.annotations.UpdateProvider;
|
||||||
|
import org.apache.ibatis.cache.decorators.FifoCache;
|
||||||
|
import org.apache.ibatis.cache.decorators.LruCache;
|
||||||
|
import org.apache.ibatis.cache.decorators.SoftCache;
|
||||||
|
import org.apache.ibatis.cache.decorators.WeakCache;
|
||||||
|
import org.apache.ibatis.cache.impl.PerpetualCache;
|
||||||
|
import org.apache.ibatis.javassist.util.proxy.ProxyFactory;
|
||||||
|
import org.apache.ibatis.javassist.util.proxy.RuntimeSupport;
|
||||||
|
import org.apache.ibatis.logging.Log;
|
||||||
|
import org.apache.ibatis.logging.commons.JakartaCommonsLoggingImpl;
|
||||||
|
import org.apache.ibatis.logging.jdk14.Jdk14LoggingImpl;
|
||||||
|
import org.apache.ibatis.logging.log4j2.Log4j2Impl;
|
||||||
|
import org.apache.ibatis.logging.nologging.NoLoggingImpl;
|
||||||
|
import org.apache.ibatis.logging.slf4j.Slf4jImpl;
|
||||||
|
import org.apache.ibatis.logging.stdout.StdOutImpl;
|
||||||
|
import org.apache.ibatis.reflection.TypeParameterResolver;
|
||||||
|
import org.apache.ibatis.scripting.defaults.RawLanguageDriver;
|
||||||
|
import org.apache.ibatis.scripting.xmltags.XMLLanguageDriver;
|
||||||
|
import org.apache.ibatis.session.SqlSessionFactory;
|
||||||
|
import org.mybatis.spring.SqlSessionFactoryBean;
|
||||||
|
import org.mybatis.spring.mapper.MapperFactoryBean;
|
||||||
|
import org.mybatis.spring.mapper.MapperScannerConfigurer;
|
||||||
|
import org.springframework.aot.hint.MemberCategory;
|
||||||
|
import org.springframework.aot.hint.RuntimeHints;
|
||||||
|
import org.springframework.aot.hint.RuntimeHintsRegistrar;
|
||||||
|
import org.springframework.beans.PropertyValue;
|
||||||
|
import org.springframework.beans.factory.BeanFactory;
|
||||||
|
import org.springframework.beans.factory.BeanFactoryAware;
|
||||||
|
import org.springframework.beans.factory.aot.BeanFactoryInitializationAotContribution;
|
||||||
|
import org.springframework.beans.factory.aot.BeanFactoryInitializationAotProcessor;
|
||||||
|
import org.springframework.beans.factory.aot.BeanRegistrationExcludeFilter;
|
||||||
|
import org.springframework.beans.factory.config.BeanDefinition;
|
||||||
|
import org.springframework.beans.factory.config.ConfigurableBeanFactory;
|
||||||
|
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
|
||||||
|
import org.springframework.beans.factory.config.ConstructorArgumentValues;
|
||||||
|
import org.springframework.beans.factory.support.MergedBeanDefinitionPostProcessor;
|
||||||
|
import org.springframework.beans.factory.support.RegisteredBean;
|
||||||
|
import org.springframework.beans.factory.support.RootBeanDefinition;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
import org.springframework.context.annotation.ImportRuntimeHints;
|
||||||
|
import org.springframework.core.ResolvableType;
|
||||||
|
import org.springframework.util.ClassUtils;
|
||||||
|
import org.springframework.util.ReflectionUtils;
|
||||||
|
|
||||||
|
import java.lang.annotation.Annotation;
|
||||||
|
import java.lang.reflect.Method;
|
||||||
|
import java.lang.reflect.ParameterizedType;
|
||||||
|
import java.lang.reflect.Type;
|
||||||
|
import java.util.*;
|
||||||
|
import java.util.function.Function;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
import java.util.stream.Stream;
|
||||||
|
|
||||||
|
@Configuration(proxyBeanMethods = false)
|
||||||
|
@ImportRuntimeHints(MyBatisNativeConfiguration.MyBaitsRuntimeHintsRegistrar.class)
|
||||||
|
public class MyBatisNativeConfiguration {
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
MyBatisBeanFactoryInitializationAotProcessor myBatisBeanFactoryInitializationAotProcessor() {
|
||||||
|
return new MyBatisBeanFactoryInitializationAotProcessor();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean
|
||||||
|
static MyBatisMapperFactoryBeanPostProcessor myBatisMapperFactoryBeanPostProcessor() {
|
||||||
|
return new MyBatisMapperFactoryBeanPostProcessor();
|
||||||
|
}
|
||||||
|
|
||||||
|
static class MyBaitsRuntimeHintsRegistrar implements RuntimeHintsRegistrar {
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void registerHints(RuntimeHints hints, ClassLoader classLoader) {
|
||||||
|
Stream.of(RawLanguageDriver.class,
|
||||||
|
XMLLanguageDriver.class,
|
||||||
|
RuntimeSupport.class,
|
||||||
|
ProxyFactory.class,
|
||||||
|
Slf4jImpl.class,
|
||||||
|
Log.class,
|
||||||
|
JakartaCommonsLoggingImpl.class,
|
||||||
|
Log4j2Impl.class,
|
||||||
|
Jdk14LoggingImpl.class,
|
||||||
|
StdOutImpl.class,
|
||||||
|
NoLoggingImpl.class,
|
||||||
|
SqlSessionFactory.class,
|
||||||
|
PerpetualCache.class,
|
||||||
|
FifoCache.class,
|
||||||
|
LruCache.class,
|
||||||
|
SoftCache.class,
|
||||||
|
WeakCache.class,
|
||||||
|
SqlSessionFactoryBean.class,
|
||||||
|
ArrayList.class,
|
||||||
|
HashMap.class,
|
||||||
|
TreeSet.class,
|
||||||
|
HashSet.class
|
||||||
|
).forEach(x -> hints.reflection().registerType(x, MemberCategory.values()));
|
||||||
|
Stream.of(
|
||||||
|
"org/apache/ibatis/builder/xml/*.dtd",
|
||||||
|
"org/apache/ibatis/builder/xml/*.xsd"
|
||||||
|
).forEach(hints.resources()::registerPattern);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static class MyBatisBeanFactoryInitializationAotProcessor
|
||||||
|
implements BeanFactoryInitializationAotProcessor, BeanRegistrationExcludeFilter {
|
||||||
|
|
||||||
|
private final Set<Class<?>> excludeClasses = new HashSet<>();
|
||||||
|
|
||||||
|
MyBatisBeanFactoryInitializationAotProcessor() {
|
||||||
|
excludeClasses.add(MapperScannerConfigurer.class);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override public boolean isExcludedFromAotProcessing(RegisteredBean registeredBean) {
|
||||||
|
return excludeClasses.contains(registeredBean.getBeanClass());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public BeanFactoryInitializationAotContribution processAheadOfTime(ConfigurableListableBeanFactory beanFactory) {
|
||||||
|
String[] beanNames = beanFactory.getBeanNamesForType(MapperFactoryBean.class);
|
||||||
|
if (beanNames.length == 0) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return (context, code) -> {
|
||||||
|
RuntimeHints hints = context.getRuntimeHints();
|
||||||
|
for (String beanName : beanNames) {
|
||||||
|
BeanDefinition beanDefinition = beanFactory.getBeanDefinition(beanName.substring(1));
|
||||||
|
PropertyValue mapperInterface = beanDefinition.getPropertyValues().getPropertyValue("mapperInterface");
|
||||||
|
if (mapperInterface != null && mapperInterface.getValue() != null) {
|
||||||
|
Class<?> mapperInterfaceType = (Class<?>) mapperInterface.getValue();
|
||||||
|
if (mapperInterfaceType != null) {
|
||||||
|
registerReflectionTypeIfNecessary(mapperInterfaceType, hints);
|
||||||
|
hints.proxies().registerJdkProxy(mapperInterfaceType);
|
||||||
|
hints.resources()
|
||||||
|
.registerPattern(mapperInterfaceType.getName().replace('.', '/').concat(".xml"));
|
||||||
|
registerMapperRelationships(mapperInterfaceType, hints);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private void registerMapperRelationships(Class<?> mapperInterfaceType, RuntimeHints hints) {
|
||||||
|
Method[] methods = ReflectionUtils.getAllDeclaredMethods(mapperInterfaceType);
|
||||||
|
for (Method method : methods) {
|
||||||
|
if (method.getDeclaringClass() != Object.class) {
|
||||||
|
ReflectionUtils.makeAccessible(method);
|
||||||
|
registerSqlProviderTypes(method, hints, SelectProvider.class, SelectProvider::value, SelectProvider::type);
|
||||||
|
registerSqlProviderTypes(method, hints, InsertProvider.class, InsertProvider::value, InsertProvider::type);
|
||||||
|
registerSqlProviderTypes(method, hints, UpdateProvider.class, UpdateProvider::value, UpdateProvider::type);
|
||||||
|
registerSqlProviderTypes(method, hints, DeleteProvider.class, DeleteProvider::value, DeleteProvider::type);
|
||||||
|
Class<?> returnType = MyBatisMapperTypeUtils.resolveReturnClass(mapperInterfaceType, method);
|
||||||
|
registerReflectionTypeIfNecessary(returnType, hints);
|
||||||
|
MyBatisMapperTypeUtils.resolveParameterClasses(mapperInterfaceType, method)
|
||||||
|
.forEach(x -> registerReflectionTypeIfNecessary(x, hints));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@SafeVarargs
|
||||||
|
private <T extends Annotation> void registerSqlProviderTypes(
|
||||||
|
Method method, RuntimeHints hints, Class<T> annotationType, Function<T, Class<?>>... providerTypeResolvers) {
|
||||||
|
for (T annotation : method.getAnnotationsByType(annotationType)) {
|
||||||
|
for (Function<T, Class<?>> providerTypeResolver : providerTypeResolvers) {
|
||||||
|
registerReflectionTypeIfNecessary(providerTypeResolver.apply(annotation), hints);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void registerReflectionTypeIfNecessary(Class<?> type, RuntimeHints hints) {
|
||||||
|
if (!type.isPrimitive() && !type.getName().startsWith("java")) {
|
||||||
|
hints.reflection().registerType(type, MemberCategory.values());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
static class MyBatisMapperTypeUtils {
|
||||||
|
private MyBatisMapperTypeUtils() {
|
||||||
|
// NOP
|
||||||
|
}
|
||||||
|
|
||||||
|
static Class<?> resolveReturnClass(Class<?> mapperInterface, Method method) {
|
||||||
|
Type resolvedReturnType = TypeParameterResolver.resolveReturnType(method, mapperInterface);
|
||||||
|
return typeToClass(resolvedReturnType, method.getReturnType());
|
||||||
|
}
|
||||||
|
|
||||||
|
static Set<Class<?>> resolveParameterClasses(Class<?> mapperInterface, Method method) {
|
||||||
|
return Stream.of(TypeParameterResolver.resolveParamTypes(method, mapperInterface))
|
||||||
|
.map(x -> typeToClass(x, x instanceof Class ? (Class<?>) x : Object.class)).collect(Collectors.toSet());
|
||||||
|
}
|
||||||
|
|
||||||
|
private static Class<?> typeToClass(Type src, Class<?> fallback) {
|
||||||
|
Class<?> result = null;
|
||||||
|
if (src instanceof Class<?>) {
|
||||||
|
if (((Class<?>) src).isArray()) {
|
||||||
|
result = ((Class<?>) src).getComponentType();
|
||||||
|
} else {
|
||||||
|
result = (Class<?>) src;
|
||||||
|
}
|
||||||
|
} else if (src instanceof ParameterizedType parameterizedType) {
|
||||||
|
int index = (parameterizedType.getRawType() instanceof Class
|
||||||
|
&& Map.class.isAssignableFrom((Class<?>) parameterizedType.getRawType())
|
||||||
|
&& parameterizedType.getActualTypeArguments().length > 1) ? 1 : 0;
|
||||||
|
Type actualType = parameterizedType.getActualTypeArguments()[index];
|
||||||
|
result = typeToClass(actualType, fallback);
|
||||||
|
}
|
||||||
|
if (result == null) {
|
||||||
|
result = fallback;
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
static class MyBatisMapperFactoryBeanPostProcessor implements MergedBeanDefinitionPostProcessor, BeanFactoryAware {
|
||||||
|
|
||||||
|
private static final org.apache.commons.logging.Log LOG = LogFactory.getLog(
|
||||||
|
MyBatisMapperFactoryBeanPostProcessor.class);
|
||||||
|
|
||||||
|
private static final String MAPPER_FACTORY_BEAN = "org.mybatis.spring.mapper.MapperFactoryBean";
|
||||||
|
|
||||||
|
private ConfigurableBeanFactory beanFactory;
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void setBeanFactory(BeanFactory beanFactory) {
|
||||||
|
this.beanFactory = (ConfigurableBeanFactory) beanFactory;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void postProcessMergedBeanDefinition(RootBeanDefinition beanDefinition, Class<?> beanType, String beanName) {
|
||||||
|
if (ClassUtils.isPresent(MAPPER_FACTORY_BEAN, this.beanFactory.getBeanClassLoader())) {
|
||||||
|
resolveMapperFactoryBeanTypeIfNecessary(beanDefinition);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void resolveMapperFactoryBeanTypeIfNecessary(RootBeanDefinition beanDefinition) {
|
||||||
|
if (!beanDefinition.hasBeanClass() || !MapperFactoryBean.class.isAssignableFrom(beanDefinition.getBeanClass())) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (beanDefinition.getResolvableType().hasUnresolvableGenerics()) {
|
||||||
|
Class<?> mapperInterface = getMapperInterface(beanDefinition);
|
||||||
|
if (mapperInterface != null) {
|
||||||
|
// Exposes a generic type information to context for prevent early initializing
|
||||||
|
ConstructorArgumentValues constructorArgumentValues = new ConstructorArgumentValues();
|
||||||
|
constructorArgumentValues.addGenericArgumentValue(mapperInterface);
|
||||||
|
beanDefinition.setConstructorArgumentValues(constructorArgumentValues);
|
||||||
|
beanDefinition.setTargetType(ResolvableType.forClassWithGenerics(beanDefinition.getBeanClass(), mapperInterface));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private Class<?> getMapperInterface(RootBeanDefinition beanDefinition) {
|
||||||
|
try {
|
||||||
|
return (Class<?>) beanDefinition.getPropertyValues().get("mapperInterface");
|
||||||
|
}
|
||||||
|
catch (Exception e) {
|
||||||
|
LOG.debug("Fail getting mapper interface type.", e);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
|
import org.apache.ibatis.session.SqlSessionFactory;
|
||||||
|
import org.mybatis.spring.SqlSessionFactoryBean;
|
||||||
|
import org.mybatis.spring.SqlSessionTemplate;
|
||||||
|
import org.springframework.beans.factory.annotation.Qualifier;
|
||||||
|
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||||
|
import org.springframework.boot.jdbc.DataSourceBuilder;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
import org.springframework.context.annotation.Primary;
|
||||||
|
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
|
||||||
|
import org.springframework.transaction.support.TransactionTemplate;
|
||||||
|
|
||||||
|
import com.zaxxer.hikari.HikariDataSource;
|
||||||
|
|
||||||
|
import javax.sql.DataSource;
|
||||||
|
|
||||||
|
@Configuration
|
||||||
|
public class SqlConfiguration {
|
||||||
|
@Bean(name = "datasource-main")
|
||||||
|
@ConfigurationProperties(prefix = "spring.datasource-main")
|
||||||
|
public DataSource dataSourceMain() {
|
||||||
|
DataSource dataSource = DataSourceBuilder.create().build();
|
||||||
|
applyBusyTimeout(dataSource);
|
||||||
|
return dataSource;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean(name = "datasource-cache")
|
||||||
|
@ConfigurationProperties(prefix = "spring.datasource-cache")
|
||||||
|
public DataSource dataSourceCache() {
|
||||||
|
return DataSourceBuilder.create().build();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 给主库连接打开 busy 等待。
|
||||||
|
*
|
||||||
|
* <p>主库是单文件 SQLite,任务生命周期会并发开启短事务(创建、状态回执、删除)。
|
||||||
|
* 默认没有 busy 超时,第二个写事务会立刻拿到 SQLITE_BUSY 并被当作 500;
|
||||||
|
* 打开等待后,写冲突会按序排队重试而不是失败。
|
||||||
|
*/
|
||||||
|
private void applyBusyTimeout(DataSource dataSource) {
|
||||||
|
if (dataSource instanceof HikariDataSource hikari)
|
||||||
|
hikari.setConnectionInitSql("PRAGMA busy_timeout=5000");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean(name = "sqlSessionFactory-main")
|
||||||
|
public SqlSessionFactory sqlSessionFactoryMain(@Qualifier("datasource-main") DataSource dataSource) throws Exception {
|
||||||
|
SqlSessionFactoryBean sessionFactoryBean = new SqlSessionFactoryBean();
|
||||||
|
sessionFactoryBean.setDataSource(dataSource);
|
||||||
|
return sessionFactoryBean.getObject();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean(name = "sqlSessionFactory-cache")
|
||||||
|
public SqlSessionFactory sqlSessionFactoryCache(@Qualifier("datasource-cache") DataSource dataSource) throws Exception {
|
||||||
|
SqlSessionFactoryBean sessionFactoryBean = new SqlSessionFactoryBean();
|
||||||
|
sessionFactoryBean.setDataSource(dataSource);
|
||||||
|
return sessionFactoryBean.getObject();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean(name = "sqlSessionTemplate-main")
|
||||||
|
public SqlSessionTemplate sqlSessionTemplateMain(@Qualifier("sqlSessionFactory-main") SqlSessionFactory sqlSessionFactory) {
|
||||||
|
return new SqlSessionTemplate(sqlSessionFactory);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Bean(name = "sqlSessionTemplate-cache")
|
||||||
|
public SqlSessionTemplate sqlSessionTemplateCache(@Qualifier("sqlSessionFactory-cache") SqlSessionFactory sqlSessionFactory) {
|
||||||
|
return new SqlSessionTemplate(sqlSessionFactory);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 主库事务管理器。
|
||||||
|
*
|
||||||
|
* <p>必须显式绑定到 datasource-main:应用里有两个 DataSource,Spring Boot 的自动配置
|
||||||
|
* 会退避,若没有这个 bean 就没有任何事务管理器,@Transactional 会静默失效。
|
||||||
|
* @Primary 同时让「不指定名字就注入 TransactionManager」的旧代码继续可用。
|
||||||
|
*/
|
||||||
|
@Bean(name = "mainTransactionManager")
|
||||||
|
@Primary
|
||||||
|
public DataSourceTransactionManager mainTransactionManager(@Qualifier("datasource-main") DataSource dataSource) {
|
||||||
|
return new DataSourceTransactionManager(dataSource);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 主库编程式事务模板。
|
||||||
|
*
|
||||||
|
* <p>任务生命周期需要把「短事务」放在网络调用之外,并且要求「回调里捕获异常后不再抛出」
|
||||||
|
* 不能错误提交,因此调用方必须用 {@code execute} 的返回状态或再次抛出决定回滚,
|
||||||
|
* 而不是在回调里吞掉异常(见 GalleryTaskLifecycleService)。
|
||||||
|
*/
|
||||||
|
@Bean(name = "mainTransactionTemplate")
|
||||||
|
@Primary
|
||||||
|
public TransactionTemplate mainTransactionTemplate(@Qualifier("mainTransactionManager") DataSourceTransactionManager transactionManager) {
|
||||||
|
return new TransactionTemplate(transactionManager);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.TaskOperation;
|
||||||
|
import com.lion.lionwebsite.Service.GalleryTaskLifecycleService;
|
||||||
|
import com.lion.lionwebsite.Service.RemoteService;
|
||||||
|
import com.lion.lionwebsite.Service.TaskReportHandler;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.boot.context.event.ApplicationReadyEvent;
|
||||||
|
import org.springframework.context.event.EventListener;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 装配协议 v2 的双向依赖,并做启动恢复。
|
||||||
|
*
|
||||||
|
* <p>RemoteService 消费上报时需要生命周期服务,而生命周期服务下发命令时需要 RemoteService。
|
||||||
|
* 直接互相注入构造参数会形成环,所以这里在两端都构造完成之后用 setter 接上;
|
||||||
|
* 同时也把「启动时把中断的 SUBMITTING 判为 UNKNOWN、释放 RESERVED 占位」这一步接到应用就绪事件上。
|
||||||
|
*
|
||||||
|
* <p>顺序是不可交换的:
|
||||||
|
* <ol>
|
||||||
|
* <li>先绑定上报处理器、下发判定解析器与 operationId 解析器;</li>
|
||||||
|
* <li>再执行启动恢复(SUBMITTING → UNKNOWN、释放 RESERVED);</li>
|
||||||
|
* <li>最后才允许建立连接与自动重投。连接由本类显式触发,不再由 RemoteService 的
|
||||||
|
* PostConstruct 在装配前发起,否则会把状态不明的任务盲目重投给节点。</li>
|
||||||
|
* </ol>
|
||||||
|
* 恢复失败时仍然允许连接(保持节点在线/心跳),但明确禁止自动派发。
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
@Slf4j
|
||||||
|
@RequiredArgsConstructor
|
||||||
|
public class TaskLifecycleWiring {
|
||||||
|
|
||||||
|
final RemoteService remoteService;
|
||||||
|
|
||||||
|
final GalleryTaskLifecycleService lifecycleService;
|
||||||
|
|
||||||
|
@EventListener(ApplicationReadyEvent.class)
|
||||||
|
public void wire() {
|
||||||
|
TaskReportHandler handler = lifecycleService;
|
||||||
|
remoteService.setTaskReportHandler(handler);
|
||||||
|
remoteService.setDownloadOperationIdResolver(lifecycleService::currentOperationId);
|
||||||
|
remoteService.setDispatchResolver(lifecycleService::resolveDispatch);
|
||||||
|
boolean recoverySafe = false;
|
||||||
|
try {
|
||||||
|
int recovered = lifecycleService.recoverInterruptedSubmissions();
|
||||||
|
recoverySafe = true;
|
||||||
|
if (recovered > 0)
|
||||||
|
log.warn("启动恢复:{} 个中断的创建/预占任务已处理(提交中的标为提交未知)", recovered);
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
// 恢复失败说明库中任务真实状态未知:可以保持连接,但绝不能自动重投/派发。
|
||||||
|
log.error("启动恢复失败,本次启动将拒绝自动重投未完成任务", e);
|
||||||
|
}
|
||||||
|
// 绑定与恢复完成之后才连接:连接成功后的自动重投受 lifecycleReady/recoverySafe 双重约束。
|
||||||
|
remoteService.startAfterWiring(recoverySafe);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
package com.lion.lionwebsite.Configuration;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Service.WebSocketService;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import org.springframework.context.annotation.Configuration;
|
||||||
|
import org.springframework.web.socket.config.annotation.EnableWebSocket;
|
||||||
|
import org.springframework.web.socket.config.annotation.WebSocketConfigurer;
|
||||||
|
import org.springframework.web.socket.config.annotation.WebSocketHandlerRegistry;
|
||||||
|
|
||||||
|
@Configuration
|
||||||
|
@EnableWebSocket
|
||||||
|
@RequiredArgsConstructor
|
||||||
|
public class WebsocketConfiguration implements WebSocketConfigurer {
|
||||||
|
|
||||||
|
final WebSocketService webSocketService;
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
|
||||||
|
registry.addHandler(webSocketService, "/ws/").setAllowedOriginPatterns("*");
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -2,49 +2,46 @@ package com.lion.lionwebsite.Controller;
|
|||||||
|
|
||||||
import com.lion.lionwebsite.Service.CollectService;
|
import com.lion.lionwebsite.Service.CollectService;
|
||||||
import com.lion.lionwebsite.Service.GalleryManageService;
|
import com.lion.lionwebsite.Service.GalleryManageService;
|
||||||
import com.lion.lionwebsite.Service.UserServiceImpl;
|
import com.lion.lionwebsite.Service.RemoteService;
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
import com.lion.lionwebsite.Service.UserService;
|
||||||
import com.lion.lionwebsite.Util.Response;
|
import com.lion.lionwebsite.Util.Response;
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.web.bind.annotation.*;
|
import org.springframework.web.bind.annotation.*;
|
||||||
|
|
||||||
|
import java.util.concurrent.Callable;
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
@RestController
|
@RestController
|
||||||
@RequestMapping("/GalleryManage")
|
@RequestMapping("/GalleryManage")
|
||||||
@Slf4j
|
@Slf4j
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class GalleryManageController {
|
public class GalleryManageController {
|
||||||
@Resource
|
final GalleryManageService galleryManageService;
|
||||||
GalleryManageService galleryManageService;
|
|
||||||
|
|
||||||
@Resource
|
final CollectService collectService;
|
||||||
CollectService collectService;
|
|
||||||
|
|
||||||
@Resource
|
final UserService userService;
|
||||||
CustomUtil customUtil;
|
|
||||||
|
|
||||||
@Resource
|
final RemoteService remoteService;
|
||||||
UserServiceImpl userService;
|
|
||||||
|
|
||||||
@PostMapping("")
|
@PostMapping("")
|
||||||
public String create_task(String link, String targetResolution, String AuthCode,
|
public String create_task(String link, String targetResolution, String AuthCode){
|
||||||
@RequestParam(value = "tags", required = false) List<Integer> tags,
|
|
||||||
@RequestParam(value = "mode", defaultValue = "3", required = false)String mode){
|
|
||||||
if(link == null || targetResolution == null)
|
if(link == null || targetResolution == null)
|
||||||
return Response._failure("参数不全");
|
return Response._failure("参数不全");
|
||||||
link = customUtil.restoreUrl(link);
|
return galleryManageService.createTask(link, targetResolution, AuthCode);
|
||||||
if(link == null)
|
|
||||||
return Response._failure("链接错误");
|
|
||||||
return galleryManageService.createTask(link, targetResolution, AuthCode, tags, Byte.parseByte(mode));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping("/update")
|
@PostMapping("/reconnect")
|
||||||
public String updateGallery(String link){
|
public String reconnect(){
|
||||||
return galleryManageService.updateGallery(link);
|
return galleryManageService.reconnect();
|
||||||
|
}
|
||||||
|
|
||||||
|
@PostMapping("/test")
|
||||||
|
public String test(){
|
||||||
|
remoteService.checkAvailability();
|
||||||
|
return "";
|
||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("")
|
@GetMapping("")
|
||||||
@@ -53,40 +50,29 @@ public class GalleryManageController {
|
|||||||
|
|
||||||
if(type == null)
|
if(type == null)
|
||||||
return Response._failure("参数不全");
|
return Response._failure("参数不全");
|
||||||
switch (type) {
|
return switch (type) {
|
||||||
case "link" -> {
|
case "link" -> galleryManageService.selectTaskByLink(param);
|
||||||
param = customUtil.restoreUrl(param);
|
case "gid" -> galleryManageService.selectTaskByGid(Integer.parseInt(param));
|
||||||
if (param == null)
|
case "all" -> galleryManageService.selectAllGallery(userId);
|
||||||
return Response._failure("链接错误");
|
case "name" -> galleryManageService.selectGalleryByName(param);
|
||||||
return galleryManageService.selectTaskByLink(param);
|
case "downloader" -> galleryManageService.selectGalleryByDownloader(AuthCode);
|
||||||
}
|
default -> Response._failure("参数错误");
|
||||||
case "gid" -> {
|
};
|
||||||
return galleryManageService.selectTaskByGid(Integer.parseInt(param));
|
|
||||||
}
|
|
||||||
case "all" -> {
|
|
||||||
return galleryManageService.selectAllGallery(userId);
|
|
||||||
}
|
|
||||||
case "name" -> {
|
|
||||||
return galleryManageService.selectGalleryByName(param);
|
|
||||||
}
|
|
||||||
case "undone" -> {
|
|
||||||
return galleryManageService.selectUnDoneGallery();
|
|
||||||
}
|
|
||||||
case "downloader" -> {
|
|
||||||
return galleryManageService.selectGalleryByDownloader(AuthCode);
|
|
||||||
}
|
|
||||||
default -> {
|
|
||||||
return Response._default();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@DeleteMapping("")
|
@DeleteMapping("")
|
||||||
public String deleteTask(Integer gid, String AuthCode, @RequestParam(value = "mode", defaultValue = "3", required = false)String mode){
|
public String deleteTask(Integer gid, String AuthCode){
|
||||||
if(gid == null)
|
if(gid == null)
|
||||||
return Response._failure("参数不全");
|
return Response._failure("参数不全");
|
||||||
|
return galleryManageService.deleteGalleryByGid(gid, AuthCode);
|
||||||
|
}
|
||||||
|
|
||||||
return galleryManageService.deleteGalleryByGid(gid, AuthCode, Byte.parseByte(mode));
|
/** 按 gid 查实际下载人昵称,仅管理员可用。 */
|
||||||
|
@GetMapping("/downloader")
|
||||||
|
public String selectDownloader(Integer gid, String AuthCode){
|
||||||
|
if(gid == null)
|
||||||
|
return Response._failure("参数不全");
|
||||||
|
return galleryManageService.selectDownloaderByGid(gid, AuthCode);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -105,27 +91,25 @@ public class GalleryManageController {
|
|||||||
return galleryManageService.getWeekUsedAmount();
|
return galleryManageService.getWeekUsedAmount();
|
||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/thumbnail/{name}")
|
@PostMapping("/cache")
|
||||||
public void getThumbnail(HttpServletRequest request, HttpServletResponse response, @PathVariable("name") String name){
|
public String cacheImageKeys(String url){
|
||||||
galleryManageService.getThumbnail(request, response, name.replace(".webp", ""));
|
return galleryManageService.cacheImagesKey(url);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@GetMapping("/onlineImage/{page}")
|
@GetMapping("/onlineImage/{page}")
|
||||||
public void getOnlineImage(Integer gid, @PathVariable("page") Short page, HttpServletRequest request, HttpServletResponse response){
|
public Callable<?> getCacheImage(String gid, @PathVariable("page") int page, HttpServletRequest request, HttpServletResponse response){
|
||||||
galleryManageService.getOnlineImage(gid, page, request, response);
|
return galleryManageService.getCachedImage(gid, page, request, response);
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@PostMapping("/share")
|
|
||||||
public String shareGallery(Integer gid, Integer userId, Integer expireHour){
|
|
||||||
if(userId.equals(3))
|
|
||||||
return galleryManageService.shareGallery(gid, expireHour);
|
|
||||||
return Response._failure("非法访问");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping("/reset")
|
@PostMapping("/reset")
|
||||||
public String resetUndone(){
|
public String resetUndone(){
|
||||||
return galleryManageService.resetUndone();
|
return galleryManageService.resetUndone();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@PostMapping("/retry")
|
||||||
|
public String retryGallery(Integer gid){
|
||||||
|
if(gid == null)
|
||||||
|
return Response._failure("参数不全");
|
||||||
|
return galleryManageService.retryGallery(gid);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,28 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Controller;
|
|
||||||
|
|
||||||
import org.springframework.stereotype.Controller;
|
|
||||||
import org.springframework.web.bind.annotation.GetMapping;
|
|
||||||
|
|
||||||
@Controller
|
|
||||||
public class NavigationController {
|
|
||||||
@GetMapping("/personal/")
|
|
||||||
public String personal_index(){
|
|
||||||
return "self";
|
|
||||||
}
|
|
||||||
|
|
||||||
@GetMapping("/personal/mobile")
|
|
||||||
public String personal_mobile(){
|
|
||||||
return "selfMobile";
|
|
||||||
}
|
|
||||||
|
|
||||||
@GetMapping("/")
|
|
||||||
public String index(){
|
|
||||||
return "index";
|
|
||||||
}
|
|
||||||
|
|
||||||
@GetMapping("/mobile")
|
|
||||||
public String mobile(){
|
|
||||||
return "mobile";
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
||||||
@@ -1,83 +1,123 @@
|
|||||||
package com.lion.lionwebsite.Controller;
|
package com.lion.lionwebsite.Controller;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Service.LocalServiceImpl;
|
import com.lion.lionwebsite.Service.LocalService;
|
||||||
import com.lion.lionwebsite.Service.PersonalServiceImpl;
|
import com.lion.lionwebsite.Service.PersonalService;
|
||||||
import com.lion.lionwebsite.Util.FileDownload;
|
import com.lion.lionwebsite.Util.PanelLoginTicket;
|
||||||
import com.lion.lionwebsite.Util.Response;
|
import com.lion.lionwebsite.Util.Response;
|
||||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import jakarta.servlet.http.HttpSession;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.http.MediaType;
|
||||||
import org.springframework.web.bind.annotation.GetMapping;
|
import org.springframework.web.bind.annotation.GetMapping;
|
||||||
import org.springframework.web.bind.annotation.PostMapping;
|
import org.springframework.web.bind.annotation.PostMapping;
|
||||||
import org.springframework.web.bind.annotation.RequestMapping;
|
import org.springframework.web.bind.annotation.RequestMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RequestParam;
|
||||||
import org.springframework.web.bind.annotation.RestController;
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
import org.springframework.web.multipart.MultipartFile;
|
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
|
|
||||||
|
|
||||||
@RestController
|
@RestController
|
||||||
@Slf4j
|
@Slf4j
|
||||||
|
@RequiredArgsConstructor
|
||||||
@RequestMapping("/personal")
|
@RequestMapping("/personal")
|
||||||
public class PersonalController {
|
public class PersonalController {
|
||||||
|
|
||||||
@Resource
|
/** 会话有效期,与 PersonalHub 的面板会话保持一致:14 天滑动过期。 */
|
||||||
PersonalServiceImpl personalService;
|
private static final int SESSION_MAX_INACTIVE_SECONDS = 60 * 60 * 24 * 14;
|
||||||
|
|
||||||
@Resource
|
final PersonalService personalService;
|
||||||
LocalServiceImpl localService;
|
|
||||||
|
|
||||||
@GetMapping("/sub/self")
|
final LocalService localService;
|
||||||
public void sub(HttpServletResponse response, HttpServletRequest request){
|
|
||||||
FileDownload.export(request, response, "sub/sub.txt");
|
final PanelLoginTicket panelLoginTicket;
|
||||||
|
|
||||||
|
@GetMapping("/")
|
||||||
|
public void index(HttpServletResponse resp) throws IOException {
|
||||||
|
resp.sendRedirect("/index");
|
||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/files")
|
/**
|
||||||
public String file(String path) throws IOException {
|
* 一键登录:校验机器人签发的 HMAC 票据,通过后建立会话并跳到个人面板。
|
||||||
return personalService.getFiles(path);
|
*
|
||||||
|
* <p>票据一次性之外的防护全靠有效期,因此失败的票据不重定向到登录页,
|
||||||
|
* 而是直接回到提示页,避免把参数回显到浏览器历史里。
|
||||||
|
*/
|
||||||
|
@GetMapping("/login")
|
||||||
|
public void login(@RequestParam(value = "t", required = false) String ticket,
|
||||||
|
HttpServletRequest request,
|
||||||
|
HttpServletResponse response) throws IOException {
|
||||||
|
if (!panelLoginTicket.verify(ticket)) {
|
||||||
|
log.warn("一键登录票据无效或已过期,来自 {}", clientIp(request));
|
||||||
|
// 重定向目标用浏览器可见路径:nginx 的 location / 会把它改写成 /personal/…
|
||||||
|
response.sendRedirect("/denied");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 先作废旧会话再建新的,避免会话固定攻击。
|
||||||
|
HttpSession existing = request.getSession(false);
|
||||||
|
if (existing != null)
|
||||||
|
existing.invalidate();
|
||||||
|
HttpSession session = request.getSession(true);
|
||||||
|
session.setAttribute("personalAuthenticated", Boolean.TRUE);
|
||||||
|
session.setMaxInactiveInterval(SESSION_MAX_INACTIVE_SECONDS);
|
||||||
|
|
||||||
|
log.info("一键登录成功,来自 {}", clientIp(request));
|
||||||
|
response.sendRedirect("/index");
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping("/uploadFile")
|
@GetMapping("/login/logout")
|
||||||
public String uploadFile(String path, String fileName, MultipartFile file){
|
public void logout(HttpServletRequest request, HttpServletResponse response) throws IOException {
|
||||||
return personalService.uploadFile(path, fileName, file);
|
HttpSession session = request.getSession(false);
|
||||||
|
if (session != null)
|
||||||
|
session.invalidate();
|
||||||
|
response.sendRedirect("/denied");
|
||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/private/**")
|
/** 未登录或票据失效时的提示页;纯静态文案,不含任何可推断的信息。 */
|
||||||
public void getFile(HttpServletRequest request, HttpServletResponse response, String path){
|
@GetMapping(value = "/denied", produces = MediaType.TEXT_HTML_VALUE)
|
||||||
personalService.download(request, response, path);
|
public String denied() {
|
||||||
|
return """
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="zh-CN">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<title>需要登录 · LionWebsite</title>
|
||||||
|
<style>
|
||||||
|
body { margin:0; display:flex; min-height:100vh; align-items:center; justify-content:center;
|
||||||
|
background:#f6f7f9; color:#1f2328;
|
||||||
|
font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",sans-serif; }
|
||||||
|
.card { max-width:26rem; padding:2rem; background:#fff; border:1px solid #e2e5e9;
|
||||||
|
border-radius:10px; box-shadow:0 1px 2px rgba(0,0,0,.04); }
|
||||||
|
h1 { margin:0 0 .75rem; font-size:1.15rem; }
|
||||||
|
p { margin:0; line-height:1.7; color:#4a5259; }
|
||||||
|
code { background:#f0f2f5; padding:.1rem .35rem; border-radius:4px; }
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="card">
|
||||||
|
<h1>需要登录</h1>
|
||||||
|
<p>请在机器人里发送 <code>/login</code>,用返回的一次性链接打开个人面板。</p>
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
""";
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping("/share")
|
private static String clientIp(HttpServletRequest request) {
|
||||||
public String shareFile(String path, Integer expireHour) {
|
String forwarded = request.getHeader("X-Forwarded-For");
|
||||||
return personalService.shareFile(path, expireHour);
|
if (forwarded != null && !forwarded.isBlank())
|
||||||
}
|
return forwarded.split(",")[0].trim();
|
||||||
|
return request.getRemoteAddr();
|
||||||
@PostMapping("/compress")
|
|
||||||
public String compress(String path){
|
|
||||||
return personalService.compress(path);
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping("/delete")
|
|
||||||
public String deleteFile(String path){
|
|
||||||
return personalService.deleteFile(path);
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping("/extendShareTime")
|
|
||||||
public String extendShareTime(String path, Integer extendHour) {
|
|
||||||
return personalService.extendShareTime(path, extendHour);
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping("/cancelShare")
|
|
||||||
public String cancelShare(String path){
|
|
||||||
return personalService.cancelShare(path);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping("/updateSub")
|
@PostMapping("/updateSub")
|
||||||
public String updateSub() throws IOException {
|
public String updateSub() throws IOException {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
if(localService.updateSub(true))
|
if(localService.updateSub())
|
||||||
response.success();
|
response.success();
|
||||||
else
|
else
|
||||||
response.failure();
|
response.failure();
|
||||||
@@ -90,7 +130,12 @@ public class PersonalController {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/ip")
|
@GetMapping("/ip")
|
||||||
public String ip() throws JsonProcessingException {
|
public String ip() {
|
||||||
return personalService.getIp();
|
return personalService.getIp();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@PostMapping("/message2me")
|
||||||
|
public String message2me(String message) {
|
||||||
|
return personalService.message2me(message);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,32 +2,40 @@ package com.lion.lionwebsite.Controller;
|
|||||||
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.User;
|
import com.lion.lionwebsite.Domain.User;
|
||||||
import com.lion.lionwebsite.Service.PublicServiceImpl;
|
import com.lion.lionwebsite.Service.PublicService;
|
||||||
|
import com.lion.lionwebsite.Service.QueryService;
|
||||||
import com.lion.lionwebsite.Service.RemoteService;
|
import com.lion.lionwebsite.Service.RemoteService;
|
||||||
import com.lion.lionwebsite.Util.FileDownload;
|
import com.lion.lionwebsite.Service.SubService;
|
||||||
|
import com.lion.lionwebsite.Service.UserService;
|
||||||
|
import com.lion.lionwebsite.Util.CustomUtil;
|
||||||
import com.lion.lionwebsite.Util.Response;
|
import com.lion.lionwebsite.Util.Response;
|
||||||
import jakarta.annotation.Resource;
|
import tools.jackson.databind.node.ObjectNode;
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.web.bind.annotation.*;
|
import org.springframework.web.bind.annotation.*;
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.nio.charset.StandardCharsets;
|
import java.nio.charset.StandardCharsets;
|
||||||
import java.util.LinkedList;
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
@RestController
|
@RestController
|
||||||
@Slf4j
|
@Slf4j
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class PublicController {
|
public class PublicController {
|
||||||
|
|
||||||
final List<String> black_share_codes = new LinkedList<>();
|
final PublicService publicService;
|
||||||
|
|
||||||
@Resource
|
final RemoteService remoteService;
|
||||||
PublicServiceImpl publicService;
|
|
||||||
|
|
||||||
@Resource
|
final SubService subService;
|
||||||
RemoteService remoteService;
|
|
||||||
|
final QueryService queryService;
|
||||||
|
|
||||||
|
@GetMapping("/")
|
||||||
|
public void index(HttpServletResponse resp) throws IOException {
|
||||||
|
resp.sendRedirect("/index");
|
||||||
|
}
|
||||||
|
|
||||||
@GetMapping("/ip")
|
@GetMapping("/ip")
|
||||||
public void ip(HttpServletRequest request, String auth, HttpServletResponse response) throws IOException {
|
public void ip(HttpServletRequest request, String auth, HttpServletResponse response) throws IOException {
|
||||||
@@ -42,52 +50,26 @@ public class PublicController {
|
|||||||
response.getOutputStream().write(ip.getBytes(StandardCharsets.UTF_8));
|
response.getOutputStream().write(ip.getBytes(StandardCharsets.UTF_8));
|
||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/sub/{client}/{AuthCode}")
|
@GetMapping("/sub/{client}/{key}")
|
||||||
public void publicSub(@PathVariable("AuthCode") String AuthCode,
|
public void publicSub(@PathVariable("key") String key,
|
||||||
@PathVariable("client") String client,
|
@PathVariable("client") String client,
|
||||||
HttpServletResponse response,
|
HttpServletResponse response,
|
||||||
HttpServletRequest request) throws IOException {
|
HttpServletRequest request) {
|
||||||
|
subService.updateSub(response, request, client, key);
|
||||||
if(AuthCode == null || client == null || !AuthCode.equals("covid"))
|
|
||||||
return;
|
|
||||||
|
|
||||||
switch (client){
|
|
||||||
case "v2ray" -> FileDownload.export(request, response, "sub/DouNaiV2ray.txt");
|
|
||||||
case "clash" -> FileDownload.export(request, response, "sub/DouNaiClash.txt");
|
|
||||||
default -> response.getOutputStream().write("client error".getBytes(StandardCharsets.UTF_8));
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
@GetMapping("/GetFile/{path}")
|
|
||||||
public void getFile(HttpServletRequest request, HttpServletResponse response, String ShareCode, @PathVariable("path") String path) throws IOException {
|
|
||||||
synchronized (black_share_codes) {
|
|
||||||
if (black_share_codes.contains(ShareCode)) {
|
|
||||||
String ip = request.getHeader("X-Forwarded-For") == null ? request.getRemoteAddr() : request.getHeader("X-Forwarded-For");
|
|
||||||
log.info("dispatch request ip:{} file:{}", ip, path);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
log.info("ShareCode:{}", ShareCode);
|
|
||||||
log.info("Path:{}", path);
|
|
||||||
boolean result = publicService.GetFile(request, response, ShareCode);
|
|
||||||
|
|
||||||
if(!result)
|
|
||||||
black_share_codes.add(ShareCode);
|
|
||||||
|
|
||||||
if(black_share_codes.size() > 100)
|
|
||||||
black_share_codes.remove(0);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping("/validate")
|
@PostMapping("/validate")
|
||||||
public String validate(String AuthCode){
|
public String validate(String AuthCode){
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
User user = publicService.getUserId(AuthCode);
|
User user = publicService.getUserId(AuthCode);
|
||||||
String isAvailable = remoteService.isDead() ? "false": "true";
|
// 身份字段直接作为结构化 data 返回;共享前端解码器兼容旧版本字符串。
|
||||||
response.success(String.format("{\"userId\": %d, " +
|
ObjectNode identity = CustomUtil.objectMapper.createObjectNode();
|
||||||
"\"username\": \"%s\", " +
|
identity.put("userId", user.getId());
|
||||||
"\"isAvailable\": %s}", user.getId(), user.getUsername(), isAvailable));
|
identity.put("username", user.getUsername());
|
||||||
|
identity.put("isAvailable", !remoteService.isDead());
|
||||||
|
// 管理员标记随登录一起下发,前端据此决定是否显示下载人信息与筛选。
|
||||||
|
identity.put("isAdmin", user.getId() == UserService.ADMIN_USER_ID);
|
||||||
|
response.success(identity);
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -97,8 +79,7 @@ public class PublicController {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/GalleryManage/ehThumbnail")
|
@GetMapping("/GalleryManage/ehThumbnail")
|
||||||
public void getEhThumbnail(String path, HttpServletResponse response){
|
public void getEhThumbnail(String path, HttpServletRequest request, HttpServletResponse response){
|
||||||
publicService.getEhThumbnail(path, response);
|
queryService.getEhThumbnail(path, request, response);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,8 +1,7 @@
|
|||||||
package com.lion.lionwebsite.Controller;
|
package com.lion.lionwebsite.Controller;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Service.QueryService;
|
import com.lion.lionwebsite.Service.QueryService;
|
||||||
import jakarta.annotation.Resource;
|
import lombok.RequiredArgsConstructor;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
|
||||||
import org.springframework.web.bind.annotation.GetMapping;
|
import org.springframework.web.bind.annotation.GetMapping;
|
||||||
import org.springframework.web.bind.annotation.RequestMapping;
|
import org.springframework.web.bind.annotation.RequestMapping;
|
||||||
import org.springframework.web.bind.annotation.RestController;
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
@@ -10,20 +9,13 @@ import org.springframework.web.bind.annotation.RestController;
|
|||||||
|
|
||||||
@RestController
|
@RestController
|
||||||
@RequestMapping("/query")
|
@RequestMapping("/query")
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class QueryController {
|
public class QueryController {
|
||||||
|
|
||||||
@Resource
|
final QueryService queryService;
|
||||||
QueryService queryService;
|
|
||||||
|
|
||||||
|
|
||||||
@GetMapping("")
|
@GetMapping("")
|
||||||
public String query(String keyword, String prev, String next){
|
public String query(String keyword, String prev, String next){
|
||||||
return queryService.query(keyword, prev, next);
|
return queryService.query(keyword, prev, next);
|
||||||
}
|
}
|
||||||
|
|
||||||
@GetMapping("/image")
|
|
||||||
public void image(HttpServletResponse response, String path){
|
|
||||||
queryService.image(response, path);
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
package com.lion.lionwebsite.Controller;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Service.SubService;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import org.springframework.web.bind.annotation.*;
|
||||||
|
|
||||||
|
@RestController
|
||||||
|
@RequestMapping("/personal/subBind/")
|
||||||
|
@RequiredArgsConstructor
|
||||||
|
public class SubController {
|
||||||
|
final SubService subService;
|
||||||
|
|
||||||
|
@PostMapping("")
|
||||||
|
public String addSubBind(String user, Integer accountId){
|
||||||
|
return subService.insertSubBind(user, accountId);
|
||||||
|
}
|
||||||
|
|
||||||
|
@PutMapping("")
|
||||||
|
public String resetKey(String user){
|
||||||
|
return subService.resetKey(user);
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("all")
|
||||||
|
public String getAllSubBind(){
|
||||||
|
return subService.selectAllSubBind();
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("allRecord")
|
||||||
|
public String getAllSubUpdateRecord(){
|
||||||
|
return subService.SelectAllSubUpdateRecord();
|
||||||
|
}
|
||||||
|
|
||||||
|
@DeleteMapping("")
|
||||||
|
public String deleteSubBind(String user){
|
||||||
|
return subService.deleteSubBind(user);
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("accounts")
|
||||||
|
public String getAccounts(){
|
||||||
|
return subService.listSubscriptionAccounts();
|
||||||
|
}
|
||||||
|
|
||||||
|
@PostMapping("accounts")
|
||||||
|
public String addAccount(String name, String upstreamKey,
|
||||||
|
@RequestParam(defaultValue = "true") boolean filterHighMultiplier,
|
||||||
|
@RequestParam(defaultValue = "true") boolean enabled){
|
||||||
|
return subService.insertSubscriptionAccount(name, upstreamKey, filterHighMultiplier, enabled);
|
||||||
|
}
|
||||||
|
|
||||||
|
@PutMapping("accounts/{id}")
|
||||||
|
public String updateAccount(@PathVariable Integer id, String name, String upstreamKey,
|
||||||
|
@RequestParam(defaultValue = "true") boolean filterHighMultiplier,
|
||||||
|
@RequestParam(defaultValue = "true") boolean enabled){
|
||||||
|
return subService.updateSubscriptionAccount(id, name, upstreamKey, filterHighMultiplier, enabled);
|
||||||
|
}
|
||||||
|
|
||||||
|
@PostMapping("accounts/{id}/refresh")
|
||||||
|
public String refreshAccount(@PathVariable Integer id){
|
||||||
|
return subService.refreshSubscriptionAccount(id);
|
||||||
|
}
|
||||||
|
|
||||||
|
@DeleteMapping("accounts/{id}")
|
||||||
|
public String deleteAccount(@PathVariable Integer id){
|
||||||
|
return subService.deleteSubscriptionAccount(id);
|
||||||
|
}
|
||||||
|
|
||||||
|
@PutMapping("{user}/account")
|
||||||
|
public String rebind(@PathVariable String user, Integer accountId){
|
||||||
|
return subService.rebind(user, accountId);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
package com.lion.lionwebsite.Controller;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Service.SubscriptionEventService;
|
||||||
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
|
import org.springframework.http.ResponseEntity;
|
||||||
|
import org.springframework.web.bind.annotation.PostMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
@RestController
|
||||||
|
public class SubscriptionEventController {
|
||||||
|
private final SubscriptionEventService events;
|
||||||
|
|
||||||
|
public SubscriptionEventController(SubscriptionEventService events) { this.events = events; }
|
||||||
|
|
||||||
|
@PostMapping(value = "/internal/subscription-events", produces = "application/json")
|
||||||
|
public ResponseEntity<Map<String, Object>> ingest(HttpServletRequest request) throws IOException {
|
||||||
|
// Bound actual bytes even for chunked requests and dishonest Content-Length headers.
|
||||||
|
byte[] body = request.getInputStream().readNBytes(SubscriptionEventService.MAX_BODY + 1);
|
||||||
|
if (body.length > SubscriptionEventService.MAX_BODY)
|
||||||
|
return ResponseEntity.badRequest().body(Map.of("error", "invalid subscription events"));
|
||||||
|
if (!events.verify(body, request.getHeader("X-Subscription-Timestamp"), request.getHeader("X-Subscription-Signature")))
|
||||||
|
return ResponseEntity.status(403).body(Map.of("error", "authentication failed"));
|
||||||
|
try {
|
||||||
|
return ResponseEntity.ok(events.ingest(events.parseBatch(body)));
|
||||||
|
} catch (IllegalArgumentException invalid) {
|
||||||
|
return ResponseEntity.badRequest().body(Map.of("error", "invalid subscription events"));
|
||||||
|
}
|
||||||
|
// All infrastructure failures propagate as 5xx; never acknowledge an uncommitted batch.
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,44 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Controller;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Service.TagService;
|
|
||||||
import com.lion.lionwebsite.Util.Response;
|
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import org.springframework.web.bind.annotation.*;
|
|
||||||
|
|
||||||
@RestController
|
|
||||||
@RequestMapping("/GalleryManage")
|
|
||||||
public class TagController {
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
TagService tagService;
|
|
||||||
|
|
||||||
@PostMapping("/tag")
|
|
||||||
public String createTag(String tag){
|
|
||||||
return tagService.createTag(tag);
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping("/tagAndMark")
|
|
||||||
public String createTagAndMark(Integer gid, String tag){
|
|
||||||
return tagService.createTagAndMark(gid, tag);
|
|
||||||
}
|
|
||||||
|
|
||||||
@DeleteMapping("/tag")
|
|
||||||
public String deleteTag(Integer tid){
|
|
||||||
return tid == null ? Response._failure("参数错误") : tagService.deleteTag(tid);
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping("/mark")
|
|
||||||
public String markTag(Integer gid, Integer tid){
|
|
||||||
return tagService.markTag(gid, tid);
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping("/disMark")
|
|
||||||
public String disMarkTag(Integer gid, Integer tid){
|
|
||||||
return tagService.disMarkTag(gid, tid);
|
|
||||||
}
|
|
||||||
|
|
||||||
@GetMapping("/allTag")
|
|
||||||
public String allTag(){
|
|
||||||
return tagService.selectAllTag();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,15 +1,15 @@
|
|||||||
package com.lion.lionwebsite.Controller;
|
package com.lion.lionwebsite.Controller;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Service.UserServiceImpl;
|
import com.lion.lionwebsite.Service.UserService;
|
||||||
import jakarta.annotation.Resource;
|
import lombok.RequiredArgsConstructor;
|
||||||
import org.springframework.web.bind.annotation.*;
|
import org.springframework.web.bind.annotation.*;
|
||||||
|
|
||||||
|
|
||||||
@RestController
|
@RestController
|
||||||
@RequestMapping("/personal/user")
|
@RequestMapping("/personal/user")
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class UserController {
|
public class UserController {
|
||||||
@Resource
|
final UserService userService;
|
||||||
UserServiceImpl userService;
|
|
||||||
|
|
||||||
@GetMapping("")
|
@GetMapping("")
|
||||||
public String getAllUser(){
|
public String getAllUser(){
|
||||||
|
|||||||
@@ -1,26 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.CRC32Cache;
|
|
||||||
import org.apache.ibatis.annotations.Delete;
|
|
||||||
import org.apache.ibatis.annotations.Insert;
|
|
||||||
import org.apache.ibatis.annotations.Mapper;
|
|
||||||
import org.apache.ibatis.annotations.Select;
|
|
||||||
|
|
||||||
@Mapper
|
|
||||||
public interface CRC32CacheMapper {
|
|
||||||
@Insert("insert into CRC32Cache (path, lastModifyTime, crc32) values (#{path}, #{lastModifyTime}, #{crc32})")
|
|
||||||
void insertCache(CRC32Cache crc32Cache);
|
|
||||||
|
|
||||||
@Select("select * from CRC32Cache where path=#{path}")
|
|
||||||
CRC32Cache selectCache(String path);
|
|
||||||
|
|
||||||
@Select("select * from CRC32Cache where path like '%' || #{name} || '%'")
|
|
||||||
CRC32Cache selectCacheByName(String name);
|
|
||||||
|
|
||||||
// @Update("update CRC32Cache set lastModifyTime=#{lastModifyTime}, crc32=#{crc32} where path=#{path}")
|
|
||||||
// void updateCache(CRC32Cache crc32Cache);
|
|
||||||
|
|
||||||
@Delete("delete from CRC32Cache where path=#{path}")
|
|
||||||
void deleteCacheByPath(String path);
|
|
||||||
|
|
||||||
}
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.PageNameCache;
|
|
||||||
import org.apache.ibatis.annotations.*;
|
|
||||||
|
|
||||||
@Mapper
|
|
||||||
public interface PageNameCacheMapper {
|
|
||||||
|
|
||||||
@Insert("insert into pageName (gid, page, pageName) values (#{gid}, #{page}, #{pageName})")
|
|
||||||
void insertPageNameCache(PageNameCache pageNameCache);
|
|
||||||
|
|
||||||
|
|
||||||
@Select("select pageName from pageName where gid=#{gid} and page=#{page}")
|
|
||||||
String selectPageName(@Param("gid") int gid, @Param("page") short page);
|
|
||||||
|
|
||||||
|
|
||||||
@Delete("delete from pageName where gid=#{gid}")
|
|
||||||
void deletePageNameByGid(int gid);
|
|
||||||
|
|
||||||
@Delete("delete from pageName where gid=#{gid} and page=#{page}")
|
|
||||||
void deletePageName(PageNameCache pageNameCache);
|
|
||||||
}
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.ShareFile;
|
|
||||||
import org.apache.ibatis.annotations.*;
|
|
||||||
|
|
||||||
import java.util.ArrayList;
|
|
||||||
import java.util.Date;
|
|
||||||
|
|
||||||
@Mapper
|
|
||||||
public interface ShareFileMapper {
|
|
||||||
@Insert("insert into ShareFile (ShareCode, FilePath, ExpireTime) values (#{ShareCode}, #{FilePath}, #{ExpireTime})")
|
|
||||||
void insertShareFile(@Param("ShareCode")String ShareCode, @Param("FilePath")String FilePath, @Param("ExpireTime") Date ExpireTime);
|
|
||||||
|
|
||||||
@Select("select * from ShareFile where ShareCode=#{ShareCode}")
|
|
||||||
ShareFile selectShareFileByShareCode(String ShareCode);
|
|
||||||
|
|
||||||
@Select("select * from ShareFile where FilePath=#{FilePath}")
|
|
||||||
ShareFile selectShareFileByFilePath(String FilePath);
|
|
||||||
|
|
||||||
@Select("select * from ShareFile where FilePath like '%' || #{FilePath} || '%'")
|
|
||||||
ArrayList<ShareFile> selectShareFilesByFilePath(String FilePath);
|
|
||||||
|
|
||||||
@Delete("delete from ShareFile where ShareCode=#{ShareCode}")
|
|
||||||
void deleteShareFile(String ShareCode);
|
|
||||||
|
|
||||||
@Select("select * from ShareFile")
|
|
||||||
ShareFile[] selectAllShareFile();
|
|
||||||
|
|
||||||
@Update("update ShareFile set ExpireTime=#{ExpireTime}, ShareCode=#{ShareCode} where FilePath=#{FilePath}")
|
|
||||||
void updateShareFile(ShareFile ShareFile);
|
|
||||||
}
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.Tag;
|
|
||||||
import com.lion.lionwebsite.Domain.TagMark;
|
|
||||||
import org.apache.ibatis.annotations.*;
|
|
||||||
|
|
||||||
import java.util.ArrayList;
|
|
||||||
|
|
||||||
@Mapper
|
|
||||||
public interface TagMapper {
|
|
||||||
@Insert("insert into tag (tag) values (#{tag})")
|
|
||||||
@Options(useGeneratedKeys = true, keyProperty = "id", keyColumn = "id")
|
|
||||||
void insertTag(Tag tag);
|
|
||||||
|
|
||||||
// @Select("select id, tag, usage from tag where id=#{id}")
|
|
||||||
// Tag selectTagById(int id);
|
|
||||||
//
|
|
||||||
// @Select("select id from tag where tag=#{tag}")
|
|
||||||
// int selectTidByTag(String tag);
|
|
||||||
|
|
||||||
@Select("select count(id) from tag where tag=#{tag}")
|
|
||||||
int selectTagExistByTag(String tag);
|
|
||||||
|
|
||||||
@Select("select count(id) from tag where id=#{id}")
|
|
||||||
int selectTagExistById(int id);
|
|
||||||
|
|
||||||
|
|
||||||
@Select("select id, tag, usage from tag")
|
|
||||||
ArrayList<Tag> selectAllTag();
|
|
||||||
|
|
||||||
@Delete("delete from tag where id=#{id}")
|
|
||||||
void deleteTagById(int id);
|
|
||||||
|
|
||||||
@Select("select count(id) from galleryTag where tid=#{tid}")
|
|
||||||
int selectTagUsage(int tid);
|
|
||||||
|
|
||||||
@Select("select count(id) from galleryTag where gid=#{gid} and tid=#{tid}")
|
|
||||||
int selectIsMark(@Param("gid") int gid, @Param("tid") int tid);
|
|
||||||
|
|
||||||
@Select("select tid from galleryTag where gid=#{gid}")
|
|
||||||
ArrayList<Integer> selectTagByGid(int gid);
|
|
||||||
|
|
||||||
@Insert("insert into galleryTag (gid, tid) values (#{gid}, #{tid})")
|
|
||||||
int markTag(@Param("gid") int gid, @Param("tid") int tid);
|
|
||||||
|
|
||||||
@Delete("delete from galleryTag where gid=#{gid} and tid=#{tid}")
|
|
||||||
int disMarkTag(@Param("gid") int gid, @Param("tid") int tid);
|
|
||||||
|
|
||||||
@Select("select gid, tid from galleryTag")
|
|
||||||
ArrayList<TagMark> selectAllMark();
|
|
||||||
|
|
||||||
@Update("update tag set usage=usage+1 where id=#{tid}")
|
|
||||||
void incrTagUsage(int tid);
|
|
||||||
|
|
||||||
@Update("update tag set usage=usage-1 where id=#{tid}")
|
|
||||||
void decrTagUsage(int tid);
|
|
||||||
|
|
||||||
@Delete("delete from galleryTag where gid=#{gid}")
|
|
||||||
void disMarkTagByGid(int gid);
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
package com.lion.lionwebsite.Dao.cache;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.GidToKey;
|
||||||
|
import com.lion.lionwebsite.Domain.ImageKeyCache;
|
||||||
|
import org.apache.ibatis.annotations.Insert;
|
||||||
|
import org.apache.ibatis.annotations.Mapper;
|
||||||
|
import org.apache.ibatis.annotations.Param;
|
||||||
|
import org.apache.ibatis.annotations.Delete;
|
||||||
|
import org.apache.ibatis.annotations.Select;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
@Mapper
|
||||||
|
public interface ImageCacheMapper {
|
||||||
|
@Insert("insert into ImageKeyCache values (#{gid}, #{imgkey}, #{page})")
|
||||||
|
void insertImageKeyCache(ImageKeyCache imageKeyCache);
|
||||||
|
|
||||||
|
@Insert("insert into gidToKey values (#{gid}, #{key}, #{pages})")
|
||||||
|
void insertGidToKey(GidToKey gidToKey);
|
||||||
|
|
||||||
|
@Select("select * from ImageKeyCache where gid=#{gid} and page=#{page}")
|
||||||
|
ImageKeyCache selectImageKeyCacheByGidAndPage(@Param("gid") String gid, @Param("page") int page);
|
||||||
|
|
||||||
|
@Select("select * from gidToKey where gid=#{gid}")
|
||||||
|
GidToKey selectKeyByGid(String gid);
|
||||||
|
|
||||||
|
/** 已缓存的页 key 行数,用于判断索引是否完整(半截缓存需要重建)。 */
|
||||||
|
@Select("select count(*) from ImageKeyCache where gid=#{gid}")
|
||||||
|
int countImageKeyCacheByGid(String gid);
|
||||||
|
|
||||||
|
@Delete("delete from ImageKeyCache where gid=#{gid}")
|
||||||
|
void deleteImageKeyCacheByGid(String gid);
|
||||||
|
|
||||||
|
@Delete("delete from gidToKey where gid=#{gid}")
|
||||||
|
void deleteGidToKey(String gid);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 某个 gid 已落库的页码列表(升序)。
|
||||||
|
*
|
||||||
|
* <p>只按行数判断完整性会把「页码 1 和 3、共 2 行」当成完整索引,
|
||||||
|
* 因此完整性判定改为按页码集合比对——先取回页码,再校验它是否恰好是 1..pages。
|
||||||
|
*/
|
||||||
|
@Select("select page from ImageKeyCache where gid=#{gid} order by page")
|
||||||
|
List<Integer> selectPageNumbersByGid(String gid);
|
||||||
|
|
||||||
|
/** 该 gid 下 imgkey 为空的脏行数;任何一行脏行都说明索引不可用。 */
|
||||||
|
@Select("select count(*) from ImageKeyCache where gid=#{gid} and (imgkey is null or trim(imgkey)='')")
|
||||||
|
int countInvalidImageKeysByGid(String gid);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 批量写入页 key(单条多值 INSERT)。
|
||||||
|
*
|
||||||
|
* <p>整本画廊的索引可能上千页,逐行 INSERT 会拉长写事务并放大锁竞争;
|
||||||
|
* 调用方按固定大小分片后,一个写事务里只需少量语句即可写完。
|
||||||
|
*/
|
||||||
|
// MyBatis 只在注解值「以 <script> 开头」时才按 XML 动态 SQL 解析,因此这里不能有前导空白。
|
||||||
|
@Insert("<script>"
|
||||||
|
+ "insert into ImageKeyCache (gid, imgkey, page) values "
|
||||||
|
+ "<foreach collection='pages' item='p' separator=','>"
|
||||||
|
+ "(#{p.gid}, #{p.imgkey}, #{p.page})"
|
||||||
|
+ "</foreach>"
|
||||||
|
+ "</script>")
|
||||||
|
int insertImageKeyCacheBatch(@Param("pages") List<ImageKeyCache> pages);
|
||||||
|
}
|
||||||
+5
-1
@@ -1,4 +1,4 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
import org.apache.ibatis.annotations.*;
|
import org.apache.ibatis.annotations.*;
|
||||||
|
|
||||||
@@ -20,4 +20,8 @@ public interface CollectMapper {
|
|||||||
|
|
||||||
@Select("select collector from collect where gid=#{gid}")
|
@Select("select collector from collect where gid=#{gid}")
|
||||||
ArrayList<Integer> selectCollectorByGid(int gid);
|
ArrayList<Integer> selectCollectorByGid(int gid);
|
||||||
|
|
||||||
|
/** 任务删除完成后清理该 gid 的全部收藏记录(与删任务行同一事务)。 */
|
||||||
|
@Delete("delete from collect where gid=#{gid}")
|
||||||
|
void deleteByGid(@Param("gid") int gid);
|
||||||
}
|
}
|
||||||
+4
-1
@@ -1,4 +1,4 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
||||||
import org.apache.ibatis.annotations.Mapper;
|
import org.apache.ibatis.annotations.Mapper;
|
||||||
@@ -16,6 +16,9 @@ public interface CustomConfigurationMapper {
|
|||||||
// @Delete("delete from customConfiguration where parameter=#{parameter}")
|
// @Delete("delete from customConfiguration where parameter=#{parameter}")
|
||||||
// void deleteConfiguration(CustomConfiguration configuration);
|
// void deleteConfiguration(CustomConfiguration configuration);
|
||||||
|
|
||||||
|
@Update("update customConfiguration set value=cast(value as integer)+#{amount} where parameter=#{parameter}")
|
||||||
|
void incrementConfiguration(@Param("parameter") String parameter, @Param("amount") long amount);
|
||||||
|
|
||||||
@Select("select * from customConfiguration where parameter=#{parameter}")
|
@Select("select * from customConfiguration where parameter=#{parameter}")
|
||||||
CustomConfiguration selectConfiguration(String parameter);
|
CustomConfiguration selectConfiguration(String parameter);
|
||||||
}
|
}
|
||||||
+12
-7
@@ -1,4 +1,4 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.Gallery;
|
import com.lion.lionwebsite.Domain.Gallery;
|
||||||
import org.apache.ibatis.annotations.*;
|
import org.apache.ibatis.annotations.*;
|
||||||
@@ -9,8 +9,8 @@ public interface GalleryMapper {
|
|||||||
|
|
||||||
|
|
||||||
@Insert("insert into gallery" +
|
@Insert("insert into gallery" +
|
||||||
" (gid, name, link, language, pages, status, fileSize, createTime, proceeding, resolution, displayFileSize, downloader, mode)\n" +
|
" (gid, name, link, language, pages, status, fileSize, createTime, proceeding, resolution, displayFileSize, downloader, thumb_link, is_download)\n" +
|
||||||
" values (#{gid}, #{name}, #{link}, #{language}, #{pages}, #{status}, #{fileSize}, #{createTime}, #{proceeding}, #{resolution}, #{displayFileSize}, #{downloader}, #{mode})")
|
" values (#{gid}, #{name}, #{link}, #{language}, #{pages}, #{status}, #{fileSize}, #{createTime}, #{proceeding}, #{resolution}, #{displayFileSize}, #{downloader}, #{thumb_link}, #{is_download})")
|
||||||
void insertGallery(Gallery gallery);
|
void insertGallery(Gallery gallery);
|
||||||
|
|
||||||
@Select("select * from gallery where link=#{link}")
|
@Select("select * from gallery where link=#{link}")
|
||||||
@@ -25,10 +25,10 @@ public interface GalleryMapper {
|
|||||||
@Select("select * from gallery where downloader=#{downloader}")
|
@Select("select * from gallery where downloader=#{downloader}")
|
||||||
Gallery[] selectGalleryByDownloader(int downloader);
|
Gallery[] selectGalleryByDownloader(int downloader);
|
||||||
|
|
||||||
@Select("select * from gallery where status in ('已提交', '下载中')")
|
@Select("select * from gallery where status in ('已提交', '下载中', '等待压缩', '压缩中')")
|
||||||
Gallery[] selectUnDoneGalleries();
|
Gallery[] selectUnDoneGalleries();
|
||||||
|
|
||||||
@Select("select * from gallery")
|
@Select("select * from gallery order by createTime")
|
||||||
Gallery[] selectAllGallery();
|
Gallery[] selectAllGallery();
|
||||||
|
|
||||||
@Select("select * from gallery where name like #{name} limit 1")
|
@Select("select * from gallery where name like #{name} limit 1")
|
||||||
@@ -36,9 +36,10 @@ public interface GalleryMapper {
|
|||||||
|
|
||||||
|
|
||||||
@Update("""
|
@Update("""
|
||||||
update gallery set name=#{name}, link=#{link}, language=#{language},
|
update gallery set name=#{name}, link=#{link}, language=#{language}, downloader=#{downloader},
|
||||||
pages=#{pages}, status=#{status}, fileSize=#{fileSize}, createTime=#{createTime},
|
pages=#{pages}, status=#{status}, fileSize=#{fileSize}, createTime=#{createTime},
|
||||||
proceeding=#{proceeding}, resolution=#{resolution}, displayFileSize=#{displayFileSize}, mode=#{mode}
|
proceeding=#{proceeding}, resolution=#{resolution}, displayFileSize=#{displayFileSize},
|
||||||
|
thumb_link=#{thumb_link}, is_download=#{is_download}
|
||||||
where gid=#{gid}""")
|
where gid=#{gid}""")
|
||||||
void updateGallery(Gallery gallery);
|
void updateGallery(Gallery gallery);
|
||||||
|
|
||||||
@@ -47,4 +48,8 @@ public interface GalleryMapper {
|
|||||||
|
|
||||||
@Delete("delete from gallery where gid=#{gid}")
|
@Delete("delete from gallery where gid=#{gid}")
|
||||||
void deleteGalleryByGid(Integer gid);
|
void deleteGalleryByGid(Integer gid);
|
||||||
|
|
||||||
|
/** 只改状态列,用于「待删除」这类不牵动其他字段的意图标记。 */
|
||||||
|
@Update("update gallery set status=#{status} where gid=#{gid}")
|
||||||
|
int updateGalleryStatus(@Param("gid") int gid, @Param("status") String status);
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.SubBind;
|
||||||
|
import com.lion.lionwebsite.Domain.SubUpdateRecord;
|
||||||
|
import com.lion.lionwebsite.Domain.SubscriptionAccount;
|
||||||
|
import org.apache.ibatis.annotations.*;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
|
||||||
|
@Mapper
|
||||||
|
public interface SubMapper {
|
||||||
|
@Insert("insert into subscription_account (name, upstream_key, filter_high_multiplier, enabled, created_at, updated_at) values (#{name}, #{upstreamKey}, #{filterHighMultiplier}, #{enabled}, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)")
|
||||||
|
@Options(useGeneratedKeys = true, keyProperty = "id")
|
||||||
|
void insertSubscriptionAccount(SubscriptionAccount account);
|
||||||
|
|
||||||
|
@Select("select id, name, upstream_key as upstreamKey, filter_high_multiplier as filterHighMultiplier, enabled, last_success_at as lastSuccessAt, last_error as lastError, created_at as createdAt, updated_at as updatedAt, next_refresh_at as nextRefreshAt, last_success_epoch as lastSuccessEpoch, (select count(*) from sub_bind sb where sb.subscription_account_id = sa.id) as boundUserCount from subscription_account sa order by id")
|
||||||
|
ArrayList<SubscriptionAccount> selectAllSubscriptionAccounts();
|
||||||
|
|
||||||
|
@Select("select id, name, upstream_key as upstreamKey, filter_high_multiplier as filterHighMultiplier, enabled, last_success_at as lastSuccessAt, last_error as lastError, created_at as createdAt, updated_at as updatedAt, next_refresh_at as nextRefreshAt, last_success_epoch as lastSuccessEpoch, (select count(*) from sub_bind sb where sb.subscription_account_id = sa.id) as boundUserCount from subscription_account sa where id=#{id}")
|
||||||
|
SubscriptionAccount selectSubscriptionAccount(Integer id);
|
||||||
|
|
||||||
|
@Select("select count(*) from subscription_account where name=#{name}")
|
||||||
|
int countSubscriptionAccountName(String name);
|
||||||
|
|
||||||
|
@Select("select count(*) from subscription_account where upstream_key=#{upstreamKey}")
|
||||||
|
int countSubscriptionAccountKey(String upstreamKey);
|
||||||
|
|
||||||
|
@Update("update subscription_account set name=#{name}, upstream_key=#{upstreamKey}, filter_high_multiplier=#{filterHighMultiplier}, enabled=#{enabled}, updated_at=CURRENT_TIMESTAMP, last_error=null where id=#{id}")
|
||||||
|
void updateSubscriptionAccount(SubscriptionAccount account);
|
||||||
|
|
||||||
|
// last_success_at 仍写 SQLite 的 UTC 文本时间,仅供页面显示;
|
||||||
|
// last_success_epoch 是调度判断用的权威时刻,两者口径不同,不要互相推导。
|
||||||
|
@Update("update subscription_account set last_success_at=CURRENT_TIMESTAMP, last_success_epoch=#{epoch}, last_error=null, updated_at=CURRENT_TIMESTAMP where id=#{id}")
|
||||||
|
void markSubscriptionRefreshSuccess(@Param("id") Integer id, @Param("epoch") long epoch);
|
||||||
|
|
||||||
|
@Update("update subscription_account set last_error=#{error}, updated_at=CURRENT_TIMESTAMP where id=#{id}")
|
||||||
|
void markSubscriptionRefreshFailure(@Param("id") Integer id, @Param("error") String error);
|
||||||
|
|
||||||
|
/** 排定下一次刷新时刻;失败重试与次日分槽都走这里。 */
|
||||||
|
@Update("update subscription_account set next_refresh_at=#{nextRefreshAt}, updated_at=CURRENT_TIMESTAMP where id=#{id}")
|
||||||
|
void updateNextRefreshAt(@Param("id") Integer id, @Param("nextRefreshAt") long nextRefreshAt);
|
||||||
|
|
||||||
|
@Delete("delete from subscription_account where id=#{id}")
|
||||||
|
void deleteSubscriptionAccount(Integer id);
|
||||||
|
|
||||||
|
@Insert("insert into sub_bind (key, user, subscription_account_id, user_id) values (#{key}, #{user}, #{subscriptionAccountId}, (select id from User where username=#{user}))")
|
||||||
|
void insertSubBind(SubBind subBind);
|
||||||
|
|
||||||
|
@Select("select sb.key, coalesce(u.username, sb.user) as user, sb.user_id as userId, sb.subscription_account_id as subscriptionAccountId, sa.name as subscriptionAccountName, sa.enabled as subscriptionAccountEnabled, sa.filter_high_multiplier as filterHighMultiplier from sub_bind sb left join User u on u.id=sb.user_id left join subscription_account sa on sa.id=sb.subscription_account_id order by sb.user")
|
||||||
|
ArrayList<SubBind> selectAllSubBind();
|
||||||
|
|
||||||
|
@Select("select sb.key, coalesce(u.username, sb.user) as user, sb.user_id as userId, sb.subscription_account_id as subscriptionAccountId, sa.name as subscriptionAccountName, sa.enabled as subscriptionAccountEnabled, sa.filter_high_multiplier as filterHighMultiplier from sub_bind sb left join User u on u.id=sb.user_id left join subscription_account sa on sa.id=sb.subscription_account_id where sb.key=#{key}")
|
||||||
|
SubBind selectSubBind(String key);
|
||||||
|
|
||||||
|
@Select("select count(key) from sub_bind where key=#{key}")
|
||||||
|
boolean selectSubBindExist(String key);
|
||||||
|
|
||||||
|
@Select("select count(*) from sub_bind where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
|
||||||
|
int countSubBindByUser(String user);
|
||||||
|
|
||||||
|
@Update("update sub_bind set subscription_account_id=#{accountId} where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
|
||||||
|
int updateSubBindAccount(@Param("user") String user, @Param("accountId") Integer accountId);
|
||||||
|
|
||||||
|
@Update("update sub_bind set key=#{key} where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
|
||||||
|
int updateSubBindKey(@Param("user") String user, @Param("key") String key);
|
||||||
|
|
||||||
|
@Delete("delete from sub_bind where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
|
||||||
|
void deleteSubBind(String user);
|
||||||
|
|
||||||
|
@Insert("insert into sub_update_record (user, ip, UA, time, location, user_id) values (#{user}, #{ip}, #{UA}, #{time}, #{location}, coalesce(#{userId}, (select id from User where username=#{user})))")
|
||||||
|
@Options(useGeneratedKeys = true, keyProperty = "id")
|
||||||
|
void insertSubUpdateRecord(SubUpdateRecord subUpdateRecord);
|
||||||
|
|
||||||
|
@Select("select id,user,ip,UA,time,location,user_id as userId from sub_update_record order by time desc")
|
||||||
|
ArrayList<SubUpdateRecord> selectAllSubUpdateRecord();
|
||||||
|
|
||||||
|
@Delete("delete from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
|
||||||
|
void deleteSubUpdateRecord(String user);
|
||||||
|
|
||||||
|
@Select("select count(user) from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user})")
|
||||||
|
Integer selectUpdateRecordCount(String user);
|
||||||
|
|
||||||
|
@Select("select id from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user}) order by time asc,id asc limit 1")
|
||||||
|
Integer selectMinUpdateRecordId(String user);
|
||||||
|
|
||||||
|
/** 后台补齐归属地:只更新该用户最近一条记录,避免覆盖更早的历史记录。 */
|
||||||
|
@Update("update sub_update_record set location=#{location} where id=(select max(id) from sub_update_record where user_id=(select id from User where username=#{user}) or (user_id is null and user=#{user}))")
|
||||||
|
void updateLatestSubUpdateRecordLocation(@Param("user") String user, @Param("location") String location);
|
||||||
|
|
||||||
|
@Update("update sub_update_record set location=#{location} where id=#{id}")
|
||||||
|
void updateSubUpdateRecordLocation(@Param("id") int id, @Param("location") String location);
|
||||||
|
|
||||||
|
@Update("update sub_bind set user=#{newName} where user_id=#{userId} or (user_id is null and user=#{oldName})")
|
||||||
|
void renameBoundUser(@Param("userId") int userId, @Param("oldName") String oldName,
|
||||||
|
@Param("newName") String newName);
|
||||||
|
|
||||||
|
@Delete("delete from sub_update_record where id=#{id}")
|
||||||
|
void deleteSubUpdateRecordById(int id);
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.SubscriptionEvent;
|
||||||
|
import com.lion.lionwebsite.Domain.SubUpdateRecord;
|
||||||
|
import com.lion.lionwebsite.Domain.SubscriptionEventBinding;
|
||||||
|
import org.apache.ibatis.annotations.*;
|
||||||
|
|
||||||
|
@Mapper
|
||||||
|
public interface SubscriptionEventMapper {
|
||||||
|
/** Reserve SQLite's writer before the receipt read; a deferred read-to-write upgrade can fail immediately. */
|
||||||
|
@Update("update subscription_event_receipt set ingested_at=ingested_at where 0")
|
||||||
|
void reserveWriter();
|
||||||
|
|
||||||
|
@Select("select fingerprint from subscription_event_receipt where event_id=#{eventId}")
|
||||||
|
String fingerprint(String eventId);
|
||||||
|
|
||||||
|
@Insert("insert into subscription_event_receipt(event_id,key_hash,client,occurred_at,source,ingested_at,fingerprint,outcome) values(#{event.eventId},#{event.keyHash},#{event.client},#{event.occurredAt},#{source},#{ingestedAt},#{fingerprint},#{outcome})")
|
||||||
|
void insertReceipt(@Param("event") SubscriptionEvent event, @Param("source") String source,
|
||||||
|
@Param("ingestedAt") long ingestedAt, @Param("fingerprint") String fingerprint,
|
||||||
|
@Param("outcome") String outcome);
|
||||||
|
|
||||||
|
@Select("select key_hash as keyHash,user_id as userId,user_name as userName from subscription_event_binding where key_hash=#{hash}")
|
||||||
|
SubscriptionEventBinding binding(String hash);
|
||||||
|
|
||||||
|
@Insert("insert into subscription_event_binding(key_hash,user_id,user_name,first_seen_at,last_seen_at) values(#{hash},#{userId},#{userName},#{now},#{now}) on conflict(key_hash) do update set user_name=excluded.user_name,last_seen_at=excluded.last_seen_at where subscription_event_binding.user_id IS excluded.user_id")
|
||||||
|
void rememberBinding(@Param("hash") String hash, @Param("userId") Integer userId,
|
||||||
|
@Param("userName") String userName, @Param("now") long now);
|
||||||
|
|
||||||
|
@Insert("insert into sub_update_record(user,ip,UA,time,location,user_id) values(#{user},#{ip},#{UA},#{time},#{location},#{userId})")
|
||||||
|
@Options(useGeneratedKeys = true, keyProperty = "id")
|
||||||
|
void insertRecord(SubUpdateRecord record);
|
||||||
|
|
||||||
|
@Delete("delete from sub_update_record where (user_id=#{userId} or (user_id is null and user=#{userName})) and id not in (select id from sub_update_record where user_id=#{userId} or (user_id is null and user=#{userName}) order by time desc,id desc limit 10)")
|
||||||
|
void pruneRecords(@Param("userId") Integer userId, @Param("userName") String userName);
|
||||||
|
}
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.TaskOperation;
|
||||||
|
import org.apache.ibatis.annotations.*;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 任务操作记录的持久化入口(主库)。
|
||||||
|
*
|
||||||
|
* <p>所有写操作都在调用方的事务里执行;这里的注解只描述 SQL,不包含业务判断。
|
||||||
|
* 主键为 gid,保证「同一 gid 只能有一条有效操作意图」。
|
||||||
|
*/
|
||||||
|
@Mapper
|
||||||
|
public interface TaskOperationRepository {
|
||||||
|
|
||||||
|
@Select("select * from task_operation where gid=#{gid}")
|
||||||
|
TaskOperation selectByGid(int gid);
|
||||||
|
|
||||||
|
/** 生成该 gid 的稳定下载 operationId:只有还没有记录时才写入。 */
|
||||||
|
@Insert("insert or ignore into task_operation (gid, operationId, deleteOperationId, state, lastRevision," +
|
||||||
|
" lastReportId, updateTime) values (#{gid}, #{operationId}, null, #{state}, 0, null, #{updateTime})")
|
||||||
|
int insertIfAbsent(@Param("gid") int gid, @Param("operationId") String operationId,
|
||||||
|
@Param("state") String state, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
@Insert("insert into task_operation (gid, operationId, deleteOperationId, state, lastRevision, lastReportId, updateTime)" +
|
||||||
|
" values (#{gid}, #{operationId}, #{deleteOperationId}, #{state}, #{lastRevision}, #{lastReportId}, #{updateTime})")
|
||||||
|
void insert(TaskOperation operation);
|
||||||
|
|
||||||
|
/** 推进创建意图的状态(RESERVED → SUBMITTING → SUBMITTED/UNKNOWN)。 */
|
||||||
|
@Update("update task_operation set state=#{state}, updateTime=#{updateTime} where gid=#{gid}")
|
||||||
|
int updateState(@Param("gid") int gid, @Param("state") String state, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 带前置条件的创建状态推进:只有当前状态与 operationId 都匹配时才写入。
|
||||||
|
*
|
||||||
|
* <p>这是「提交前先确认这一代意图仍然有效」的 CAS:若期间被删除/重建,
|
||||||
|
* 返回值 0,调用方必须停下,绝不能再产生上游副作用。
|
||||||
|
*/
|
||||||
|
@Update("update task_operation set state=#{state}, updateTime=#{updateTime}" +
|
||||||
|
" where gid=#{gid} and operationId=#{operationId} and state=#{expectedState}")
|
||||||
|
int compareAndSetCreateState(@Param("gid") int gid, @Param("operationId") String operationId,
|
||||||
|
@Param("expectedState") String expectedState, @Param("state") String state,
|
||||||
|
@Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 只按当前删除意图 ID 匹配的删除完成:期间若已被删除/重建,则不改动新世代记录。
|
||||||
|
*/
|
||||||
|
@Update("update task_operation set deleteOperationId=null, state=#{state}, updateTime=#{updateTime}" +
|
||||||
|
" where gid=#{gid} and deleteOperationId=#{deleteOperationId}")
|
||||||
|
int completeDeleteIfSameGeneration(@Param("gid") int gid, @Param("deleteOperationId") String deleteOperationId,
|
||||||
|
@Param("state") String state, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 删除「本次创建意图」的预占记录:必须同时匹配 gid 与 operationId,且状态仍是
|
||||||
|
* RESERVED/SUBMITTING。
|
||||||
|
*
|
||||||
|
* <p>三重匹配保证释放流程只能清掉自己那一代:期间被删除/重建(新 operationId 或墓碑)
|
||||||
|
* 的记录一律不动,避免把新世代抹掉。
|
||||||
|
*/
|
||||||
|
@Delete("delete from task_operation where gid=#{gid} and operationId=#{operationId}" +
|
||||||
|
" and state in ('RESERVED','SUBMITTING')")
|
||||||
|
int deleteReservationIfSameGeneration(@Param("gid") int gid, @Param("operationId") String operationId);
|
||||||
|
|
||||||
|
/** 记录/替换当前下载 operationId 并推进状态。 */
|
||||||
|
@Update("update task_operation set operationId=#{operationId}, state=#{state}, updateTime=#{updateTime} where gid=#{gid}")
|
||||||
|
int updateOperationId(@Param("gid") int gid, @Param("operationId") String operationId,
|
||||||
|
@Param("state") String state, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 给「有记录但缺 operationId」的历史任务补一个稳定 operationId。
|
||||||
|
* 只在确实缺失时写入,避免覆盖并发创建出来的新世代。
|
||||||
|
*/
|
||||||
|
@Update("update task_operation set operationId=#{operationId}, updateTime=#{updateTime}" +
|
||||||
|
" where gid=#{gid} and operationId is null")
|
||||||
|
int assignOperationIdIfMissing(@Param("gid") int gid, @Param("operationId") String operationId,
|
||||||
|
@Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 墓碑复活:删除完成后的 gid 允许用全新 operationId 重新创建。
|
||||||
|
* revision 与回执一并清零,旧操作的迟到上报会因 operationId 不匹配被丢弃。
|
||||||
|
*/
|
||||||
|
@Update("update task_operation set operationId=#{operationId}, deleteOperationId=null, state=#{state}," +
|
||||||
|
" lastRevision=0, lastReportId=null, updateTime=#{updateTime} where gid=#{gid}")
|
||||||
|
int reactivate(@Param("gid") int gid, @Param("operationId") String operationId,
|
||||||
|
@Param("state") String state, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/** 记录删除意图与其稳定的删除操作 ID(重试必须复用同一个 ID)。 */
|
||||||
|
@Update("update task_operation set deleteOperationId=#{deleteOperationId}, state=#{state}, updateTime=#{updateTime} where gid=#{gid}")
|
||||||
|
int updateDeleteIntent(@Param("gid") int gid, @Param("deleteOperationId") String deleteOperationId,
|
||||||
|
@Param("state") String state, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 写入状态上报回执(revision + reportId)。
|
||||||
|
*
|
||||||
|
* <p>回执与任务行更新在同一个事务里提交,提交成功之后才回 ACK;
|
||||||
|
* 因此节点重复投递同一份上报时,只能看到已经提交过的回执,不会重复落库或重复通知。
|
||||||
|
*/
|
||||||
|
@Update("update task_operation set lastRevision=#{revision}, lastReportId=#{reportId}, updateTime=#{updateTime}" +
|
||||||
|
" where gid=#{gid}")
|
||||||
|
int updateReportReceipt(@Param("gid") int gid, @Param("revision") long revision,
|
||||||
|
@Param("reportId") String reportId, @Param("updateTime") long updateTime);
|
||||||
|
|
||||||
|
@Select("select * from task_operation where state=#{state}")
|
||||||
|
java.util.List<TaskOperation> selectByState(String state);
|
||||||
|
|
||||||
|
@Delete("delete from task_operation where gid=#{gid}")
|
||||||
|
int deleteByGid(int gid);
|
||||||
|
}
|
||||||
+15
-2
@@ -1,4 +1,4 @@
|
|||||||
package com.lion.lionwebsite.Dao;
|
package com.lion.lionwebsite.Dao.normal;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.User;
|
import com.lion.lionwebsite.Domain.User;
|
||||||
import org.apache.ibatis.annotations.*;
|
import org.apache.ibatis.annotations.*;
|
||||||
@@ -11,10 +11,23 @@ public interface UserMapper {
|
|||||||
@Select("select * from User where AuthCode=#{AuthCode}")
|
@Select("select * from User where AuthCode=#{AuthCode}")
|
||||||
User selectUserByAuthCode(String AuthCode);
|
User selectUserByAuthCode(String AuthCode);
|
||||||
|
|
||||||
|
@Select("select * from User where username=#{username}")
|
||||||
|
User selectUserByUsername(String username);
|
||||||
|
|
||||||
|
@Select("select * from User where id=#{id}")
|
||||||
|
User selectUserById(int id);
|
||||||
|
|
||||||
@Select("select AuthCode from User")
|
@Select("select AuthCode from User")
|
||||||
String[] selectAllAuthCode();
|
String[] selectAllAuthCode();
|
||||||
|
|
||||||
@Select("select AuthCode from User")
|
/**
|
||||||
|
* 仅返回仍启用的授权码。
|
||||||
|
*
|
||||||
|
* <p>此前这条 SQL 与 {@link #selectAllAuthCode()} 完全相同(都没有 isEnable 条件),
|
||||||
|
* 于是「停用用户」后刷新授权码集合依旧把该码放行,isEnable 形同虚设。
|
||||||
|
* isEnable 为空按 DDL 默认值 true 处理,避免历史行被锁死。
|
||||||
|
*/
|
||||||
|
@Select("select AuthCode from User where isEnable is null or isEnable = 1")
|
||||||
String[] selectEnableAuthCode();
|
String[] selectEnableAuthCode();
|
||||||
|
|
||||||
@Select("select * from User")
|
@Select("select * from User")
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Domain;
|
|
||||||
|
|
||||||
import lombok.AllArgsConstructor;
|
|
||||||
import lombok.Data;
|
|
||||||
import lombok.NoArgsConstructor;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
@AllArgsConstructor
|
|
||||||
@NoArgsConstructor
|
|
||||||
public class CRC32Cache {
|
|
||||||
String path;
|
|
||||||
long lastModifyTime;
|
|
||||||
String crc32;
|
|
||||||
}
|
|
||||||
@@ -13,22 +13,22 @@ import java.util.Map;
|
|||||||
@Data
|
@Data
|
||||||
public class Gallery {
|
public class Gallery {
|
||||||
@JsonProperty("name")
|
@JsonProperty("name")
|
||||||
private String name; //本子名字
|
private String name; //图片名字
|
||||||
|
|
||||||
@JsonProperty("gid")
|
@JsonProperty("gid")
|
||||||
private int gid; //gid
|
private int gid; //gid
|
||||||
|
|
||||||
@JsonProperty("link")
|
@JsonProperty("link")
|
||||||
private String link; //本子链接
|
private String link; //图片链接
|
||||||
|
|
||||||
@JsonProperty("language")
|
@JsonProperty("language")
|
||||||
private String language; //本子语言
|
private String language; //图片语言
|
||||||
|
|
||||||
@JsonProperty("pages")
|
@JsonProperty("pages")
|
||||||
private int pages; //本子页数
|
private int pages; //图片页数
|
||||||
|
|
||||||
@JsonProperty("status")
|
@JsonProperty("status")
|
||||||
private String status; //本子当前状态
|
private String status; //图片当前状态
|
||||||
|
|
||||||
@JsonIgnore
|
@JsonIgnore
|
||||||
private long fileSize; //文件大小
|
private long fileSize; //文件大小
|
||||||
@@ -48,6 +48,11 @@ public class Gallery {
|
|||||||
@JsonProperty("downloader")
|
@JsonProperty("downloader")
|
||||||
private int downloader; //下载人
|
private int downloader; //下载人
|
||||||
|
|
||||||
|
/** 下载人昵称。仅管理员查询时填充,普通用户拿不到,避免暴露他人信息。 */
|
||||||
|
@JsonProperty("downloaderName")
|
||||||
|
@JsonInclude(JsonInclude.Include.NON_EMPTY)
|
||||||
|
private String downloaderName;
|
||||||
|
|
||||||
@JsonProperty("collector")
|
@JsonProperty("collector")
|
||||||
@JsonInclude(JsonInclude.Include.NON_EMPTY)
|
@JsonInclude(JsonInclude.Include.NON_EMPTY)
|
||||||
private String collector; //收藏人
|
private String collector; //收藏人
|
||||||
@@ -64,7 +69,11 @@ public class Gallery {
|
|||||||
@JsonInclude(JsonInclude.Include.NON_NULL)
|
@JsonInclude(JsonInclude.Include.NON_NULL)
|
||||||
private Map<String, String> availableResolution; //可选分辨率
|
private Map<String, String> availableResolution; //可选分辨率
|
||||||
|
|
||||||
@JsonProperty("mode")
|
@JsonProperty("isDownload")
|
||||||
@JsonInclude(JsonInclude.Include.NON_EMPTY)
|
@JsonInclude(JsonInclude.Include.NON_EMPTY)
|
||||||
private byte mode; //rd 可读可下载 r可读 d可下载
|
private boolean is_download; //0 仅查看 1 仅下载源文件 2 仅存储预览图 3 全部
|
||||||
|
|
||||||
|
@JsonProperty("thumb_link")
|
||||||
|
@JsonInclude(JsonInclude.Include.NON_EMPTY)
|
||||||
|
private String thumb_link;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,9 @@ import lombok.Data;
|
|||||||
@Data
|
@Data
|
||||||
public class GalleryForQuery {
|
public class GalleryForQuery {
|
||||||
|
|
||||||
|
@JsonProperty("gid")
|
||||||
|
String gid;
|
||||||
|
|
||||||
@JsonProperty("name")
|
@JsonProperty("name")
|
||||||
String name;
|
String name;
|
||||||
|
|
||||||
|
|||||||
@@ -6,33 +6,20 @@ import lombok.Data;
|
|||||||
|
|
||||||
@Data
|
@Data
|
||||||
public class GalleryTask {
|
public class GalleryTask {
|
||||||
|
|
||||||
public static byte DOWNLOADING = 1;
|
public static byte DOWNLOADING = 1;
|
||||||
|
|
||||||
public static byte DOWNLOAD_COMPLETE = 2;
|
public static byte DOWNLOAD_COMPLETE = 2;
|
||||||
|
public static byte COMPRESSING = 3;
|
||||||
public static byte DOWNLOAD_QUEUED = 3;
|
|
||||||
|
|
||||||
public static byte COMPRESS_COMPLETE = 4;
|
public static byte COMPRESS_COMPLETE = 4;
|
||||||
|
|
||||||
|
|
||||||
public static byte DOWNLOAD_ALL = 3;
|
|
||||||
public static byte DOWNLOAD_PREVIEW = 2;
|
|
||||||
public static byte DOWNLOAD_SOURCE = 1;
|
|
||||||
|
|
||||||
@JsonInclude(JsonInclude.Include.NON_NULL)
|
@JsonInclude(JsonInclude.Include.NON_NULL)
|
||||||
private String name;
|
private String name;
|
||||||
|
|
||||||
private int gid;
|
private int gid;
|
||||||
|
|
||||||
private int pages;
|
|
||||||
|
|
||||||
private byte status;
|
private byte status;
|
||||||
|
|
||||||
private int proceeding;
|
private int proceeding;
|
||||||
|
|
||||||
private byte type;
|
|
||||||
|
|
||||||
@JsonIgnore
|
@JsonIgnore
|
||||||
private String path;
|
private String path;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,14 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
public class GidToKey {
|
||||||
|
String gid;
|
||||||
|
String key;
|
||||||
|
int pages;
|
||||||
|
|
||||||
|
public String toUrl(){
|
||||||
|
return "https://exhentai.org/g/" + gid + "/" + key;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
public class ImageKeyCache {
|
||||||
|
String gid;
|
||||||
|
int page;
|
||||||
|
String imgkey;
|
||||||
|
}
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Domain;
|
|
||||||
|
|
||||||
import lombok.AllArgsConstructor;
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
@AllArgsConstructor
|
|
||||||
public class MaskDomain {
|
|
||||||
String raw;
|
|
||||||
String mask;
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Domain;
|
|
||||||
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
import java.util.Date;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
public class ShareFile {
|
|
||||||
String ShareCode;
|
|
||||||
String FilePath;
|
|
||||||
Date ExpireTime;
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.AllArgsConstructor;
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@AllArgsConstructor
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class SubBind {
|
||||||
|
String key;
|
||||||
|
String user;
|
||||||
|
Integer subscriptionAccountId;
|
||||||
|
String subscriptionAccountName;
|
||||||
|
boolean subscriptionAccountEnabled;
|
||||||
|
boolean filterHighMultiplier;
|
||||||
|
Integer userId;
|
||||||
|
|
||||||
|
public SubBind(String key, String user, Integer accountId, String accountName,
|
||||||
|
boolean accountEnabled, boolean filterHighMultiplier) {
|
||||||
|
this(key, user, accountId, accountName, accountEnabled, filterHighMultiplier, null);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.AllArgsConstructor;
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
import java.util.Date;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@AllArgsConstructor
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class SubUpdateRecord {
|
||||||
|
int id;
|
||||||
|
String user;
|
||||||
|
String ip;
|
||||||
|
String UA;
|
||||||
|
Date time;
|
||||||
|
String location;
|
||||||
|
Integer userId;
|
||||||
|
|
||||||
|
public SubUpdateRecord(int id, String user, String ip, String UA, Date time, String location) {
|
||||||
|
this(id, user, ip, UA, time, location, null);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
import java.util.Date;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class SubscriptionAccount {
|
||||||
|
private Integer id;
|
||||||
|
private String name;
|
||||||
|
private String upstreamKey;
|
||||||
|
private boolean filterHighMultiplier;
|
||||||
|
private boolean enabled;
|
||||||
|
private Date lastSuccessAt;
|
||||||
|
private String lastError;
|
||||||
|
private Date createdAt;
|
||||||
|
private Date updatedAt;
|
||||||
|
private Integer boundUserCount;
|
||||||
|
private String v2Url;
|
||||||
|
private String clashUrl;
|
||||||
|
/**
|
||||||
|
* 下一次应刷新的时刻(Epoch 毫秒),由分散调度器排定。
|
||||||
|
* 刻意用整数而非 DATETIME:SQLite 文本时间戳会被 JDBC 按本地时区解释,
|
||||||
|
* 实测偏差 8 小时,不能用于时间比较。
|
||||||
|
*/
|
||||||
|
private Long nextRefreshAt;
|
||||||
|
/** 最近一次成功刷新的真实时刻(Epoch 毫秒),用于「24 小时内必刷一次」的判断与陈旧告警。 */
|
||||||
|
private Long lastSuccessEpoch;
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.AllArgsConstructor;
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
/** Validated, authenticated event. Never contains the public subscription key. */
|
||||||
|
@Data
|
||||||
|
@AllArgsConstructor
|
||||||
|
public class SubscriptionEvent {
|
||||||
|
private String eventId;
|
||||||
|
private String keyHash;
|
||||||
|
private String client;
|
||||||
|
private String clientIp;
|
||||||
|
private String userAgent;
|
||||||
|
private long occurredAt;
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
public class SubscriptionEventBinding {
|
||||||
|
private String keyHash;
|
||||||
|
private Integer userId;
|
||||||
|
private String userName;
|
||||||
|
}
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Domain;
|
|
||||||
|
|
||||||
import lombok.AllArgsConstructor;
|
|
||||||
import lombok.Data;
|
|
||||||
import lombok.NoArgsConstructor;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
@NoArgsConstructor
|
|
||||||
@AllArgsConstructor
|
|
||||||
public class Tag {
|
|
||||||
int id;
|
|
||||||
String tag;
|
|
||||||
int usage;
|
|
||||||
}
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Domain;
|
|
||||||
|
|
||||||
import lombok.Data;
|
|
||||||
import lombok.NoArgsConstructor;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
@NoArgsConstructor
|
|
||||||
public class TagMark {
|
|
||||||
int gid;
|
|
||||||
int tid;
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
package com.lion.lionwebsite.Domain;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 单个 gid 的下载生命周期操作记录(协议 v2 的持久化意图)。
|
||||||
|
*
|
||||||
|
* <p>这张表是「先持久化意图、再产生外部副作用」的落点:
|
||||||
|
* <ul>
|
||||||
|
* <li>CREATE 意图先于上游 POST 落库,用来保证同一 gid 只有一个未完成意图;</li>
|
||||||
|
* <li>提交结果不确定时只能停在 {@link #STATE_UNKNOWN},绝不自动重复上游 POST;</li>
|
||||||
|
* <li>DELETE 意图先于节点删除落库,节点确认后(或文件已不存在)才在同一事务里删任务行并留墓碑。</li>
|
||||||
|
* </ul>
|
||||||
|
*
|
||||||
|
* <p>旧任务行没有对应操作记录,查询时按 gid 惰性补建(见 GalleryTaskLifecycleService)。
|
||||||
|
*/
|
||||||
|
@Data
|
||||||
|
public class TaskOperation {
|
||||||
|
|
||||||
|
/** 已登记创建意图,尚未向上游提交。 */
|
||||||
|
public static final String STATE_RESERVED = "RESERVED";
|
||||||
|
/** 正在向上游提交;此时不可再发第二次 POST。 */
|
||||||
|
public static final String STATE_SUBMITTING = "SUBMITTING";
|
||||||
|
/** 上游确认已接受下载请求。 */
|
||||||
|
public static final String STATE_SUBMITTED = "SUBMITTED";
|
||||||
|
/**
|
||||||
|
* 提交结果未知:超时、断连或在 SUBMITTING 期间重启。
|
||||||
|
* 上游不支持幂等,因此这里只能等对账(reconcile),不能自动重发。
|
||||||
|
*/
|
||||||
|
public static final String STATE_UNKNOWN = "UNKNOWN";
|
||||||
|
/** 已登记删除意图,等待节点确认。 */
|
||||||
|
public static final String STATE_DELETE_PENDING = "DELETE_PENDING";
|
||||||
|
/** 节点已确认删除,任务行也随之删除;本行作为墓碑保留,用于停止迟到状态的重放。 */
|
||||||
|
public static final String STATE_DELETED = "DELETED";
|
||||||
|
|
||||||
|
private int gid;
|
||||||
|
|
||||||
|
/** 当前有效的下载操作 ID;同一 gid 的 DOWNLOAD 重试必须复用它。 */
|
||||||
|
private String operationId;
|
||||||
|
|
||||||
|
/** 独立于下载操作 ID 的删除操作 ID,删除重试复用它。 */
|
||||||
|
private String deleteOperationId;
|
||||||
|
|
||||||
|
private String state;
|
||||||
|
|
||||||
|
/** 最近一次应用的状态上报 revision;用于丢弃乱序/重放的旧上报。 */
|
||||||
|
private long lastRevision;
|
||||||
|
|
||||||
|
/** 节点已确认收到的最新上报 ID;用于重复 ACK 的幂等判断。 */
|
||||||
|
private String lastReportId;
|
||||||
|
|
||||||
|
private long updateTime;
|
||||||
|
}
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
package com.lion.lionwebsite.Exception;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 图片索引构建失败。
|
||||||
|
*
|
||||||
|
* <p>调用方需要区分三类结果,才能给出正确的用户可见响应:
|
||||||
|
* 链接本身有问题({@link Reason#INVALID_LINK},属于请求错误)、
|
||||||
|
* 画廊已不存在({@link Reason#UNAVAILABLE},重试无意义)、
|
||||||
|
* 上游或数据库暂时不可用({@link Reason#TRANSIENT},可以重试)。
|
||||||
|
*/
|
||||||
|
public class GalleryIndexException extends Exception {
|
||||||
|
|
||||||
|
public enum Reason {
|
||||||
|
/** 链接无法解析出 gid 或 key。 */
|
||||||
|
INVALID_LINK,
|
||||||
|
/** 上游明确表示画廊不存在(下架/删除/解析不到页 key)。 */
|
||||||
|
UNAVAILABLE,
|
||||||
|
/** 上游返回的页 key 无法自洽(gid 不匹配、页码重复/跳号、imgkey 为空)。 */
|
||||||
|
MALFORMED,
|
||||||
|
/** 上游波动或索引写入失败,重试可能成功。 */
|
||||||
|
TRANSIENT
|
||||||
|
}
|
||||||
|
|
||||||
|
private final Reason reason;
|
||||||
|
|
||||||
|
public GalleryIndexException(Reason reason, String message) {
|
||||||
|
super(message);
|
||||||
|
this.reason = reason;
|
||||||
|
}
|
||||||
|
|
||||||
|
public GalleryIndexException(Reason reason, String message, Throwable cause) {
|
||||||
|
super(message, cause);
|
||||||
|
this.reason = reason;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Reason getReason() {
|
||||||
|
return reason;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 重试没有意义(链接错误或画廊不可用);上游数据畸形与会话级故障仍可重试。 */
|
||||||
|
public boolean isPermanent() {
|
||||||
|
return reason == Reason.INVALID_LINK || reason == Reason.UNAVAILABLE;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,19 +1,18 @@
|
|||||||
package com.lion.lionwebsite.Filter;
|
package com.lion.lionwebsite.Filter;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Dao.UserMapper;
|
import com.lion.lionwebsite.Dao.normal.UserMapper;
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
import com.lion.lionwebsite.Util.CustomUtil;
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import jakarta.servlet.*;
|
import jakarta.servlet.*;
|
||||||
import jakarta.servlet.annotation.WebFilter;
|
import jakarta.servlet.annotation.WebFilter;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
|
|
||||||
@WebFilter(filterName = "AccessFilter", urlPatterns = {"/validate"})
|
@WebFilter(filterName = "AccessFilter", urlPatterns = {"/validate"})
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class AccessFilter implements Filter {
|
public class AccessFilter implements Filter {
|
||||||
|
|
||||||
@Resource
|
final UserMapper userMapper;
|
||||||
UserMapper userMapper;
|
|
||||||
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
|
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
|
||||||
|
|||||||
@@ -4,41 +4,17 @@ import jakarta.servlet.*;
|
|||||||
import jakarta.servlet.annotation.WebFilter;
|
import jakarta.servlet.annotation.WebFilter;
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
import org.springframework.scheduling.annotation.Scheduled;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
|
||||||
import java.io.FileWriter;
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.util.Calendar;
|
import java.time.LocalTime;
|
||||||
|
import java.time.format.DateTimeFormatter;
|
||||||
|
|
||||||
@WebFilter(filterName = "AdaptorFilter", urlPatterns = {"/", "/personal/"})
|
@WebFilter(filterName = "AdaptorFilter", urlPatterns = {"/", "/personal/"})
|
||||||
|
@Slf4j
|
||||||
public class AdaptorFilter implements Filter {
|
public class AdaptorFilter implements Filter {
|
||||||
|
|
||||||
FileWriter writer;
|
private static final DateTimeFormatter TIME_FORMATTER = DateTimeFormatter.ofPattern("HH:mm:ss");
|
||||||
|
|
||||||
AdaptorFilter(){
|
|
||||||
Calendar calendar = Calendar.getInstance();
|
|
||||||
try {
|
|
||||||
writer = new FileWriter(String.format("log/AccessLog_%s-%s-%s.log",
|
|
||||||
calendar.get(Calendar.YEAR), calendar.get(Calendar.MONTH) + 1, calendar.get(Calendar.DAY_OF_MONTH)));
|
|
||||||
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Scheduled(cron = "0 0 0 * * *")
|
|
||||||
void changeDate(){
|
|
||||||
Calendar calendar = Calendar.getInstance();
|
|
||||||
try {
|
|
||||||
if(writer != null)
|
|
||||||
writer.close();
|
|
||||||
|
|
||||||
writer = new FileWriter(String.format("log/AccessLog_%s-%s-%s.log",
|
|
||||||
calendar.get(Calendar.YEAR), calendar.get(Calendar.MONTH) + 1, calendar.get(Calendar.DAY_OF_MONTH)));
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
|
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
|
||||||
@@ -50,17 +26,15 @@ public class AdaptorFilter implements Filter {
|
|||||||
if(UserAgent == null)
|
if(UserAgent == null)
|
||||||
return;
|
return;
|
||||||
|
|
||||||
String AuthCode = request.getParameter("AuthCode") == null ? "null" : request.getParameter("AuthCode");
|
// 这里是全站请求日志,AuthCode 是真正的凭据;只记录是否携带,绝不落明文。
|
||||||
|
boolean hasAuthCode = request.getParameter("AuthCode") != null;
|
||||||
String ServletPath = request.getServletPath();
|
String ServletPath = request.getServletPath();
|
||||||
String ip = request.getHeader("X-Forwarded-For") == null ? request.getRemoteAddr(): request.getHeader("X-Forwarded-For");
|
String ip = request.getHeader("X-Forwarded-For") == null ? request.getRemoteAddr(): request.getHeader("X-Forwarded-For");
|
||||||
Calendar calendar = Calendar.getInstance();
|
String now = LocalTime.now().format(TIME_FORMATTER);
|
||||||
String now = String.format("%s:%s:%s", calendar.get(Calendar.HOUR_OF_DAY), calendar.get(Calendar.MINUTE), calendar.get(Calendar.SECOND));
|
|
||||||
|
|
||||||
|
|
||||||
//日志
|
//日志
|
||||||
System.out.printf("%s ip:%s \tpath:%s \tAuthCode:%s ua:%s\n", now, ip, ServletPath, AuthCode, UserAgent.length() > 61 ? UserAgent.substring(0, 60): UserAgent);
|
log.info("{} ip:{} \tpath:{} \tAuthCode:{} ua:{}", now, ip, ServletPath,
|
||||||
writer.write(String.format("%s ip:%s \tpath:%s \tAuthCode:%s ua:%s\n", now, ip, ServletPath, AuthCode, UserAgent));
|
hasAuthCode ? "present" : "absent", UserAgent.length() > 61 ? UserAgent.substring(0, 60): UserAgent);
|
||||||
writer.flush();
|
|
||||||
|
|
||||||
//如果是验证,则直接跳转
|
//如果是验证,则直接跳转
|
||||||
if(ServletPath.equals("/validate"))
|
if(ServletPath.equals("/validate"))
|
||||||
@@ -68,7 +42,7 @@ public class AdaptorFilter implements Filter {
|
|||||||
|
|
||||||
//如果不是,则根据UA判断是否跳转
|
//如果不是,则根据UA判断是否跳转
|
||||||
else if ((UserAgent.contains("Android") || UserAgent.contains("iPhone")))
|
else if ((UserAgent.contains("Android") || UserAgent.contains("iPhone")))
|
||||||
if (ServletPath.equals("/personal/") && AuthCode.equals("alone"))
|
if (ServletPath.equals("/personal/") && "alone".equals(request.getParameter("AuthCode")))
|
||||||
response.sendRedirect("/mobile?AuthCode=alone");
|
response.sendRedirect("/mobile?AuthCode=alone");
|
||||||
else
|
else
|
||||||
response.sendRedirect("/mobile");
|
response.sendRedirect("/mobile");
|
||||||
|
|||||||
@@ -2,13 +2,40 @@ package com.lion.lionwebsite.Interceptor;
|
|||||||
|
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import jakarta.servlet.http.HttpSession;
|
||||||
import org.springframework.web.servlet.HandlerInterceptor;
|
import org.springframework.web.servlet.HandlerInterceptor;
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 个人管理区的访问闸门:有效会话或合法 AuthCode。
|
||||||
|
*
|
||||||
|
* <p>会话来自机器人签发的登录链接(见 {@code PanelLoginTicket} 与
|
||||||
|
* {@code /personal/login});固定的 {@code alone} 授权码保留给下载器与存储节点推送,
|
||||||
|
* 因此这里两条路径都放行,等那些调用方切换完成后再退役字面量。
|
||||||
|
*/
|
||||||
public class PersonalInterceptor implements HandlerInterceptor {
|
public class PersonalInterceptor implements HandlerInterceptor {
|
||||||
|
|
||||||
|
/** 会话标记:{@code /personal/login} 校验票据后写入。 */
|
||||||
|
public static final String SESSION_ATTRIBUTE = "personalAuthenticated";
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler){
|
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler){
|
||||||
return request.getParameter("AuthCode") != null && request.getParameter("AuthCode").equals("alone");
|
if (authenticatedSession(request))
|
||||||
|
return true;
|
||||||
|
|
||||||
|
String authCode = request.getParameter("AuthCode");
|
||||||
|
if (authCode != null && authCode.equals("alone"))
|
||||||
|
return true;
|
||||||
|
|
||||||
|
// 会话缺失或过期时回 401,前端据此提示「去机器人发 /login」而不是静默失败。
|
||||||
|
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 只认服务端写入的会话标记,不信任客户端可伪造的 Cookie 内容。 */
|
||||||
|
public static boolean authenticatedSession(HttpServletRequest request) {
|
||||||
|
// 已有会话才可能已登录;不调用 getSession(),以免为匿名访问创建空会话。
|
||||||
|
HttpSession session = request.getSession(false);
|
||||||
|
return session != null && Boolean.TRUE.equals(session.getAttribute(SESSION_ATTRIBUTE));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,25 +1,32 @@
|
|||||||
package com.lion.lionwebsite.Interceptor;
|
package com.lion.lionwebsite.Interceptor;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Dao.UserMapper;
|
import com.lion.lionwebsite.Dao.normal.UserMapper;
|
||||||
import jakarta.annotation.Resource;
|
import jakarta.annotation.PostConstruct;
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import org.jetbrains.annotations.NotNull;
|
||||||
import org.springframework.stereotype.Component;
|
import org.springframework.stereotype.Component;
|
||||||
import org.springframework.web.servlet.HandlerInterceptor;
|
import org.springframework.web.servlet.HandlerInterceptor;
|
||||||
|
|
||||||
|
|
||||||
@Component
|
@Component
|
||||||
|
@RequiredArgsConstructor
|
||||||
public class TaskHandlerInterceptor implements HandlerInterceptor {
|
public class TaskHandlerInterceptor implements HandlerInterceptor {
|
||||||
|
|
||||||
@Resource
|
final UserMapper userMapper;
|
||||||
UserMapper userMapper;
|
|
||||||
|
// 由 PostConstruct 加载后仅被读(发布式更新)与被测试线程读取;
|
||||||
|
// volatile 保证 refresh 后其它线程立即看到新数组,避免停用的授权码短暂仍可用。
|
||||||
|
volatile String[] AuthCodes;
|
||||||
|
|
||||||
|
@PostConstruct
|
||||||
|
void init() {
|
||||||
|
AuthCodes = userMapper.selectAllAuthCode();
|
||||||
|
}
|
||||||
|
|
||||||
String[] AuthCodes = null;
|
|
||||||
@Override
|
@Override
|
||||||
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler){
|
public boolean preHandle(HttpServletRequest request, @NotNull HttpServletResponse response, @NotNull Object handler){
|
||||||
if(AuthCodes == null)
|
|
||||||
AuthCodes = userMapper.selectAllAuthCode();
|
|
||||||
|
|
||||||
String auth = request.getParameter("AuthCode");
|
String auth = request.getParameter("AuthCode");
|
||||||
|
|
||||||
if(auth != null)
|
if(auth != null)
|
||||||
|
|||||||
@@ -1,14 +1,20 @@
|
|||||||
package com.lion.lionwebsite;
|
package com.lion.lionwebsite;
|
||||||
|
|
||||||
|
|
||||||
|
import org.mybatis.spring.annotation.MapperScan;
|
||||||
|
import org.mybatis.spring.annotation.MapperScans;
|
||||||
import org.springframework.boot.SpringApplication;
|
import org.springframework.boot.SpringApplication;
|
||||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||||
import org.springframework.boot.web.servlet.ServletComponentScan;
|
import org.springframework.boot.web.server.servlet.context.ServletComponentScan;
|
||||||
import org.springframework.scheduling.annotation.EnableScheduling;
|
import org.springframework.scheduling.annotation.EnableScheduling;
|
||||||
|
|
||||||
@SpringBootApplication
|
@SpringBootApplication
|
||||||
@EnableScheduling
|
@EnableScheduling
|
||||||
@ServletComponentScan(value = "com.lion.lionwebsite.Filter")
|
@ServletComponentScan(value = "com.lion.lionwebsite.Filter")
|
||||||
|
@MapperScans({
|
||||||
|
@MapperScan(basePackages = "com.lion.lionwebsite.Dao.normal", sqlSessionTemplateRef = "sqlSessionTemplate-main", sqlSessionFactoryRef = "sqlSessionFactory-main"),
|
||||||
|
@MapperScan(basePackages = "com.lion.lionwebsite.Dao.cache", sqlSessionTemplateRef = "sqlSessionTemplate-cache", sqlSessionFactoryRef = "sqlSessionFactory-cache")
|
||||||
|
})
|
||||||
public class LionWebsiteApplication {
|
public class LionWebsiteApplication {
|
||||||
|
|
||||||
public static void main(String[] args) {
|
public static void main(String[] args) {
|
||||||
|
|||||||
@@ -0,0 +1,319 @@
|
|||||||
|
package com.lion.lionwebsite.Maintenance;
|
||||||
|
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.sql.Connection;
|
||||||
|
import java.sql.DriverManager;
|
||||||
|
import java.sql.PreparedStatement;
|
||||||
|
import java.sql.ResultSet;
|
||||||
|
import java.sql.SQLException;
|
||||||
|
import java.sql.Statement;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.LinkedHashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 缓存库图片索引的离线体检与修复工具。
|
||||||
|
*
|
||||||
|
* <p>它是独立可执行类(只用 JDBC,不依赖 Spring / MyBatis),因此可以在服务停止后
|
||||||
|
* 直接对数据库文件运行,也可以先对生产库的副本演练。</p>
|
||||||
|
*
|
||||||
|
* <pre>
|
||||||
|
* java -cp sqlite-jdbc.jar GalleryIndexMaintenance.java --db /path/cache.db
|
||||||
|
* java -cp sqlite-jdbc.jar GalleryIndexMaintenance.java --db /path/cache.db --apply
|
||||||
|
* </pre>
|
||||||
|
*
|
||||||
|
* <p>它只做两类写入:删除不完整或有冲突的 gid 整组索引(gidToKey 头部加该 gid 的全部
|
||||||
|
* 页 key),以及删除没有头部记录的孤儿页 key。被删掉的 gid 由下一次在线阅读重新抓取,
|
||||||
|
* 因此不需要猜测保留哪一组 key。工具不建表、不改列、不碰其它业务表。</p>
|
||||||
|
*/
|
||||||
|
public final class GalleryIndexMaintenance {
|
||||||
|
|
||||||
|
/** 生产运行时数据库所在目录;默认拒绝直接写入,必须显式加 --allow-production。 */
|
||||||
|
private static final String PRODUCTION_DIR = "/home/lionwebsite";
|
||||||
|
|
||||||
|
private static final String PLAIN_REPAIR_NOTE =
|
||||||
|
"这些画廊将在下次被访问时重新抓取。";
|
||||||
|
|
||||||
|
public static void main(String[] args) {
|
||||||
|
Map<String, String> options = parseOptions(args);
|
||||||
|
if (options.containsKey("help") || !options.containsKey("db")) {
|
||||||
|
printUsage();
|
||||||
|
System.exit(options.containsKey("help") ? 0 : 2);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
Path database = Path.of(options.get("db")).toAbsolutePath().normalize();
|
||||||
|
boolean apply = options.containsKey("apply");
|
||||||
|
if (!Files.isRegularFile(database)) {
|
||||||
|
System.err.println("数据库不存在: " + database);
|
||||||
|
System.exit(2);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (apply && isProductionPath(database) && !options.containsKey("allow-production")) {
|
||||||
|
System.err.println("拒绝写入生产库路径 " + database
|
||||||
|
+ "。请先在副本上演练;确需在本机生产库执行时显式加 --allow-production,"
|
||||||
|
+ "并确保服务已停止。");
|
||||||
|
System.exit(3);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try (Connection connection = open(database)) {
|
||||||
|
Report before = inspect(connection);
|
||||||
|
System.out.println(before.render(apply ? "修复前" : "体检结果"));
|
||||||
|
|
||||||
|
if (!apply) {
|
||||||
|
System.out.println();
|
||||||
|
System.out.println("dry-run:未做任何写入。确认无误后加 --apply 执行修复。");
|
||||||
|
System.exit(before.isHealthy() ? 0 : 1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (before.isHealthy()) {
|
||||||
|
System.out.println();
|
||||||
|
System.out.println("索引已健康,无需修复。");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
connection.setAutoCommit(false);
|
||||||
|
int removed = repair(connection, before.badGids, before.orphanGids);
|
||||||
|
connection.commit();
|
||||||
|
System.out.println();
|
||||||
|
System.out.println("已删除 " + removed + " 个 gid 的索引整组," + PLAIN_REPAIR_NOTE);
|
||||||
|
|
||||||
|
Report after = inspect(connection);
|
||||||
|
System.out.println();
|
||||||
|
System.out.println(after.render("修复后"));
|
||||||
|
if (!after.isHealthy()) {
|
||||||
|
System.err.println("修复后仍未达到健康状态,请人工检查。");
|
||||||
|
System.exit(4);
|
||||||
|
}
|
||||||
|
} catch (SQLException e) {
|
||||||
|
System.err.println("执行失败: " + e.getMessage());
|
||||||
|
e.printStackTrace(System.err);
|
||||||
|
System.exit(5);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static Connection open(Path database) throws SQLException {
|
||||||
|
Connection connection = DriverManager.getConnection("jdbc:sqlite:" + database);
|
||||||
|
try (Statement statement = connection.createStatement()) {
|
||||||
|
statement.execute("pragma busy_timeout=5000");
|
||||||
|
statement.execute("pragma foreign_keys=off");
|
||||||
|
}
|
||||||
|
return connection;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 是否位于生产运行时目录;工具据此拒绝在未显式确认时写入生产库。 */
|
||||||
|
static boolean isProductionPath(Path database) {
|
||||||
|
return database.startsWith(Path.of(PRODUCTION_DIR).toAbsolutePath().normalize());
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 体检 ----------
|
||||||
|
|
||||||
|
static Report inspect(Connection connection) throws SQLException {
|
||||||
|
Report report = new Report();
|
||||||
|
report.hasHeaderTable = tableExists(connection, "gidToKey");
|
||||||
|
report.hasPageTable = tableExists(connection, "ImageKeyCache");
|
||||||
|
if (!report.hasHeaderTable || !report.hasPageTable)
|
||||||
|
return report;
|
||||||
|
|
||||||
|
report.headerRows = count(connection, "select count(*) from gidToKey");
|
||||||
|
report.pageRows = count(connection, "select count(*) from ImageKeyCache");
|
||||||
|
report.duplicateHeaderGids = count(connection,
|
||||||
|
"select count(*) from (select gid from gidToKey group by gid having count(*) > 1)");
|
||||||
|
report.duplicatePageGroups = count(connection,
|
||||||
|
"select count(*) from (select gid, page from ImageKeyCache group by gid, page having count(*) > 1)");
|
||||||
|
|
||||||
|
try (PreparedStatement statement = connection.prepareStatement(BAD_GID_QUERY);
|
||||||
|
ResultSet rows = statement.executeQuery()) {
|
||||||
|
while (rows.next()) {
|
||||||
|
String gid = rows.getString(1);
|
||||||
|
report.badGids.add(gid);
|
||||||
|
report.badReasons.put(gid, describe(rows));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
try (PreparedStatement statement = connection.prepareStatement(
|
||||||
|
"select distinct gid from ImageKeyCache where gid not in (select gid from gidToKey)");
|
||||||
|
ResultSet rows = statement.executeQuery()) {
|
||||||
|
while (rows.next())
|
||||||
|
report.orphanGids.add(rows.getString(1));
|
||||||
|
}
|
||||||
|
return report;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 一个 gid 触发的全部不健康条件集中在一句里检查,避免多次全表扫描。
|
||||||
|
*
|
||||||
|
* <p>必须按 gid 分组后再判断:重复 gid(同一 gid 有多条头部记录)本身就是要清除的对象,
|
||||||
|
* 而重复行里的某一条可能恰好与页 key 数一致,逐行判断会把它当成健康记录漏掉。
|
||||||
|
*/
|
||||||
|
private static final String BAD_GID_QUERY = """
|
||||||
|
select g.gid as gid,
|
||||||
|
count(*) as header_rows,
|
||||||
|
max(g.pages) as pages,
|
||||||
|
min(g.key) as key,
|
||||||
|
(select count(*) from ImageKeyCache i where i.gid = g.gid) as page_rows,
|
||||||
|
(select count(distinct i.page) from ImageKeyCache i where i.gid = g.gid) as distinct_pages,
|
||||||
|
(select coalesce(min(i.page), 0) from ImageKeyCache i where i.gid = g.gid) as min_page,
|
||||||
|
(select coalesce(max(i.page), 0) from ImageKeyCache i where i.gid = g.gid) as max_page,
|
||||||
|
(select count(*) from ImageKeyCache i where i.gid = g.gid
|
||||||
|
and (i.imgkey is null or trim(i.imgkey) = '')) as dirty_keys
|
||||||
|
from gidToKey g
|
||||||
|
group by g.gid
|
||||||
|
having count(*) > 1
|
||||||
|
or max(g.pages) is null or max(g.pages) <= 0
|
||||||
|
or min(g.key) is null or trim(min(g.key)) = ''
|
||||||
|
or (select count(*) from ImageKeyCache i where i.gid = g.gid) <> max(g.pages)
|
||||||
|
or (select count(distinct i.page) from ImageKeyCache i where i.gid = g.gid) <> max(g.pages)
|
||||||
|
or (select coalesce(min(i.page), 0) from ImageKeyCache i where i.gid = g.gid) <> 1
|
||||||
|
or (select coalesce(max(i.page), 0) from ImageKeyCache i where i.gid = g.gid) <> max(g.pages)
|
||||||
|
or (select count(*) from ImageKeyCache i where i.gid = g.gid
|
||||||
|
and (i.imgkey is null or trim(i.imgkey) = '')) > 0
|
||||||
|
order by g.gid
|
||||||
|
""";
|
||||||
|
|
||||||
|
private static String describe(ResultSet row) throws SQLException {
|
||||||
|
List<String> reasons = new ArrayList<>();
|
||||||
|
int headerRows = row.getInt("header_rows");
|
||||||
|
int pages = row.getInt("pages");
|
||||||
|
String key = row.getString("key");
|
||||||
|
int pageRows = row.getInt("page_rows");
|
||||||
|
int distinctPages = row.getInt("distinct_pages");
|
||||||
|
int minPage = row.getInt("min_page");
|
||||||
|
int maxPage = row.getInt("max_page");
|
||||||
|
int dirtyKeys = row.getInt("dirty_keys");
|
||||||
|
if (headerRows > 1) reasons.add("重复头部记录 " + headerRows + " 条");
|
||||||
|
if (key == null || key.isBlank()) reasons.add("key 为空");
|
||||||
|
if (pages <= 0) reasons.add("pages 非法(" + pages + ")");
|
||||||
|
if (pageRows != pages) reasons.add("页 key 行数 " + pageRows + " 与 pages " + pages + " 不符");
|
||||||
|
if (distinctPages != pages) reasons.add("页码去重后 " + distinctPages + " 个");
|
||||||
|
if (minPage != 1 || maxPage != pages) reasons.add("页码区间 " + minPage + "-" + maxPage);
|
||||||
|
if (dirtyKeys > 0) reasons.add("imgkey 为空 " + dirtyKeys + " 行");
|
||||||
|
return String.join("、", reasons);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 修复 ----------
|
||||||
|
|
||||||
|
/** 删除不健康 gid 的整组索引与孤儿页 key,返回被移除索引的 gid 个数。 */
|
||||||
|
static int repair(Connection connection, List<String> badGids, List<String> orphanGids)
|
||||||
|
throws SQLException {
|
||||||
|
int touched = 0;
|
||||||
|
try (PreparedStatement pages = connection.prepareStatement("delete from ImageKeyCache where gid = ?");
|
||||||
|
PreparedStatement header = connection.prepareStatement("delete from gidToKey where gid = ?")) {
|
||||||
|
for (String gid : badGids) {
|
||||||
|
pages.setString(1, gid);
|
||||||
|
pages.executeUpdate();
|
||||||
|
header.setString(1, gid);
|
||||||
|
header.executeUpdate();
|
||||||
|
touched++;
|
||||||
|
}
|
||||||
|
for (String gid : orphanGids) {
|
||||||
|
pages.setString(1, gid);
|
||||||
|
if (pages.executeUpdate() > 0)
|
||||||
|
touched++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return touched;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 基础设施 ----------
|
||||||
|
|
||||||
|
private static boolean tableExists(Connection connection, String table) throws SQLException {
|
||||||
|
try (PreparedStatement statement = connection.prepareStatement(
|
||||||
|
"select count(*) from sqlite_master where type='table' and name=?")) {
|
||||||
|
statement.setString(1, table);
|
||||||
|
try (ResultSet rows = statement.executeQuery()) {
|
||||||
|
return rows.next() && rows.getInt(1) > 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int count(Connection connection, String sql) throws SQLException {
|
||||||
|
try (Statement statement = connection.createStatement();
|
||||||
|
ResultSet rows = statement.executeQuery(sql)) {
|
||||||
|
return rows.next() ? rows.getInt(1) : 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static Map<String, String> parseOptions(String[] args) {
|
||||||
|
Map<String, String> options = new LinkedHashMap<>();
|
||||||
|
for (int i = 0; i < args.length; i++) {
|
||||||
|
String arg = args[i];
|
||||||
|
if (!arg.startsWith("--"))
|
||||||
|
continue;
|
||||||
|
String name = arg.substring(2);
|
||||||
|
if (name.equals("apply") || name.equals("help") || name.equals("allow-production")) {
|
||||||
|
options.put(name, "true");
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (i + 1 >= args.length) {
|
||||||
|
System.err.println("缺少参数值: " + arg);
|
||||||
|
System.exit(2);
|
||||||
|
}
|
||||||
|
options.put(name, args[++i]);
|
||||||
|
}
|
||||||
|
return options;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void printUsage() {
|
||||||
|
System.out.println("""
|
||||||
|
用法: java -cp <sqlite-jdbc.jar> GalleryIndexMaintenance.java --db <cache.db> [--apply] [--allow-production]
|
||||||
|
|
||||||
|
--db <path> 必填,缓存库文件
|
||||||
|
--apply 执行修复;不加则只做 dry-run 体检
|
||||||
|
--allow-production 允许写入 /home/lionwebsite 下的数据库(默认拒绝)
|
||||||
|
|
||||||
|
退出码: 0 健康或修复成功 / 1 存在待修复项(dry-run) / 2 参数错误 /
|
||||||
|
3 拒绝写生产路径 / 4 修复后仍不健康 / 5 执行异常
|
||||||
|
""");
|
||||||
|
}
|
||||||
|
|
||||||
|
private GalleryIndexMaintenance() { }
|
||||||
|
|
||||||
|
/** 一次体检结果。 */
|
||||||
|
static final class Report {
|
||||||
|
boolean hasHeaderTable;
|
||||||
|
boolean hasPageTable;
|
||||||
|
int headerRows;
|
||||||
|
int pageRows;
|
||||||
|
int duplicateHeaderGids;
|
||||||
|
int duplicatePageGroups;
|
||||||
|
final List<String> badGids = new ArrayList<>();
|
||||||
|
final List<String> orphanGids = new ArrayList<>();
|
||||||
|
final Map<String, String> badReasons = new LinkedHashMap<>();
|
||||||
|
|
||||||
|
boolean isHealthy() {
|
||||||
|
return hasHeaderTable && hasPageTable && badGids.isEmpty() && orphanGids.isEmpty();
|
||||||
|
}
|
||||||
|
|
||||||
|
String render(String title) {
|
||||||
|
StringBuilder out = new StringBuilder();
|
||||||
|
out.append("=== 图片索引").append(title).append("(").append(Instant.now()).append(")===\n");
|
||||||
|
if (!hasHeaderTable || !hasPageTable) {
|
||||||
|
out.append("缺少缓存表: gidToKey=").append(hasHeaderTable)
|
||||||
|
.append(" ImageKeyCache=").append(hasPageTable).append('\n');
|
||||||
|
return out.toString();
|
||||||
|
}
|
||||||
|
out.append("gidToKey 行数: ").append(headerRows).append('\n');
|
||||||
|
out.append("ImageKeyCache 行数: ").append(pageRows).append('\n');
|
||||||
|
out.append("重复 gid 组: ").append(duplicateHeaderGids).append('\n');
|
||||||
|
out.append("重复 (gid,page) 组: ").append(duplicatePageGroups).append('\n');
|
||||||
|
out.append("不完整或冲突 gid: ").append(badGids.size()).append('\n');
|
||||||
|
int limit = 20;
|
||||||
|
for (int i = 0; i < badGids.size() && i < limit; i++) {
|
||||||
|
String gid = badGids.get(i);
|
||||||
|
out.append(" - gid=").append(gid).append(' ')
|
||||||
|
.append(badReasons.getOrDefault(gid, "")).append('\n');
|
||||||
|
}
|
||||||
|
if (badGids.size() > limit)
|
||||||
|
out.append(" ... 其余 ").append(badGids.size() - limit).append(" 个略\n");
|
||||||
|
out.append("孤儿页 key 的 gid: ").append(orphanGids.size()).append('\n');
|
||||||
|
out.append(isHealthy() ? "结论: 健康\n" : "结论: 需要修复\n");
|
||||||
|
return out.toString();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,5 +1,8 @@
|
|||||||
package com.lion.lionwebsite.Message;
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
@Data
|
||||||
public class AbstractMessage {
|
public class AbstractMessage {
|
||||||
|
|
||||||
public static final byte DOWNLOAD_POST_MESSAGE = 1;
|
public static final byte DOWNLOAD_POST_MESSAGE = 1;
|
||||||
@@ -10,12 +13,28 @@ public class AbstractMessage {
|
|||||||
|
|
||||||
public static final byte RESPONSE_MESSAGE = 0;
|
public static final byte RESPONSE_MESSAGE = 0;
|
||||||
|
|
||||||
public static final byte UPDATE_GALLERY_MESSAGE = 4;
|
|
||||||
|
|
||||||
public static final byte GALLERY_PAGE_QUERY_MESSAGE = 5;
|
|
||||||
|
|
||||||
public static final byte IDENTITY_MESSAGE = 6;
|
public static final byte IDENTITY_MESSAGE = 6;
|
||||||
public static final byte GALLERY_REQUEST_MESSAGE = 101;
|
|
||||||
|
public static final byte MAINTAIN_MESSAGE = 7;
|
||||||
|
|
||||||
|
public static final byte AVAILABLE_CHECK_MESSAGE = 8;
|
||||||
|
|
||||||
|
public static final byte SUBSCRIPTION_SNAPSHOT_MESSAGE = 9;
|
||||||
|
|
||||||
|
// ---- 协议 v2(可靠任务)。旧类型 0..9 的字段保持不变,不能新增字段。 ----
|
||||||
|
|
||||||
|
/** 能力协商:{@code {protocolVersion: 2, features: ["reliable-task-v2"]}}。 */
|
||||||
|
public static final byte PROTOCOL_HELLO_MESSAGE = 10;
|
||||||
|
|
||||||
|
/** 带稳定 operationId 的任务命令(DOWNLOAD/DELETE)。 */
|
||||||
|
public static final byte TASK_COMMAND_MESSAGE = 11;
|
||||||
|
|
||||||
|
/** 带 reportId + revision 的状态上报,替代旧的无确认上报。 */
|
||||||
|
public static final byte TASK_REPORT_MESSAGE = 12;
|
||||||
|
|
||||||
|
/** 主站提交事务成功后的上报回执,节点据此清理快照。 */
|
||||||
|
public static final byte TASK_REPORT_ACK_MESSAGE = 13;
|
||||||
|
|
||||||
public byte messageType;
|
public byte messageType;
|
||||||
|
|
||||||
public int messageId;
|
public int messageId;
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
public class AvailableCheckMessage extends AbstractMessage{
|
||||||
|
{
|
||||||
|
messageType = AVAILABLE_CHECK_MESSAGE;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,14 +7,5 @@ public class DeleteGalleryMessage extends AbstractMessage{
|
|||||||
{
|
{
|
||||||
messageType = DELETE_GALLERY_MESSAGE;
|
messageType = DELETE_GALLERY_MESSAGE;
|
||||||
}
|
}
|
||||||
|
|
||||||
public static final byte DELETE_ALL = 3;
|
|
||||||
public static final byte DELETE_PREVIEW = 2;
|
|
||||||
public static final byte DELETE_SOURCE = 1;
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
byte deleteType;
|
|
||||||
|
|
||||||
String galleryName;
|
String galleryName;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,21 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Message;
|
|
||||||
|
|
||||||
import com.fasterxml.jackson.annotation.JsonInclude;
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
public class GalleryPageQueryMessage extends AbstractMessage{
|
|
||||||
{
|
|
||||||
messageType = GALLERY_PAGE_QUERY_MESSAGE;
|
|
||||||
}
|
|
||||||
|
|
||||||
@JsonInclude(JsonInclude.Include.NON_NULL)
|
|
||||||
String name;
|
|
||||||
|
|
||||||
int page;
|
|
||||||
|
|
||||||
@JsonInclude(JsonInclude.Include.NON_NULL)
|
|
||||||
String pageName;
|
|
||||||
|
|
||||||
byte result;
|
|
||||||
}
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Message;
|
|
||||||
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
//请求预览/压缩包
|
|
||||||
public class GalleryRequestMessage extends AbstractMessage{
|
|
||||||
|
|
||||||
public static final byte SOURCE = 1;
|
|
||||||
public static final byte PREVIEW = 2;
|
|
||||||
|
|
||||||
public static final byte COMPRESS_SOURCE = 3;
|
|
||||||
|
|
||||||
{
|
|
||||||
messageType = GALLERY_REQUEST_MESSAGE;
|
|
||||||
}
|
|
||||||
|
|
||||||
String galleryName;
|
|
||||||
|
|
||||||
byte type;
|
|
||||||
|
|
||||||
short page;
|
|
||||||
|
|
||||||
short port;
|
|
||||||
}
|
|
||||||
@@ -1,10 +1,15 @@
|
|||||||
package com.lion.lionwebsite.Message;
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.AllArgsConstructor;
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
@Data
|
@Data
|
||||||
|
@AllArgsConstructor
|
||||||
|
@NoArgsConstructor
|
||||||
public class IdentityMessage extends AbstractMessage{
|
public class IdentityMessage extends AbstractMessage{
|
||||||
{
|
{
|
||||||
messageType = IDENTITY_MESSAGE;
|
messageType = IDENTITY_MESSAGE;
|
||||||
}
|
}
|
||||||
|
String identity;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,7 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
public class MaintainMessage extends AbstractMessage{
|
||||||
|
{
|
||||||
|
messageType = MAINTAIN_MESSAGE;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
package com.lion.lionwebsite.Message;
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
import tools.jackson.databind.ObjectMapper;
|
||||||
import io.netty.buffer.ByteBuf;
|
import io.netty.buffer.ByteBuf;
|
||||||
import io.netty.channel.ChannelHandlerContext;
|
import io.netty.channel.ChannelHandlerContext;
|
||||||
import io.netty.handler.codec.ByteToMessageCodec;
|
import io.netty.handler.codec.ByteToMessageCodec;
|
||||||
@@ -39,12 +39,16 @@ public class MessageCodec extends ByteToMessageCodec<AbstractMessage> {
|
|||||||
AbstractMessage abstractMessage = switch (messageType){
|
AbstractMessage abstractMessage = switch (messageType){
|
||||||
case AbstractMessage.DOWNLOAD_POST_MESSAGE -> objectMapper.readValue(metadata, DownloadPostMessage.class);
|
case AbstractMessage.DOWNLOAD_POST_MESSAGE -> objectMapper.readValue(metadata, DownloadPostMessage.class);
|
||||||
case AbstractMessage.DOWNLOAD_STATUS_MESSAGE -> objectMapper.readValue(metadata, DownloadStatusMessage.class);
|
case AbstractMessage.DOWNLOAD_STATUS_MESSAGE -> objectMapper.readValue(metadata, DownloadStatusMessage.class);
|
||||||
case AbstractMessage.GALLERY_REQUEST_MESSAGE -> objectMapper.readValue(metadata, GalleryRequestMessage.class);
|
|
||||||
case AbstractMessage.RESPONSE_MESSAGE -> objectMapper.readValue(metadata, ResponseMessage.class);
|
case AbstractMessage.RESPONSE_MESSAGE -> objectMapper.readValue(metadata, ResponseMessage.class);
|
||||||
case AbstractMessage.UPDATE_GALLERY_MESSAGE -> objectMapper.readValue(metadata, UpdateGalleryMessage.class);
|
|
||||||
case AbstractMessage.DELETE_GALLERY_MESSAGE -> objectMapper.readValue(metadata, DeleteGalleryMessage.class);
|
case AbstractMessage.DELETE_GALLERY_MESSAGE -> objectMapper.readValue(metadata, DeleteGalleryMessage.class);
|
||||||
case AbstractMessage.GALLERY_PAGE_QUERY_MESSAGE -> objectMapper.readValue(metadata, GalleryPageQueryMessage.class);
|
|
||||||
case AbstractMessage.IDENTITY_MESSAGE -> objectMapper.readValue(metadata, IdentityMessage.class);
|
case AbstractMessage.IDENTITY_MESSAGE -> objectMapper.readValue(metadata, IdentityMessage.class);
|
||||||
|
case AbstractMessage.MAINTAIN_MESSAGE -> objectMapper.readValue(metadata, MaintainMessage.class);
|
||||||
|
case AbstractMessage.AVAILABLE_CHECK_MESSAGE -> objectMapper.readValue(metadata, AvailableCheckMessage.class);
|
||||||
|
case AbstractMessage.SUBSCRIPTION_SNAPSHOT_MESSAGE -> objectMapper.readValue(metadata, SubscriptionSnapshotMessage.class);
|
||||||
|
case AbstractMessage.PROTOCOL_HELLO_MESSAGE -> objectMapper.readValue(metadata, ProtocolHelloMessage.class);
|
||||||
|
case AbstractMessage.TASK_COMMAND_MESSAGE -> objectMapper.readValue(metadata, TaskCommandMessage.class);
|
||||||
|
case AbstractMessage.TASK_REPORT_MESSAGE -> objectMapper.readValue(metadata, TaskReportMessage.class);
|
||||||
|
case AbstractMessage.TASK_REPORT_ACK_MESSAGE -> objectMapper.readValue(metadata, TaskReportAckMessage.class);
|
||||||
default -> null;
|
default -> null;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/** 协议 v2 能力协商(类型 10)。双方都以对方声明了 version>=2 为前提才启用新消息。 */
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class ProtocolHelloMessage extends AbstractMessage {
|
||||||
|
{
|
||||||
|
messageType = PROTOCOL_HELLO_MESSAGE;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static final int PROTOCOL_V2 = 2;
|
||||||
|
|
||||||
|
public static final String FEATURE_RELIABLE_TASK_V2 = "reliable-task-v2";
|
||||||
|
|
||||||
|
private int protocolVersion = PROTOCOL_V2;
|
||||||
|
|
||||||
|
private List<String> features = new ArrayList<>(List.of(FEATURE_RELIABLE_TASK_V2));
|
||||||
|
|
||||||
|
/** 对端是否声明了 v2 可靠任务能力。 */
|
||||||
|
public boolean supportsReliableTasks() {
|
||||||
|
return protocolVersion >= PROTOCOL_V2
|
||||||
|
&& features != null && features.contains(FEATURE_RELIABLE_TASK_V2);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class SubscriptionAccountSnapshot {
|
||||||
|
private Integer accountId;
|
||||||
|
private boolean enabled;
|
||||||
|
private boolean filterHighMultiplier;
|
||||||
|
private String v2ContentBase64;
|
||||||
|
private String v2Sha256;
|
||||||
|
private String clashContentBase64;
|
||||||
|
private String clashSha256;
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class SubscriptionBindingSnapshot {
|
||||||
|
private String publicKeySha256;
|
||||||
|
private Integer accountId;
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
import lombok.ToString;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
@ToString(exclude = {"payloadBase64", "signature"})
|
||||||
|
public class SubscriptionSnapshotMessage extends AbstractMessage {
|
||||||
|
{
|
||||||
|
messageType = SUBSCRIPTION_SNAPSHOT_MESSAGE;
|
||||||
|
}
|
||||||
|
|
||||||
|
private int schemaVersion;
|
||||||
|
private String revision;
|
||||||
|
private long generatedAt;
|
||||||
|
private String payloadBase64;
|
||||||
|
private String payloadSha256;
|
||||||
|
private String signature;
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class SubscriptionSnapshotPayload {
|
||||||
|
private int schemaVersion;
|
||||||
|
private List<SubscriptionAccountSnapshot> accounts = new ArrayList<>();
|
||||||
|
private List<SubscriptionBindingSnapshot> bindings = new ArrayList<>();
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 任务命令(类型 11)。
|
||||||
|
*
|
||||||
|
* <p>只有确认对端支持 v2 之后才发送。同一逻辑操作在重试/重连后必须复用同一个
|
||||||
|
* {@link #operationId},节点据此返回原结果而不重复执行副作用。
|
||||||
|
*/
|
||||||
|
@Data
|
||||||
|
public class TaskCommandMessage extends AbstractMessage {
|
||||||
|
{
|
||||||
|
messageType = TASK_COMMAND_MESSAGE;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static final String ACTION_DOWNLOAD = "DOWNLOAD";
|
||||||
|
|
||||||
|
public static final String ACTION_DELETE = "DELETE";
|
||||||
|
|
||||||
|
/** 稳定操作 ID:DOWNLOAD 与 DELETE 各自独立。 */
|
||||||
|
private String operationId;
|
||||||
|
|
||||||
|
private String action;
|
||||||
|
|
||||||
|
private int gid;
|
||||||
|
|
||||||
|
private String name;
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.NoArgsConstructor;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 上报回执(类型 13)。
|
||||||
|
*
|
||||||
|
* <p>只在任务行与回执已经提交之后才发送;重复上报(同一 reportId)也会得到回执,
|
||||||
|
* 让节点停止重放。{@link #result} 0 表示已落库。
|
||||||
|
*/
|
||||||
|
@Data
|
||||||
|
@NoArgsConstructor
|
||||||
|
public class TaskReportAckMessage extends AbstractMessage {
|
||||||
|
{
|
||||||
|
messageType = TASK_REPORT_ACK_MESSAGE;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static final byte RESULT_SUCCESS = 0;
|
||||||
|
|
||||||
|
private String reportId;
|
||||||
|
|
||||||
|
private byte result;
|
||||||
|
|
||||||
|
public TaskReportAckMessage(String reportId, byte result) {
|
||||||
|
this.reportId = reportId;
|
||||||
|
this.result = result;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
package com.lion.lionwebsite.Message;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Domain.GalleryTask;
|
||||||
|
import lombok.Data;
|
||||||
|
|
||||||
|
import java.util.LinkedHashMap;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 节点状态上报(类型 12)。
|
||||||
|
*
|
||||||
|
* <p>相比旧的无确认上报,这里带上单调递增的 {@link #revision} 与唯一 {@link #reportId}:
|
||||||
|
* 主站据此丢弃乱序/重放的旧快照,并在事务提交后回 {@link TaskReportAckMessage},
|
||||||
|
* 节点只有在收到匹配回执后才清理自己的快照。
|
||||||
|
*/
|
||||||
|
@Data
|
||||||
|
public class TaskReportMessage extends AbstractMessage {
|
||||||
|
{
|
||||||
|
messageType = TASK_REPORT_MESSAGE;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 本次上报的唯一 ID,重复投递同一份快照时保持不变。 */
|
||||||
|
private String reportId;
|
||||||
|
|
||||||
|
/** 节点侧单调递增版本号;主站只接受比已记录更大的 revision。 */
|
||||||
|
private long revision;
|
||||||
|
|
||||||
|
private GalleryTask[] galleryTasks;
|
||||||
|
|
||||||
|
/** gid → 产生该状态的下载 operationId,用于丢弃属于旧操作的迟到上报。 */
|
||||||
|
private Map<Integer, String> operationIds = new LinkedHashMap<>();
|
||||||
|
}
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Message;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Domain.GalleryTask;
|
|
||||||
import lombok.Data;
|
|
||||||
|
|
||||||
@Data
|
|
||||||
public class UpdateGalleryMessage extends AbstractMessage{
|
|
||||||
{
|
|
||||||
messageType = UPDATE_GALLERY_MESSAGE;
|
|
||||||
}
|
|
||||||
|
|
||||||
GalleryTask galleryTask;
|
|
||||||
|
|
||||||
|
|
||||||
}
|
|
||||||
@@ -1,16 +1,14 @@
|
|||||||
package com.lion.lionwebsite.Service;
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
import com.lion.lionwebsite.Dao.CollectMapper;
|
import com.lion.lionwebsite.Dao.normal.CollectMapper;
|
||||||
import com.lion.lionwebsite.Util.Response;
|
import com.lion.lionwebsite.Util.Response;
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
@Service
|
@Service
|
||||||
@Data
|
@Data
|
||||||
public class CollectService {
|
public class CollectService {
|
||||||
@Resource
|
final CollectMapper collectMapper;
|
||||||
CollectMapper collectMapper;
|
|
||||||
|
|
||||||
public String collectGallery(int gid, int collector){
|
public String collectGallery(int gid, int collector){
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
@@ -26,7 +24,7 @@ public class CollectService {
|
|||||||
public String disCollectGallery(int gid, int collector){
|
public String disCollectGallery(int gid, int collector){
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
if(collectMapper.isCollect(gid, collector) == 0) { //没有收藏
|
if(collectMapper.isCollect(gid, collector) == 0) { //没有收藏
|
||||||
response.failure("没有收藏该本子");
|
response.failure("没有收藏该图片");
|
||||||
}else{
|
}else{
|
||||||
collectMapper.disCollect(gid, collector);
|
collectMapper.disCollect(gid, collector);
|
||||||
response.success("取消收藏成功");
|
response.success("取消收藏成功");
|
||||||
|
|||||||
@@ -0,0 +1,262 @@
|
|||||||
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
|
||||||
|
import com.lion.lionwebsite.Domain.GidToKey;
|
||||||
|
import com.lion.lionwebsite.Domain.ImageKeyCache;
|
||||||
|
import com.lion.lionwebsite.Exception.GalleryIndexException;
|
||||||
|
import com.lion.lionwebsite.Util.GalleryUtil;
|
||||||
|
import com.lion.lionwebsite.Util.SingleFlight;
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
import org.springframework.transaction.support.TransactionOperations;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 图片索引(gid 到每页 imgkey)的唯一读写入口。
|
||||||
|
*
|
||||||
|
* <p>这个类替换掉原先「读缓存 / 写缓存 / 回滚」分散在业务服务里、并且逐页 INSERT
|
||||||
|
* 的写法。三件事在这里被固定下来:
|
||||||
|
*
|
||||||
|
* <ol>
|
||||||
|
* <li><b>同 gid 合并</b>:并发请求同一本画廊时只有一个线程真的去抓上游,
|
||||||
|
* 其余线程等待同一个结果(含失败),因此不会出现两个构建者互相删除对方写入的索引。</li>
|
||||||
|
* <li><b>抓取在事务外</b>:上游 HTTP 可能耗时十几秒,绝不能占着 SQLite 写事务,
|
||||||
|
* 否则整个缓存库的写入都会排队。</li>
|
||||||
|
* <li><b>整组短事务发布</b>:一个事务里先删该 gid 的旧索引,再批量写入新索引。
|
||||||
|
* 失败整体回滚,磁盘上仍保留旧的完整索引;只有写入成功这一个状态是对外可见的。</li>
|
||||||
|
* </ol>
|
||||||
|
*
|
||||||
|
* <p>依赖全部由构造函数注入:mapper 负责落库,operations 负责事务边界,
|
||||||
|
* upstream 负责把 mpv 页解析成页 key 列表。测试可以直接给假的实现,
|
||||||
|
* 不需要起 Spring 上下文,也不需要网络。
|
||||||
|
*/
|
||||||
|
public class GalleryIndexService {
|
||||||
|
|
||||||
|
private static final Logger log = LoggerFactory.getLogger(GalleryIndexService.class);
|
||||||
|
|
||||||
|
/** 单条 INSERT 携带的最大行数,避免 SQL 过长。 */
|
||||||
|
static final int BATCH_SIZE = 200;
|
||||||
|
|
||||||
|
private final ImageCacheMapper mapper;
|
||||||
|
private final TransactionOperations operations;
|
||||||
|
private final TransactionOperations readOperations;
|
||||||
|
private final GalleryKeyUpstream upstream;
|
||||||
|
private final SingleFlight<String, GidToKey> builds = new SingleFlight<>();
|
||||||
|
|
||||||
|
public GalleryIndexService(ImageCacheMapper mapper, TransactionOperations operations,
|
||||||
|
GalleryKeyUpstream upstream) {
|
||||||
|
this(mapper, operations, operations, upstream);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param operations 发布索引的写事务(删除加批量写入必须整体生效)
|
||||||
|
* @param readOperations 读取索引的一致性快照事务;传 null 表示不额外开启读事务
|
||||||
|
*/
|
||||||
|
public GalleryIndexService(ImageCacheMapper mapper, TransactionOperations operations,
|
||||||
|
TransactionOperations readOperations, GalleryKeyUpstream upstream) {
|
||||||
|
if (mapper == null || operations == null || upstream == null)
|
||||||
|
throw new IllegalArgumentException("mapper/operations/upstream 不能为空");
|
||||||
|
this.mapper = mapper;
|
||||||
|
this.operations = operations;
|
||||||
|
this.readOperations = readOperations;
|
||||||
|
this.upstream = upstream;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 保证该画廊的索引可用,返回 gid 到 key 的映射。
|
||||||
|
*
|
||||||
|
* @param url 画廊链接,形如 https://exhentai.org/g/1596929/f08534d87d/
|
||||||
|
* @throws GalleryIndexException 链接错误 / 画廊不可用 / 暂时失败,见 GalleryIndexException.Reason
|
||||||
|
*/
|
||||||
|
public GidToKey ensureIndex(String url) throws GalleryIndexException {
|
||||||
|
return ensureIndex(url, this::buildAndPublish);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 同 ensureIndex,但允许调用方提供构建动作。
|
||||||
|
*
|
||||||
|
* <p>离线重建需要「先检查再决定是否重建」的不同策略,因此把构建步骤作为参数暴露出来;
|
||||||
|
* 生产调用使用默认实现即可。
|
||||||
|
*/
|
||||||
|
GidToKey ensureIndex(String url, IndexBuilder builder) throws GalleryIndexException {
|
||||||
|
GalleryLink link = GalleryLink.parse(url);
|
||||||
|
GidToKey cached = lookup(link);
|
||||||
|
if (cached != null)
|
||||||
|
return cached;
|
||||||
|
try {
|
||||||
|
return builds.run(link.gid(), () -> {
|
||||||
|
// 进入临界区后必须再看一眼:等待期间别的线程可能已经把索引建好了。
|
||||||
|
GidToKey existing = lookup(link);
|
||||||
|
return existing != null ? existing : builder.build(url);
|
||||||
|
});
|
||||||
|
} catch (GalleryIndexException e) {
|
||||||
|
throw e;
|
||||||
|
} catch (Exception e) {
|
||||||
|
throw new GalleryIndexException(GalleryIndexException.Reason.TRANSIENT,
|
||||||
|
"构建图片索引失败 gid=" + link.gid(), e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 已缓存且完整则返回索引,否则返回 null。
|
||||||
|
*
|
||||||
|
* <p>完整性由页码集合判定,而不是行数:历史重复行与半截写入都会让行数与页数不符。
|
||||||
|
*/
|
||||||
|
GidToKey lookup(GalleryLink link) {
|
||||||
|
if (readOperations != null)
|
||||||
|
return readOperations.execute(status -> lookupInTransaction(link));
|
||||||
|
return lookupInTransaction(link);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 在同一个读事务里读头部与页码,得到一个一致的快照。
|
||||||
|
*
|
||||||
|
* <p>两个独立的 SELECT 之间可能夹着另一个线程的整组替换,于是头部是旧的、页码是新的
|
||||||
|
* (或相反),完整性判断会得出错误结论。SQLite 的读事务能看到固定快照,避免这种交错。
|
||||||
|
*/
|
||||||
|
private GidToKey lookupInTransaction(GalleryLink link) {
|
||||||
|
GidToKey header = mapper.selectKeyByGid(link.gid());
|
||||||
|
if (header == null)
|
||||||
|
return null;
|
||||||
|
if (header.getPages() <= 0 || header.getKey() == null || header.getKey().isBlank())
|
||||||
|
return null;
|
||||||
|
if (!isComplete(header))
|
||||||
|
return null;
|
||||||
|
return header;
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isComplete(GidToKey header) {
|
||||||
|
List<Integer> pages = mapper.selectPageNumbersByGid(header.getGid());
|
||||||
|
if (pages == null || pages.size() != header.getPages())
|
||||||
|
return false;
|
||||||
|
if (mapper.countInvalidImageKeysByGid(header.getGid()) != 0)
|
||||||
|
return false;
|
||||||
|
// 页码必须是严格递增的 1..pages,重复或跳号都视为不完整。
|
||||||
|
for (int i = 0; i < pages.size(); i++) {
|
||||||
|
Integer page = pages.get(i);
|
||||||
|
if (page == null || page != i + 1)
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 抓上游(事务外)后短事务整组替换。 */
|
||||||
|
private GidToKey buildAndPublish(String url) throws GalleryIndexException {
|
||||||
|
GalleryLink link = GalleryLink.parse(url);
|
||||||
|
List<ImageKeyCache> pageKeys;
|
||||||
|
try {
|
||||||
|
// 抓取必须发生在事务之外:上游可能耗时十几秒,占着 SQLite 写事务会拖垮整个缓存库。
|
||||||
|
pageKeys = upstream.fetchPageKeys(url);
|
||||||
|
} catch (Exception e) {
|
||||||
|
throw new GalleryIndexException(GalleryIndexException.Reason.TRANSIENT,
|
||||||
|
"抓取图片索引失败 gid=" + link.gid(), e);
|
||||||
|
}
|
||||||
|
if (pageKeys == null || pageKeys.isEmpty())
|
||||||
|
throw new GalleryIndexException(GalleryIndexException.Reason.UNAVAILABLE,
|
||||||
|
"该图片已下架或已被删除 gid=" + link.gid());
|
||||||
|
|
||||||
|
// 落库前先校验上游数据自洽:整组替换一旦开始就没有回头路,
|
||||||
|
// 与其把畸形页码写进唯一约束再回滚,不如在此之前就判为失败。
|
||||||
|
validatePageKeys(link, pageKeys);
|
||||||
|
|
||||||
|
GidToKey header = new GidToKey();
|
||||||
|
header.setGid(link.gid());
|
||||||
|
header.setKey(link.key());
|
||||||
|
header.setPages(pageKeys.size());
|
||||||
|
|
||||||
|
publish(header, pageKeys);
|
||||||
|
return header;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 校验上游返回的页 key 是否自洽。
|
||||||
|
*
|
||||||
|
* <p>四条要求缺一不可,否则索引会以「看起来完整」的样子写进库:
|
||||||
|
* gid 属于请求的那本画廊、页码恰好是 1..N、页码不重复、imgkey 非空。
|
||||||
|
* 任一不满足都抛出 {@link GalleryIndexException.Reason#MALFORMED},此时尚未开启写事务,
|
||||||
|
* 库中旧索引保持原样。
|
||||||
|
*/
|
||||||
|
void validatePageKeys(GalleryLink link, List<ImageKeyCache> pageKeys) throws GalleryIndexException {
|
||||||
|
int expected = pageKeys.size();
|
||||||
|
Set<Integer> seen = new HashSet<>(expected * 2);
|
||||||
|
for (ImageKeyCache row : pageKeys) {
|
||||||
|
if (row == null || row.getGid() == null || !row.getGid().equals(link.gid()))
|
||||||
|
throw malformed(link, "上游页 key 的 gid 与请求不一致");
|
||||||
|
if (row.getImgkey() == null || row.getImgkey().isBlank())
|
||||||
|
throw malformed(link, "上游页 key 的 imgkey 为空");
|
||||||
|
int page = row.getPage();
|
||||||
|
if (page < 1 || page > expected)
|
||||||
|
throw malformed(link, "上游页码越界: " + page + "(共 " + expected + " 页)");
|
||||||
|
if (!seen.add(page))
|
||||||
|
throw malformed(link, "上游页码重复: " + page);
|
||||||
|
}
|
||||||
|
if (seen.size() != expected)
|
||||||
|
throw malformed(link, "上游页码不连续,去重后 " + seen.size() + " 个,期望 " + expected + " 个");
|
||||||
|
}
|
||||||
|
|
||||||
|
private GalleryIndexException malformed(GalleryLink link, String detail) {
|
||||||
|
log.warn("上游图片索引非法 gid={} detail={}", link.gid(), detail);
|
||||||
|
return new GalleryIndexException(GalleryIndexException.Reason.MALFORMED,
|
||||||
|
detail + " gid=" + link.gid());
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 在单个事务里整组替换索引。
|
||||||
|
*
|
||||||
|
* <p>失败时事务整体回滚:要么看到旧的完整索引,要么看到新的完整索引,
|
||||||
|
* 不存在「头部已更新、页 key 缺失」的中间态。
|
||||||
|
*/
|
||||||
|
void publish(GidToKey header, List<ImageKeyCache> pageKeys) {
|
||||||
|
operations.executeWithoutResult(status -> {
|
||||||
|
mapper.deleteImageKeyCacheByGid(header.getGid());
|
||||||
|
mapper.deleteGidToKey(header.getGid());
|
||||||
|
mapper.insertGidToKey(header);
|
||||||
|
writeBatches(pageKeys);
|
||||||
|
});
|
||||||
|
log.info("图片索引已整体替换 gid={} pages={}", header.getGid(), header.getPages());
|
||||||
|
}
|
||||||
|
|
||||||
|
private void writeBatches(List<ImageKeyCache> pageKeys) {
|
||||||
|
for (int start = 0; start < pageKeys.size(); start += BATCH_SIZE) {
|
||||||
|
int end = Math.min(start + BATCH_SIZE, pageKeys.size());
|
||||||
|
mapper.insertImageKeyCacheBatch(new ArrayList<>(pageKeys.subList(start, end)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 便于日志与错误信息使用的解析结果。 */
|
||||||
|
record GalleryLink(String gid, String key) {
|
||||||
|
|
||||||
|
static GalleryLink parse(String url) throws GalleryIndexException {
|
||||||
|
String[] segments = url == null ? null : url.split("/");
|
||||||
|
Integer gid = GalleryUtil.parseGid(url);
|
||||||
|
if (gid == null || segments == null || segments.length <= 5 || segments[5].isBlank())
|
||||||
|
throw new GalleryIndexException(GalleryIndexException.Reason.INVALID_LINK,
|
||||||
|
"链接错误");
|
||||||
|
return new GalleryLink(String.valueOf(gid), segments[5].strip());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 上游取页 key 的端口。
|
||||||
|
*
|
||||||
|
* <p>生产实现只是把 URL 转交给 GalleryUtil.parseImageKeys;
|
||||||
|
* 单独抽出来是为了让本服务在测试里完全脱离网络与静态工具类。
|
||||||
|
*/
|
||||||
|
@FunctionalInterface
|
||||||
|
public interface GalleryKeyUpstream {
|
||||||
|
List<ImageKeyCache> fetchPageKeys(String url) throws Exception;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 构建索引的动作。允许抛出业务异常,因此不用 java.util.function.Function
|
||||||
|
* (后者不能抛受检异常,会让调用方被迫包装成运行时异常)。
|
||||||
|
*/
|
||||||
|
@FunctionalInterface
|
||||||
|
interface IndexBuilder {
|
||||||
|
GidToKey build(String url) throws GalleryIndexException;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,156 +1,202 @@
|
|||||||
package com.lion.lionwebsite.Service;
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
import cn.hutool.core.io.FileUtil;
|
import com.lion.lionwebsite.Dao.cache.ImageCacheMapper;
|
||||||
import cn.hutool.core.util.RandomUtil;
|
import com.lion.lionwebsite.Dao.normal.*;
|
||||||
import com.lion.lionwebsite.Dao.*;
|
|
||||||
import com.lion.lionwebsite.Domain.*;
|
import com.lion.lionwebsite.Domain.*;
|
||||||
import com.lion.lionwebsite.Exception.ResolutionNotMatchException;
|
import com.lion.lionwebsite.Exception.ResolutionNotMatchException;
|
||||||
import com.lion.lionwebsite.Error.ErrorCode;
|
import com.lion.lionwebsite.Error.ErrorCode;
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
import com.lion.lionwebsite.Util.CustomUtil;
|
||||||
import com.lion.lionwebsite.Util.FileDownload;
|
import com.lion.lionwebsite.Util.FileDownload;
|
||||||
|
import com.lion.lionwebsite.Util.ImageFileCache;
|
||||||
|
import java.nio.file.Path;
|
||||||
import com.lion.lionwebsite.Util.GalleryUtil;
|
import com.lion.lionwebsite.Util.GalleryUtil;
|
||||||
import com.lion.lionwebsite.Util.Response;
|
import com.lion.lionwebsite.Util.Response;
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
import tools.jackson.databind.node.ObjectNode;
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
import lombok.Data;
|
import lombok.Data;
|
||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
|
||||||
import org.springframework.stereotype.Service;
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
import java.io.*;
|
import java.io.*;
|
||||||
import java.time.LocalDateTime;
|
import java.net.URI;
|
||||||
import java.time.ZoneId;
|
|
||||||
import java.util.*;
|
import java.util.*;
|
||||||
|
import java.util.concurrent.Callable;
|
||||||
|
|
||||||
import static com.lion.lionwebsite.Util.CustomUtil.dateTimeFormatter;
|
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
|
||||||
import static com.lion.lionwebsite.Util.CustomUtil.fourZeroFour;
|
import static com.lion.lionwebsite.Util.GalleryUtil.*;
|
||||||
|
|
||||||
@Service
|
@Service
|
||||||
@Data
|
@Data
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@ConfigurationProperties(prefix = "gallery-manage-service")
|
|
||||||
public class GalleryManageService {
|
public class GalleryManageService {
|
||||||
String TargetPath;
|
String cachePath = "/storage/galleryCache/onlineImages/";
|
||||||
|
|
||||||
int cacheSize;
|
final GalleryMapper galleryMapper;
|
||||||
|
|
||||||
Map<Integer, String> gid2name_cache = new HashMap<>();
|
final CollectMapper collectMapper;
|
||||||
|
|
||||||
Map<Integer, Integer> gid2page_cache = new HashMap<>();
|
final CustomConfigurationMapper configurationMapper;
|
||||||
|
|
||||||
Map<String, String> page2pageNameCache = new HashMap<>();
|
final UserMapper userMapper;
|
||||||
|
|
||||||
@Resource
|
final ImageCacheMapper imageCacheMapper;
|
||||||
GalleryMapper galleryMapper;
|
|
||||||
|
|
||||||
@Resource
|
final RemoteService remoteService;
|
||||||
CollectMapper collectMapper;
|
|
||||||
|
|
||||||
@Resource
|
final PushService pushService;
|
||||||
CustomConfigurationMapper configurationMapper;
|
|
||||||
|
|
||||||
@Resource
|
/**
|
||||||
UserMapper userMapper;
|
* 任务生命周期(协议 v2)。创建/删除都先走它做持久化意图,再产生外部副作用。
|
||||||
|
* 用字段注入是为了兼容现有构造式 mock 测试与旧调用点。
|
||||||
|
*/
|
||||||
|
@org.springframework.beans.factory.annotation.Autowired(required = false)
|
||||||
|
GalleryTaskLifecycleService lifecycleService;
|
||||||
|
|
||||||
@Resource
|
@org.springframework.beans.factory.annotation.Autowired
|
||||||
ShareFileMapper shareFileMapper;
|
GalleryIndexService galleryIndexService;
|
||||||
|
|
||||||
@Resource
|
|
||||||
TagMapper tagMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
PageNameCacheMapper pageNameCacheMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
RemoteService remoteService;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 创建任务
|
* 创建任务
|
||||||
* @param link 任务链接
|
* @param link 任务链接
|
||||||
* @param targetResolution 目标分辨率
|
* @param targetResolution 目标分辨率
|
||||||
* @param AuthCode 授权码(用于记录是谁下载的)
|
* @param AuthCode 授权码(用于记录是谁下载的)
|
||||||
* @return 提交结果
|
* @return 提交结果
|
||||||
*/
|
*/
|
||||||
public String createTask(String link, String targetResolution, String AuthCode, List<Integer> tidS, byte mode){
|
public String createTask(String link, String targetResolution, String AuthCode) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
|
User user = userMapper.selectUserByAuthCode(AuthCode);
|
||||||
// return Response._failure("调试中,请勿提交任务");
|
// return Response._failure("调试中,请勿提交任务");
|
||||||
|
|
||||||
if(remoteService.isDead())
|
if (user == null) {
|
||||||
return Response._failure("节点挂了,找狮子处理");
|
response.failure("授权码无效");
|
||||||
int gid;
|
|
||||||
try {
|
|
||||||
gid = Integer.parseInt(link.split("/")[4]);
|
|
||||||
}catch (NumberFormatException e){
|
|
||||||
response.failure("链接错误");
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 段数不足会先抛 ArrayIndexOutOfBoundsException,非数字段抛 NumberFormatException;
|
||||||
|
// 只捕后者会让畸形链接穿透为 500(本项目无 @ControllerAdvice)。
|
||||||
|
Integer parsedGid = parseGidFromLink(link);
|
||||||
|
if (parsedGid == null) {
|
||||||
|
response.failure("链接错误");
|
||||||
|
pushService.taskCreateReport(user.getUsername(), "未知任务", response);
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
int gid = parsedGid;
|
||||||
|
String taskName = "任务 [" + gid + "]";
|
||||||
|
|
||||||
|
if (remoteService.isDead()) {
|
||||||
|
response.failure("节点挂了,找狮子处理");
|
||||||
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
|
// 协议 v2 生命周期:gid 唯一意图先落库(占位行「提交中」),提交前持久化 SUBMITTING,
|
||||||
|
// 上游结果不确定时报 UNKNOWN 而不是自动重发;差额额度与任务行在同一短事务里写入。
|
||||||
|
if (lifecycleService != null) {
|
||||||
|
response = lifecycleService.createTask(gid, taskName, link, targetResolution, user.getId());
|
||||||
|
Gallery current = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
taskName = current == null || current.getName() == null ? taskName : current.getName();
|
||||||
|
// 业务结果由生命周期服务给出,data 用结构化对象下发(前端解码器兼容旧字符串)。
|
||||||
|
if (response.isSuccess() && current != null)
|
||||||
|
response.success(objectMapper.valueToTree(current));
|
||||||
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
Gallery gallery;
|
Gallery gallery;
|
||||||
|
|
||||||
//判断数据库中是否有这个任务,有则返回状态
|
//判断数据库中是否有这个任务,有则返回状态
|
||||||
if((gallery = galleryMapper.selectGalleryByGid(gid)) != null){
|
if ((gallery = galleryMapper.selectGalleryByGid(gid)) != null) {
|
||||||
response.failure("任务队列已有此任务,任务状态: " + gallery.getStatus() + " 请点击查找任务");
|
response.failure("任务队列已有此任务,任务状态: " + gallery.getStatus() + " 请点击查找任务");
|
||||||
|
pushService.taskCreateReport(user.getUsername(), gallery.getName(), response);
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
//尝试下载本子,返回结果
|
//尝试下载图片,返回结果
|
||||||
try {
|
try {
|
||||||
gallery = GalleryUtil.parse(link, true, targetResolution);
|
gallery = GalleryUtil.parse(link, true, targetResolution);
|
||||||
if(gallery == null){
|
if (gallery == null || !gallery.getStatus().equals("已提交")) {
|
||||||
log.error("创建任务: {},解析失败", link);
|
log.error("创建任务失败: {},", link);
|
||||||
return Response._failure("任务解析失败,未知原因,请检查链接是否正常");
|
response.failure("提交任务失败,未知原因,请检查链接是否正常");
|
||||||
}else{
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
|
return response.toJSONString();
|
||||||
|
} else {
|
||||||
|
taskName = gallery.getName();
|
||||||
log.info("创建任务: {} 目标分辨率:{}", link, targetResolution);
|
log.info("创建任务: {} 目标分辨率:{}", link, targetResolution);
|
||||||
if(remoteService.addGalleryToQueue(gallery, mode) != 0){
|
// Persist before dispatch: the node sends its current status before its ACK.
|
||||||
|
gallery.setDownloader(user.getId());
|
||||||
|
gallery.set_download(true);
|
||||||
|
galleryMapper.insertGallery(gallery);
|
||||||
|
configurationMapper.incrementConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, gallery.getFileSize());
|
||||||
|
if (remoteService.addGalleryToQueue(gallery) != 0) {
|
||||||
log.error("传送任务{}失败, 未知原因", gallery.getName());
|
log.error("传送任务{}失败, 未知原因", gallery.getName());
|
||||||
return Response._failure("任务传送失败,未知原因");
|
response.failure("任务已保存,但节点未确认接收;请刷新任务列表后重试");
|
||||||
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}catch (ResolutionNotMatchException e){
|
} catch (ResolutionNotMatchException e) {
|
||||||
e.printStackTrace();
|
|
||||||
response.failure("提交失败,分辨率不存在");
|
response.failure("提交失败,分辨率不存在");
|
||||||
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}catch (IOException e){
|
} catch (IOException e) {
|
||||||
e.printStackTrace();
|
log.error(e.getMessage());
|
||||||
response.failure("IO错误,可能是网络波动");
|
response.failure("IO错误,可能是网络波动");
|
||||||
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
//处理下载结果,将任务插入数据库并且更新每周用量
|
// Do not overwrite an immediate node status with the original submitted state.
|
||||||
if(gallery.getStatus().equals("已提交")) {
|
Gallery current = galleryMapper.selectGalleryByGid(gallery.getGid());
|
||||||
response.success(gallery.toString());
|
response.success(objectMapper.valueToTree(current == null ? gallery : current));
|
||||||
gallery.setDownloader(userMapper.selectUserByAuthCode(AuthCode).getId());
|
pushService.taskCreateReport(user.getUsername(), taskName, response);
|
||||||
gallery.setMode(mode);
|
|
||||||
galleryMapper.insertGallery(gallery);
|
|
||||||
|
|
||||||
long usedAmount = Long.parseLong(configurationMapper.selectConfiguration(CustomConfiguration.WEEK_USED_AMOUNT).getValue());
|
|
||||||
usedAmount += gallery.getFileSize();
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, String.valueOf(usedAmount));
|
|
||||||
if(tidS != null)
|
|
||||||
for (Integer tid : tidS)
|
|
||||||
if(tagMapper.selectTagExistById(tid) > 0)
|
|
||||||
tagMapper.markTag(gallery.getGid(), tid);
|
|
||||||
}
|
|
||||||
else{
|
|
||||||
response.failure("提交失败,未知原因");
|
|
||||||
galleryMapper.deleteGalleryByGid(gallery.getGid());
|
|
||||||
}
|
|
||||||
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 根据链接查询本子
|
* 从任务链接中安全提取 gid。任何畸形输入(null、段数不足、非数字)都返回 null,
|
||||||
* @param link 链接
|
* 由调用方转成业务失败,避免异常穿透为 500。
|
||||||
|
*/
|
||||||
|
static Integer parseGidFromLink(String link) {
|
||||||
|
if (link == null)
|
||||||
|
return null;
|
||||||
|
String[] segments = link.split("/");
|
||||||
|
if (segments.length <= 4)
|
||||||
|
return null;
|
||||||
|
try {
|
||||||
|
return Integer.parseInt(segments[4]);
|
||||||
|
} catch (NumberFormatException e) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 尝试重新连接
|
||||||
|
* @return 重连结果
|
||||||
|
*/
|
||||||
|
public String reconnect(){
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
|
||||||
|
return switch (remoteService.reconnect()) {
|
||||||
|
case 0 -> response.success("重连成功").toJSONString();
|
||||||
|
case -1 -> response.failure("重连失败").toJSONString();
|
||||||
|
case -2 -> response.failure("当前未连接").toJSONString();
|
||||||
|
default -> response.failure("未知错误").toJSONString();
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 根据链接查询图片
|
||||||
|
* @param link 链接
|
||||||
* @return 查询结果
|
* @return 查询结果
|
||||||
*/
|
*/
|
||||||
public String selectTaskByLink(String link) {
|
public String selectTaskByLink(String link) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
Integer gid = parseGid(link);
|
Integer gid = parseGid(link);
|
||||||
Gallery gallery;
|
Gallery gallery;
|
||||||
if(gid != null)
|
if (gid != null)
|
||||||
gallery = galleryMapper.selectGalleryByGid(gid);
|
gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
else {
|
else {
|
||||||
response.failure("链接错误");
|
response.failure("链接错误");
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
@@ -158,114 +204,120 @@ public class GalleryManageService {
|
|||||||
log.info("查询{}", link);
|
log.info("查询{}", link);
|
||||||
|
|
||||||
//判断数据库中是否有这个任务,如果有则返回,如果没有则直接查询
|
//判断数据库中是否有这个任务,如果有则返回,如果没有则直接查询
|
||||||
if(gallery == null)
|
if (gallery == null)
|
||||||
try{
|
try {
|
||||||
gallery = GalleryUtil.parse(link, false, null);
|
gallery = GalleryUtil.parse(link, false, null);
|
||||||
if(gallery != null)
|
if (gallery != null)
|
||||||
response.success(new ObjectMapper().valueToTree(gallery).toString());
|
response.success(objectMapper.valueToTree(gallery));
|
||||||
else
|
else
|
||||||
response.failure("查询失败");
|
response.failure("查询失败");
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
e.printStackTrace();
|
log.info(e.getMessage());
|
||||||
response.failure("查询失败");
|
response.failure("查询失败");
|
||||||
}
|
}
|
||||||
|
|
||||||
else
|
else
|
||||||
response.success(new ObjectMapper().valueToTree(gallery).toString());
|
response.success(objectMapper.valueToTree(gallery));
|
||||||
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 通过gid查询本子
|
* 通过gid查询图片
|
||||||
|
*
|
||||||
* @param gid gid
|
* @param gid gid
|
||||||
* @return 查询结果
|
* @return 查询结果
|
||||||
*/
|
*/
|
||||||
public String selectTaskByGid(int gid){
|
public String selectTaskByGid(int gid) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
|
||||||
if(gallery == null)
|
if (gallery == null)
|
||||||
response.failure("未找到该本子,请使用链接");
|
response.failure("未找到该图片,请使用链接");
|
||||||
|
|
||||||
else
|
else
|
||||||
response.success(gallery.toString());
|
response.success(objectMapper.valueToTree(gallery));
|
||||||
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 查询所有本子
|
* 查询所有图片
|
||||||
|
*
|
||||||
* @return 查询结果
|
* @return 查询结果
|
||||||
*/
|
*/
|
||||||
public String selectAllGallery(int userId) {
|
public String selectAllGallery(int userId) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
Gallery[] galleries = galleryMapper.selectAllGallery();
|
Gallery[] galleries = galleryMapper.selectAllGallery();
|
||||||
|
|
||||||
if(galleries == null){
|
if (galleries == null) {
|
||||||
response.failure("没有找到本子");
|
response.failure("没有找到图片");
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
ArrayList<Integer> galleryIds = collectMapper.selectGidByCollector(userId);
|
ArrayList<Integer> galleryIds = collectMapper.selectGidByCollector(userId);
|
||||||
Iterator<Integer> idIterator;
|
// 原先对每个画廊线性扫描收藏列表(O(n*m))并就地删除迭代器元素;
|
||||||
|
// 改成集合查找后同样是「命中即标记」,复杂度降到 O(n+m)。
|
||||||
if(!galleryIds.isEmpty()) //如果该用户收藏了本子
|
if (!galleryIds.isEmpty()) {
|
||||||
galleryLoop: for (Gallery gallery : galleries) { //遍历本子
|
Set<Integer> collected = new HashSet<>(galleryIds);
|
||||||
idIterator = galleryIds.iterator();
|
for (Gallery gallery : galleries)
|
||||||
while (idIterator.hasNext()){ //遍历收藏的gid
|
if (collected.contains(gallery.getGid()))
|
||||||
Integer id = idIterator.next();
|
gallery.setCollect(true);
|
||||||
if(id.equals(gallery.getGid())){ //如果找到对应的gid,修改对应本子的属性,删除当前gid,判断是否需要跳出或者结束循环
|
|
||||||
gallery.setCollect(true);
|
|
||||||
idIterator.remove();
|
|
||||||
|
|
||||||
if(galleryIds.isEmpty())
|
|
||||||
break galleryLoop;
|
|
||||||
else
|
|
||||||
continue galleryLoop;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
HashMap<Integer, ArrayList<Integer>> tagsMap = new HashMap<>();
|
|
||||||
ArrayList<TagMark> tags = tagMapper.selectAllMark();
|
|
||||||
|
|
||||||
for (TagMark tagMark : tags) { //从数据库取出所有标记,然后筛选出 gid -> tids ,后面加个缓存
|
|
||||||
tagsMap.computeIfAbsent(tagMark.getGid(), k -> new ArrayList<>());
|
|
||||||
tagsMap.get(tagMark.getGid()).add(tagMark.getTid());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
ArrayList<Integer> temp;
|
// 下载人昵称只对管理员下发;一次建表避免逐条查询。
|
||||||
for (Gallery gallery : galleries) {
|
if (userId == UserService.ADMIN_USER_ID)
|
||||||
if((temp = tagsMap.get(gallery.getGid())) != null) {
|
fillDownloaderNames(galleries);
|
||||||
gallery.setTags(new ArrayList<>());
|
|
||||||
gallery.getTags().addAll(temp);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
response.success(new ObjectMapper().valueToTree(galleries).toString());
|
response.success(objectMapper.valueToTree(galleries));
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** 给任务列表补上下载人昵称,找不到用户时保留空值而不是报错。 */
|
||||||
|
private void fillDownloaderNames(Gallery[] galleries) {
|
||||||
|
User[] users = userMapper.selectAllUser();
|
||||||
|
Map<Integer, String> names = new HashMap<>();
|
||||||
|
if (users != null)
|
||||||
|
for (User user : users)
|
||||||
|
names.put(user.getId(), user.getUsername());
|
||||||
|
|
||||||
|
for (Gallery gallery : galleries)
|
||||||
|
gallery.setDownloaderName(names.get(gallery.getDownloader()));
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 查询未完成的本子
|
* 查询某个任务的实际下载人昵称。
|
||||||
* @return 查询结果
|
* 只有管理员可调用,普通用户一律拒绝,避免暴露他人身份。
|
||||||
|
*
|
||||||
|
* @param gid 任务 gid
|
||||||
|
* @param AuthCode 调用方授权码
|
||||||
*/
|
*/
|
||||||
public String selectUnDoneGallery() {
|
public String selectDownloaderByGid(int gid, String AuthCode) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
Gallery[] galleries = galleryMapper.selectUnDoneGalleries();
|
User requester = userMapper.selectUserByAuthCode(AuthCode);
|
||||||
|
if (requester == null || requester.getId() != UserService.ADMIN_USER_ID) {
|
||||||
|
response.failure("无权查看下载人");
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
if(galleries.length > 0)
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
response.success(new ObjectMapper().valueToTree(galleries).toString());
|
if (gallery == null) {
|
||||||
else
|
response.failure("任务不存在");
|
||||||
response.failure();
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
|
User downloader = userMapper.selectUserById(gallery.getDownloader());
|
||||||
|
ObjectNode node = objectMapper.createObjectNode();
|
||||||
|
node.put("gid", gid);
|
||||||
|
node.put("downloader", gallery.getDownloader());
|
||||||
|
node.put("downloaderName", downloader == null ? "" : downloader.getUsername());
|
||||||
|
response.success(node);
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 通过本子名查询本子
|
* 通过图片名查询图片
|
||||||
|
*
|
||||||
* @param name 名字
|
* @param name 名字
|
||||||
* @return 查询结果
|
* @return 查询结果
|
||||||
*/
|
*/
|
||||||
@@ -273,193 +325,114 @@ public class GalleryManageService {
|
|||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
Gallery gallery = galleryMapper.selectGalleryByName("%" + name + "%");
|
Gallery gallery = galleryMapper.selectGalleryByName("%" + name + "%");
|
||||||
|
|
||||||
if(gallery != null)
|
if (gallery != null)
|
||||||
response.success(new ObjectMapper().valueToTree(gallery).toString());
|
response.success(objectMapper.valueToTree(gallery));
|
||||||
else
|
else
|
||||||
response.failure("没有找到该名字的本子");
|
response.failure("没有找到该名字的图片");
|
||||||
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 查询用户下载的本子
|
* 查询用户下载的图片
|
||||||
* @param AuthCode 授权码(用于查询用户名)
|
*
|
||||||
* @return 查询结果
|
* @param AuthCode 授权码(用于查询用户名)
|
||||||
|
* @return 查询结果
|
||||||
*/
|
*/
|
||||||
public String selectGalleryByDownloader(String AuthCode){
|
public String selectGalleryByDownloader(String AuthCode) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
|
|
||||||
Gallery[] galleries = galleryMapper.selectGalleryByDownloader(userMapper.selectUserByAuthCode(AuthCode).getId());
|
Gallery[] galleries = galleryMapper.selectGalleryByDownloader(userMapper.selectUserByAuthCode(AuthCode).getId());
|
||||||
if(galleries.length > 0)
|
if (galleries.length > 0)
|
||||||
response.success(new ObjectMapper().valueToTree(galleries).toString());
|
response.success(objectMapper.valueToTree(galleries));
|
||||||
else
|
else
|
||||||
response.failure("您未下载本子");
|
response.failure("您未下载图片");
|
||||||
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
// /**
|
|
||||||
// * 获取本子图片的名字
|
|
||||||
// * @param gid gid
|
|
||||||
// * @return 名字数组
|
|
||||||
// */
|
|
||||||
// public String selectOnlineFilename(Integer gid){
|
|
||||||
// Response response = Response.generateResponse();
|
|
||||||
// String path;
|
|
||||||
//
|
|
||||||
// if((path = gid2name_cache.get(gid)) == null) {
|
|
||||||
// Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
|
||||||
//
|
|
||||||
// if (gallery == null) {
|
|
||||||
// response.failure("本子不存在");
|
|
||||||
// return response.toJSONString();
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// path = TargetPath + gallery.getName();
|
|
||||||
// gid2name_cache.put(gid, path);
|
|
||||||
// if(gid2name_cache.size() > cacheSize)
|
|
||||||
// gid2name_cache.remove(gid2name_cache.keySet().iterator().next());
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// File file = new File(path);
|
|
||||||
// if(!file.isDirectory()){
|
|
||||||
// response.failure("本子文件已被删除");
|
|
||||||
// return response.toJSONString();
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// File[] files = file.listFiles(pathname -> pathname.getName().endsWith(".webp") && !pathname.getName().startsWith("thumbnail"));
|
|
||||||
// if(files == null || files.length == 0){
|
|
||||||
// response.failure("本子文件丢失");
|
|
||||||
// return response.toJSONString();
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// ArrayList<String> images = new ArrayList<>();
|
|
||||||
// for(File image: files){
|
|
||||||
// images.add(image.getName().replace(".webp", ""));
|
|
||||||
// }
|
|
||||||
// //字符串长度相等时比较字典顺序,不相等时比较长度
|
|
||||||
// images.sort((s1, s2) ->
|
|
||||||
// s1.length() == s2.length() ? s1.compareTo(s2): s1.length() - s2.length()
|
|
||||||
// );
|
|
||||||
//
|
|
||||||
// response.success(new ObjectMapper().valueToTree(images).toString());
|
|
||||||
// return response.toJSONString();
|
|
||||||
// }
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 删除本子以及对应的文件(如果存在的话)
|
* 删除图片以及对应的文件(如果存在的话)
|
||||||
|
*
|
||||||
* @param gid gid
|
* @param gid gid
|
||||||
* @return 删除结果
|
* @return 删除结果
|
||||||
*/
|
*/
|
||||||
public String deleteGalleryByGid(Integer gid, String AuthCode, byte mode) {
|
public String deleteGalleryByGid(Integer gid, String AuthCode) {
|
||||||
Response response = Response.generateResponse();
|
Response response = Response.generateResponse();
|
||||||
String deleteType;
|
|
||||||
|
|
||||||
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
User user = userMapper.selectUserByAuthCode(AuthCode);
|
User user = userMapper.selectUserByAuthCode(AuthCode);
|
||||||
if(gallery == null){
|
if (gallery == null) {
|
||||||
response.failure("删除失败,该本子不存在");
|
response.failure("删除失败,该图片不存在");
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
if (user == null) {
|
||||||
|
response.failure("删除失败,授权码无效");
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
ArrayList<Integer> collector = collectMapper.selectCollectorByGid(gallery.getGid());
|
ArrayList<Integer> collector = collectMapper.selectCollectorByGid(gallery.getGid());
|
||||||
if(!(collector.isEmpty() || collector.size() == 1 && collector.get(0).equals(user.getId()) //判断收藏
|
// 拒绝条件与提示文案一致:被别人收藏,或者请求者不是下载人。
|
||||||
&& gallery.getDownloader() == user.getId())) //判断下载
|
// 注意不能写成 collector.isEmpty() || ...:那样在「无任何收藏」时会短路放行,
|
||||||
response.failure("删除失败,该本子已被别人收藏或你不是下载人");
|
// 从而完全跳过下载者校验,导致任何有效授权码都能删除他人任务。
|
||||||
|
boolean collectedByOthers = collector.stream().anyMatch(id -> id != user.getId());
|
||||||
else{
|
boolean isDownloader = gallery.getDownloader() == user.getId();
|
||||||
if((gallery.getMode() & mode) == 0) { //不能删除对应的,因为对应的没有
|
if (collectedByOthers || !isDownloader) {
|
||||||
if (mode == 1)
|
response.failure("删除失败,该图片已被别人收藏或你不是下载人");
|
||||||
response.failure("无法删除该本子的源文件,不存在");
|
log.info("拒绝删除 gid={}:collectedByOthers={} isDownloader={}", gid, collectedByOthers, isDownloader);
|
||||||
else if (mode == 2)
|
|
||||||
response.failure("无法删除该本子的预览图,不存在");
|
|
||||||
} else {
|
|
||||||
log.info("删除本子{}, mode:{}, delete_mode:{}", gallery.getName(), gallery.getMode(), mode);
|
|
||||||
gallery.setMode((byte) (gallery.getMode() ^ mode));
|
|
||||||
if(gallery.getMode() == 0) { //删整个本子
|
|
||||||
galleryMapper.deleteGalleryByGid(gallery.getGid()); //删除本子记录
|
|
||||||
ArrayList<Integer> tidS = tagMapper.selectTagByGid(gallery.getGid());
|
|
||||||
for (Integer tid : tidS)
|
|
||||||
tagMapper.decrTagUsage(tid);
|
|
||||||
|
|
||||||
tagMapper.disMarkTagByGid(gallery.getGid()); //删除本子标记
|
|
||||||
gid2name_cache.remove(gallery.getGid()); //删除本子缓存
|
|
||||||
page2pageNameCache.remove(gallery.getName()); //移除页数缓存
|
|
||||||
pageNameCacheMapper.deletePageNameByGid(gid);
|
|
||||||
|
|
||||||
if(new File(TargetPath + "/" + gallery.getName()).isDirectory()) //删除本地缓存
|
|
||||||
FileUtil.del(TargetPath + "/" + gallery.getName());
|
|
||||||
deleteType = "全部";
|
|
||||||
}else{
|
|
||||||
if(mode == 2 && new File(TargetPath + "/" + gallery.getName()).isDirectory()) { //删除预览缓存
|
|
||||||
FileUtil.del(TargetPath + "/" + gallery.getName());
|
|
||||||
gid2name_cache.remove(gallery.getGid()); //删除本子缓存
|
|
||||||
page2pageNameCache.remove(gallery.getName()); //移除页数缓存
|
|
||||||
}
|
|
||||||
|
|
||||||
galleryMapper.updateGallery(gallery);
|
|
||||||
deleteType = (mode == 1? "源文件": "预览图"); //用于提示
|
|
||||||
}
|
|
||||||
switch (remoteService.deleteGallery(gallery, mode)){
|
|
||||||
case ErrorCode.IO_ERROR -> response.failure("本子:" + gallery.getName() + deleteType + "删除失败,IO错误");
|
|
||||||
case ErrorCode.FILE_NOT_FOUND -> response.failure("本子:" + gallery.getName() + deleteType + "删除失败,文件不存在");
|
|
||||||
case 0 -> response.success();
|
|
||||||
}
|
|
||||||
if(response.get("result").equals("failure"))
|
|
||||||
log.info(response.getResult());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 更新本子
|
|
||||||
* @param link 目标链接
|
|
||||||
* @return 更新结果
|
|
||||||
*/
|
|
||||||
public String updateGallery(String link){
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
|
|
||||||
Integer gid = parseGid(link);
|
|
||||||
if(gid == null) {
|
|
||||||
response.failure("链接格式出错");
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
Gallery oldGallery = galleryMapper.selectGalleryByGid(gid);
|
// 通过授权后才产生外部副作用。协议 v2 的删除顺序:
|
||||||
if(oldGallery == null) {
|
// 先落「待删除」意图 → 节点确认(或文件已不存在)→ 才在同一事务删任务行并留墓碑。
|
||||||
response.failure("数据库不存在该本子,请提交下载");
|
// 失败时任务行与意图都保留,可沿原记录重试并复用同一个删除 operationId。
|
||||||
|
log.info("删除图片{}", gallery.getName());
|
||||||
|
if (lifecycleService != null) {
|
||||||
|
// 被拒绝(例如任务仍在提交中)时必须回业务失败:
|
||||||
|
// 绝不能退避到裸删除,否则会与并发的创建 POST 同时执行并写回已删/重建的 gid。
|
||||||
|
GalleryTaskLifecycleService.DeleteIntent intent = lifecycleService.beginDelete(gallery.getGid());
|
||||||
|
if (!intent.accepted()) {
|
||||||
|
response.failure(intent.message());
|
||||||
|
log.info("拒绝删除 gid={}:{}", gid, intent.message());
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
byte result = remoteService.sendDeleteCommand(intent.deleteOperationId(),
|
||||||
|
gallery.getGid(), gallery.getName());
|
||||||
|
switch (result) {
|
||||||
|
case 0 -> {
|
||||||
|
lifecycleService.completeDelete(gallery.getGid(), intent.deleteOperationId());
|
||||||
|
response.success();
|
||||||
|
}
|
||||||
|
// 文件已不存在等于目标达成:同样完成本地清理,避免留下永远删不掉的任务行。
|
||||||
|
case ErrorCode.FILE_NOT_FOUND -> {
|
||||||
|
lifecycleService.completeDelete(gallery.getGid(), intent.deleteOperationId());
|
||||||
|
response.success();
|
||||||
|
}
|
||||||
|
case ErrorCode.IO_ERROR -> {
|
||||||
|
lifecycleService.failDelete(gallery.getGid());
|
||||||
|
response.failure("图片:" + gallery.getName() + "删除失败,IO错误");
|
||||||
|
}
|
||||||
|
// 节点无响应/超时:保留记录与待删除意图,可重试。
|
||||||
|
default -> {
|
||||||
|
lifecycleService.failDelete(gallery.getGid());
|
||||||
|
response.failure("图片:" + gallery.getName() + "删除失败,节点无响应");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!response.isSuccess())
|
||||||
|
log.info(response.getData());
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
try {
|
galleryMapper.deleteGalleryByGid(gallery.getGid());
|
||||||
String newLink = GalleryUtil.queryUpdateLink(oldGallery.getLink());
|
switch (remoteService.deleteGallery(gallery)) {
|
||||||
if(newLink == null)
|
case ErrorCode.IO_ERROR -> response.failure("图片:" + gallery.getName() + "删除失败,IO错误");
|
||||||
response.failure("本子没有更新");
|
case ErrorCode.FILE_NOT_FOUND -> response.failure("图片:" + gallery.getName() + "删除失败,文件不存在");
|
||||||
|
case 0 -> response.success();
|
||||||
else{
|
// 节点无响应/超时会返回 -1 等非枚举值;必须显式判失败,
|
||||||
Gallery newGallery = GalleryUtil.parse(newLink, true, oldGallery.getResolution());
|
// 否则 result 保持未设置,末尾的 response.get("result") 会抛 NPE。
|
||||||
if(newGallery != null) {
|
default -> response.failure("图片:" + gallery.getName() + "删除失败,节点无响应");
|
||||||
galleryMapper.deleteGalleryByGid(oldGallery.getGid());
|
|
||||||
remoteService.updateGallery(newGallery, oldGallery.getMode());
|
|
||||||
newGallery.setDownloader(oldGallery.getDownloader());
|
|
||||||
newGallery.setCollector(oldGallery.getCollector());
|
|
||||||
|
|
||||||
long usedAmount = Long.parseLong(configurationMapper.selectConfiguration(CustomConfiguration.WEEK_USED_AMOUNT).getValue());
|
|
||||||
usedAmount += newGallery.getFileSize();
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, String.valueOf(usedAmount));
|
|
||||||
|
|
||||||
galleryMapper.insertGallery(newGallery);
|
|
||||||
response.success("提交更新成功,更新页数:" + newGallery.getPages());
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
response.failure("存在更新,但更新失败");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}catch (Exception e){
|
|
||||||
response.failure("获取更新失败");
|
|
||||||
}
|
}
|
||||||
|
if (!response.isSuccess())
|
||||||
|
log.info(response.getData());
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -473,126 +446,76 @@ public class GalleryManageService {
|
|||||||
CustomConfiguration lastResetAmountTime = configurationMapper.selectConfiguration(CustomConfiguration.LAST_RESET_AMOUNT_TIME);
|
CustomConfiguration lastResetAmountTime = configurationMapper.selectConfiguration(CustomConfiguration.LAST_RESET_AMOUNT_TIME);
|
||||||
Map<String, String> data = new HashMap<>();
|
Map<String, String> data = new HashMap<>();
|
||||||
|
|
||||||
data.put("weekUsedAmount", CustomUtil.fileSizeToString(Long.parseLong(weekUsedAmount.getValue())));
|
// 配置行缺失时给出默认值,避免 NPE 让用量接口整体不可用。
|
||||||
data.put("lastResetAmountTime", lastResetAmountTime.getValue());
|
String usedValue = weekUsedAmount == null || weekUsedAmount.getValue() == null
|
||||||
|
? "0" : weekUsedAmount.getValue();
|
||||||
|
long used;
|
||||||
|
try {
|
||||||
|
used = Long.parseLong(usedValue);
|
||||||
|
} catch (NumberFormatException e) {
|
||||||
|
log.warn("每周用量配置值非法,按 0 处理: {}", usedValue);
|
||||||
|
used = 0L;
|
||||||
|
}
|
||||||
|
data.put("weekUsedAmount", CustomUtil.fileSizeToString(used));
|
||||||
|
data.put("lastResetAmountTime",
|
||||||
|
lastResetAmountTime == null || lastResetAmountTime.getValue() == null
|
||||||
|
? "" : lastResetAmountTime.getValue());
|
||||||
|
|
||||||
response.success(new ObjectMapper().valueToTree(data).toString());
|
response.success(objectMapper.valueToTree(data));
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public String cacheImagesKey(String url) {
|
||||||
/**
|
try {
|
||||||
* 获取缩略图 (直接按照名字路径获取,后期可能会加入缓存)
|
GidToKey index = galleryIndexService.ensureIndex(url);
|
||||||
* @param request 请求对象
|
return Response.generateResponse().success(objectMapper.valueToTree(index)).toJSONString();
|
||||||
* @param response 响应对象
|
} catch (com.lion.lionwebsite.Exception.GalleryIndexException error) {
|
||||||
* @param name 本子名字
|
String message = switch (error.getReason()) {
|
||||||
*/
|
case INVALID_LINK -> "链接错误";
|
||||||
public void getThumbnail(HttpServletRequest request, HttpServletResponse response, String name){
|
case UNAVAILABLE -> "该图片已下架或已被删除";
|
||||||
File thumbnail = new File(TargetPath + name, "thumbnail.webp");
|
case MALFORMED, TRANSIENT -> "网络波动或其他异常";
|
||||||
|
};
|
||||||
if(!thumbnail.exists())
|
log.warn("缓存图片索引失败 reason={}", error.getReason());
|
||||||
if(remoteService.cachePreview(name, (short) 0, thumbnail) != 0){
|
return Response._failure(message);
|
||||||
fourZeroFour(response);
|
} catch (RuntimeException error) {
|
||||||
return;
|
log.warn("缓存图片索引失败 errorType={}", error.getClass().getSimpleName());
|
||||||
}
|
return Response._failure("网络波动或其他异常");
|
||||||
|
}
|
||||||
FileDownload.export(request, response, thumbnail.getAbsolutePath());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
public Callable<?> getCachedImage(String gid, Integer page, HttpServletRequest request, HttpServletResponse response) {
|
||||||
* 获取在线图片
|
Path directory = Path.of(cachePath, gid);
|
||||||
* @param gid gid
|
String name = String.valueOf(page);
|
||||||
* @param page 文件名
|
Path cached = ImageFileCache.find(directory, name);
|
||||||
* @param request 请求对象
|
if (cached != null) {
|
||||||
* @param response 响应对象
|
FileDownload.export(request, response, cached.toString());
|
||||||
*/
|
return null;
|
||||||
public void getOnlineImage(Integer gid, Short page, HttpServletRequest request, HttpServletResponse response){
|
}
|
||||||
String name;
|
return () -> {
|
||||||
|
if (response.isCommitted()) return null;
|
||||||
//本子名缓存
|
try {
|
||||||
if((name = gid2name_cache.get(gid)) == null) //内存
|
Path image = ImageFileCache.get(directory, name, () -> {
|
||||||
if((name = galleryMapper.selectGalleryNameByGid(gid)) == null) { //数据库
|
GidToKey gidToKey = imageCacheMapper.selectKeyByGid(gid);
|
||||||
fourZeroFour(response);
|
ImageKeyCache imageKey = imageCacheMapper.selectImageKeyCacheByGidAndPage(gid, page);
|
||||||
return;
|
if (gidToKey == null || imageKey == null)
|
||||||
|
throw new IOException("图片索引不存在");
|
||||||
|
for (int attempt = 0; attempt < 2; attempt++) {
|
||||||
|
String url = GalleryUtil.getImageUrl(getMpvKey(gidToKey.toUrl()), imageKey);
|
||||||
|
if (url != null) return url;
|
||||||
|
GalleryUtil.refreshMpvKey(gidToKey.toUrl());
|
||||||
|
}
|
||||||
|
throw new IOException("无法获取图片地址");
|
||||||
|
});
|
||||||
|
FileDownload.export(request, response, image.toString());
|
||||||
|
} catch (InterruptedException e) {
|
||||||
|
Thread.currentThread().interrupt();
|
||||||
|
if (!response.isCommitted()) response.sendError(503);
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.warn("获取在线图片失败: gid={} page={} errorType={}", gid, page, e.getClass().getSimpleName());
|
||||||
|
if (!response.isCommitted()) response.sendError(404);
|
||||||
}
|
}
|
||||||
else
|
return null;
|
||||||
gid2name_cache.put(gid, name);
|
};
|
||||||
|
|
||||||
//页数缓存
|
|
||||||
Integer real_page;
|
|
||||||
if((real_page = gid2page_cache.get(gid)) == null)
|
|
||||||
real_page = galleryMapper.selectGalleryByGid(gid).getPages();
|
|
||||||
|
|
||||||
//判断页数是否超出
|
|
||||||
if(real_page < page || page < 0){
|
|
||||||
fourZeroFour(response);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
//页名缓存
|
|
||||||
String pageName;
|
|
||||||
if((pageName = page2pageNameCache.get(gid + String.valueOf(page))) == null) //内存
|
|
||||||
if((pageName = pageNameCacheMapper.selectPageName(gid, page)) == null) //数据库
|
|
||||||
if ((pageName = remoteService.queryPageName(name, page)) == null) { //远程查询
|
|
||||||
fourZeroFour(response);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
//插入数据库
|
|
||||||
else
|
|
||||||
pageNameCacheMapper.insertPageNameCache(new PageNameCache(gid, page, pageName));
|
|
||||||
//插入缓存
|
|
||||||
else
|
|
||||||
page2pageNameCache.put(gid + String.valueOf(page), pageName);
|
|
||||||
|
|
||||||
|
|
||||||
//图片缓存
|
|
||||||
File file = new File(TargetPath, name + "/" + pageName);
|
|
||||||
if(!file.exists()) //硬盘
|
|
||||||
if(remoteService.cachePreview(name, page, file) != 0) { //远程
|
|
||||||
fourZeroFour(response);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
FileDownload.export(request, response, file.getAbsolutePath());
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
public String shareGallery(Integer gid, Integer expireHour){
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
|
||||||
Map<String, String> jsonObject = new HashMap<>();
|
|
||||||
if(gallery == null){
|
|
||||||
response.failure("本子不存在");
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
File file = new File(TargetPath + gallery.getName() + ".zip");
|
|
||||||
|
|
||||||
if(!file.isFile()){
|
|
||||||
response.failure("本子文件不存在");
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
ShareFile shareFile = shareFileMapper.selectShareFileByFilePath(file.getAbsolutePath());
|
|
||||||
if(shareFile != null){
|
|
||||||
jsonObject.put("shareCode", shareFile.getShareCode());
|
|
||||||
jsonObject.put("expireTime", dateTimeFormatter.format(LocalDateTime.ofInstant(shareFile.getExpireTime().toInstant(), ZoneId.systemDefault())));
|
|
||||||
response.success(new ObjectMapper().valueToTree(jsonObject).toString());
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
String ShareCode = RandomUtil.randomString(8);
|
|
||||||
Calendar expireTime = Calendar.getInstance();
|
|
||||||
|
|
||||||
expireTime.add(Calendar.HOUR, expireHour);
|
|
||||||
shareFileMapper.insertShareFile(ShareCode, file.getAbsolutePath(), expireTime.getTime());
|
|
||||||
jsonObject.put("shareCode", ShareCode);
|
|
||||||
jsonObject.put("expireTime", dateTimeFormatter.format(expireTime.getTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
|
|
||||||
response.success(new ObjectMapper().valueToTree(jsonObject).toString());
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public String resetUndone(){
|
public String resetUndone(){
|
||||||
@@ -605,22 +528,50 @@ public class GalleryManageService {
|
|||||||
|
|
||||||
Gallery[] galleries = galleryMapper.selectUnDoneGalleries();
|
Gallery[] galleries = galleryMapper.selectUnDoneGalleries();
|
||||||
if(galleries != null && galleries.length != 0) {
|
if(galleries != null && galleries.length != 0) {
|
||||||
log.info("发送未下载完成本子至节点,{}本", galleries.length);
|
// 只重投「确实已提交上游」的任务:提交未知/预占中/待删除的任务重投会变成
|
||||||
for (Gallery gallery : galleries)
|
// 未计量副作用(节点侧显示下载完成但没有任何额度记录)。
|
||||||
remoteService.addGalleryToQueue(gallery, gallery.getMode());
|
int sent = 0;
|
||||||
response.success(String.format("发送未下载完成本子至节点,%s本", galleries.length));
|
int skipped = 0;
|
||||||
|
for (Gallery gallery : galleries) {
|
||||||
|
if (lifecycleService != null
|
||||||
|
&& !lifecycleService.resolveDispatch(gallery.getGid()).dispatchable()) {
|
||||||
|
skipped++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
remoteService.addGalleryToQueue(gallery);
|
||||||
|
sent++;
|
||||||
|
}
|
||||||
|
log.info("发送未下载完成图片至节点,{}本(跳过需要对账的{}本)", sent, skipped);
|
||||||
|
if (sent == 0)
|
||||||
|
response.failure("没有可安全重投的任务,需先人工对账提交未知的任务");
|
||||||
|
else
|
||||||
|
response.success(String.format("发送未下载完成图片至节点,%s本", sent));
|
||||||
}else{
|
}else{
|
||||||
response.failure("当前没有未下载完成的本子");
|
response.failure("当前没有未下载完成的图片");
|
||||||
}
|
}
|
||||||
|
|
||||||
return response.toJSONString();
|
return response.toJSONString();
|
||||||
}
|
}
|
||||||
|
|
||||||
public static Integer parseGid(String link){
|
public String retryGallery(int gid){
|
||||||
try {
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
return Integer.parseInt(link.split("/g/")[1].split("/")[0]);
|
if(gallery == null)
|
||||||
}catch (IndexOutOfBoundsException e){
|
return Response._failure("任务不存在");
|
||||||
return null;
|
if("下载完成".equals(gallery.getStatus()))
|
||||||
|
return Response._success("下载完成");
|
||||||
|
if(remoteService.isDead())
|
||||||
|
return Response._failure("节点不在线,无法重试");
|
||||||
|
// 提交结果未知(UNKNOWN)或正在删除的任务不能重试:上游不支持幂等,重发会让
|
||||||
|
// 「可能已受理但未计量」的任务在节点侧变成下载完成却 0 额度。这类任务需人工对账。
|
||||||
|
if (lifecycleService != null) {
|
||||||
|
var decision = lifecycleService.resolveDispatch(gid);
|
||||||
|
if (decision.kind() == GalleryTaskLifecycleService.DispatchDecision.Kind.REFUSE)
|
||||||
|
return Response._failure("该任务提交结果未知或正在删除,请先人工对账后再重试");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
RemoteService.RetryResult retryResult = remoteService.retryGallery(gallery);
|
||||||
|
if(retryResult.success())
|
||||||
|
return Response._success(retryResult.message());
|
||||||
|
return Response._failure(retryResult.message());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,783 @@
|
|||||||
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.normal.*;
|
||||||
|
import com.lion.lionwebsite.Domain.*;
|
||||||
|
import com.lion.lionwebsite.Exception.ResolutionNotMatchException;
|
||||||
|
import com.lion.lionwebsite.Message.TaskCommandMessage;
|
||||||
|
import com.lion.lionwebsite.Message.TaskReportMessage;
|
||||||
|
import com.lion.lionwebsite.Util.PreparedGallery;
|
||||||
|
import com.lion.lionwebsite.Util.Response;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.beans.factory.annotation.Qualifier;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
import org.springframework.transaction.support.TransactionTemplate;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 下载任务的持久化生命周期(协议 v2)。
|
||||||
|
*
|
||||||
|
* <p>核心约束:
|
||||||
|
* <ol>
|
||||||
|
* <li><b>先保存意图,再产生外部副作用</b>:同一 gid 的唯一 CREATE 意图先落库(占位任务行 + RESERVED 操作记录),
|
||||||
|
* 通过 gid 的主键唯一约束把并发创建收敛成「一个赢家」;</li>
|
||||||
|
* <li><b>SUBMITTING 之后绝不自动重发</b>:上游不支持幂等,提交结果不确定时只能落 UNKNOWN,等显式对账/解决;</li>
|
||||||
|
* <li><b>提交结果与额度用一个短事务写入</b>:Gallery 与 weekUsedAmount 要么一起成功,要么一起回滚,
|
||||||
|
* 事务提交之后才把 DOWNLOAD 命令下发给节点;</li>
|
||||||
|
* <li><b>删除失败保留记录</b>:先落待删除意图,节点确认(或文件已不存在)后才在同一事务里删任务行并留墓碑;
|
||||||
|
* 失败则保留 Gallery 与 DELETE_PENDING 意图,可沿原记录重试且复用同一删除 operationId。</li>
|
||||||
|
* </ol>
|
||||||
|
*
|
||||||
|
* <p>所有网络调用都发生在事务之外;事务回调内部不做 I/O,也不吞异常(吞掉会错误提交)。
|
||||||
|
*/
|
||||||
|
@Service
|
||||||
|
@Slf4j
|
||||||
|
public class GalleryTaskLifecycleService implements TaskReportHandler {
|
||||||
|
|
||||||
|
final GalleryMapper galleryMapper;
|
||||||
|
|
||||||
|
final CollectMapper collectMapper;
|
||||||
|
|
||||||
|
final CustomConfigurationMapper configurationMapper;
|
||||||
|
|
||||||
|
final UserMapper userMapper;
|
||||||
|
|
||||||
|
final TaskOperationRepository taskOperationRepository;
|
||||||
|
|
||||||
|
final UpstreamGateway upstreamGateway;
|
||||||
|
|
||||||
|
final RemoteService remoteService;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 短全局创建预占锁。
|
||||||
|
*
|
||||||
|
* <p>只有「登记唯一 CREATE 意图」这一小段数据库写入在锁内,网络调用一律在锁外。
|
||||||
|
* 单进程内它把并发同 gid 创建直接收敛成一个赢家;跨进程的正确性仍由
|
||||||
|
* {@code task_operation}/{@code gallery} 的主键唯一约束兜底。
|
||||||
|
*/
|
||||||
|
private final Object reservationLock = new Object();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 测试钩子:进入上游只读解析之前执行。
|
||||||
|
* 用来让并发删除发生在预占之后、SUBMITTING 之前。生产环境是空实现。
|
||||||
|
*/
|
||||||
|
volatile Runnable beforePrepare = () -> { };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 测试钩子:SUBMITTING 已持久化之后、真正 POST 上游之前执行。
|
||||||
|
* 用来制造「提交瞬间任务被删除」的 late generation 场景。生产环境是空实现。
|
||||||
|
*/
|
||||||
|
volatile Runnable afterMarkSubmitting = () -> { };
|
||||||
|
|
||||||
|
/** 主库事务模板;测试可注入内存 sqlite 的事务模板。 */
|
||||||
|
TransactionTemplate transactions;
|
||||||
|
|
||||||
|
public GalleryTaskLifecycleService(GalleryMapper galleryMapper,
|
||||||
|
CollectMapper collectMapper,
|
||||||
|
CustomConfigurationMapper configurationMapper,
|
||||||
|
UserMapper userMapper,
|
||||||
|
TaskOperationRepository taskOperationRepository,
|
||||||
|
UpstreamGateway upstreamGateway,
|
||||||
|
RemoteService remoteService,
|
||||||
|
@Qualifier("mainTransactionTemplate") TransactionTemplate transactions) {
|
||||||
|
this.galleryMapper = galleryMapper;
|
||||||
|
this.collectMapper = collectMapper;
|
||||||
|
this.configurationMapper = configurationMapper;
|
||||||
|
this.userMapper = userMapper;
|
||||||
|
this.taskOperationRepository = taskOperationRepository;
|
||||||
|
this.upstreamGateway = upstreamGateway;
|
||||||
|
this.remoteService = remoteService;
|
||||||
|
this.transactions = transactions;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 测试用:替换事务模板或在没有 Spring 事务时退化为直连执行。 */
|
||||||
|
void setTransactions(TransactionTemplate transactions) {
|
||||||
|
this.transactions = transactions;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 一次「登记创建意图」的结果。
|
||||||
|
*
|
||||||
|
* <p>只有赢家能继续做外部副作用;输家(已存在有效意图 / 数据库竞争失败)
|
||||||
|
* 一律转成业务失败,绝不再 POST 上游。
|
||||||
|
*/
|
||||||
|
private enum ReserveOutcome {
|
||||||
|
/** 已成功登记唯一 CREATE 意图,可以做只读解析与提交。 */
|
||||||
|
RESERVED,
|
||||||
|
/** 同一 gid 已存在有效意图(或任务行)。 */
|
||||||
|
DUPLICATE,
|
||||||
|
/** 登记时的数据库竞争/失败:没有产生任何副作用,必须转业务失败。 */
|
||||||
|
CONTENDED
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 一次「登记创建意图」的返回:结果 + 赢家持有的操作记录。 */
|
||||||
|
private record Reservation(ReserveOutcome outcome, TaskOperation operation) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 单次下发决策。
|
||||||
|
*
|
||||||
|
* <p>把「能不能把这个任务交给节点」从「下载是否已提交」里拆出来:
|
||||||
|
* 未确认上游提交结果(UNKNOWN / 预占 / 待删除 / 删除墓碑)绝不能下发,
|
||||||
|
* 否则未计量的人工对账任务可能在节点侧变成「下载完成却 0 额度」。
|
||||||
|
*/
|
||||||
|
public static final class DispatchDecision {
|
||||||
|
|
||||||
|
public enum Kind {
|
||||||
|
/** 当前 operation 已是 SUBMITTED:可以按稳定 operationId 下发。 */
|
||||||
|
READY,
|
||||||
|
/** 迁移前的旧任务行:没有操作记录但历史上确实提交过下载。 */
|
||||||
|
LEGACY,
|
||||||
|
/** 不可下发:状态不明或被删除,下发会产生未计量副作用。 */
|
||||||
|
REFUSE
|
||||||
|
}
|
||||||
|
|
||||||
|
private final Kind kind;
|
||||||
|
|
||||||
|
private final String operationId;
|
||||||
|
|
||||||
|
private DispatchDecision(Kind kind, String operationId) {
|
||||||
|
this.kind = kind;
|
||||||
|
this.operationId = operationId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Kind kind() {
|
||||||
|
return kind;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String operationId() {
|
||||||
|
return operationId;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 只有 READY/LEGACY 允许真正下发到节点。 */
|
||||||
|
public boolean dispatchable() {
|
||||||
|
return kind != Kind.REFUSE && operationId != null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 拒绝下发(供调用方与测试构造)。 */
|
||||||
|
public static DispatchDecision refuse() {
|
||||||
|
return REFUSE;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 可直接下发(供测试构造)。 */
|
||||||
|
public static DispatchDecision ready(String operationId) {
|
||||||
|
return new DispatchDecision(Kind.READY, operationId);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static final DispatchDecision REFUSE = new DispatchDecision(DispatchDecision.Kind.REFUSE, null);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 一次删除登记的返回。
|
||||||
|
*
|
||||||
|
* <p>用显式结果替代「null 即退避到裸删除」的旧语义:
|
||||||
|
* 被拒绝时调用方必须回业务失败,不能绕过待删除意图直接删节点文件。
|
||||||
|
*/
|
||||||
|
public record DeleteIntent(boolean accepted, String deleteOperationId, String message) {
|
||||||
|
|
||||||
|
static DeleteIntent refused(String message) {
|
||||||
|
return new DeleteIntent(false, null, message);
|
||||||
|
}
|
||||||
|
|
||||||
|
static DeleteIntent accepted(String deleteOperationId) {
|
||||||
|
return new DeleteIntent(true, deleteOperationId, null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 在事务里执行;没有配置事务模板时候直接执行(仅便于构造 mock 测试)。 */
|
||||||
|
private <T> T inTransaction(java.util.function.Supplier<T> body) {
|
||||||
|
TransactionTemplate template = transactions;
|
||||||
|
if (template == null)
|
||||||
|
return body.get();
|
||||||
|
return template.execute(status -> {
|
||||||
|
try {
|
||||||
|
return body.get();
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
// 回调里必须把异常继续抛出,否则 Spring 会把只读了一半的事务当成正常提交。
|
||||||
|
status.setRollbackOnly();
|
||||||
|
throw e;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 创建任务的持久化前置:在同一个事务里登记唯一 CREATE 意图。
|
||||||
|
*
|
||||||
|
* <p>同时写入可见的占位任务行(状态「提交中」),这样提交期间任务列表就能解析该 gid,
|
||||||
|
* 不会出现「上游已受理但本地查不到」的空窗。
|
||||||
|
*
|
||||||
|
* @return 已登记意图(赢家)返回它;同一 gid 已有有效意图返回 {@code null}(输家)。
|
||||||
|
*/
|
||||||
|
private Reservation reserveCreateIntent(int gid, String placeholderName, int downloaderId) {
|
||||||
|
// 锁内只做数据库写入:把并发同 gid 创建串行化,网络调用留在锁外。
|
||||||
|
synchronized (reservationLock) {
|
||||||
|
try {
|
||||||
|
TaskOperation operation = inTransaction(() -> {
|
||||||
|
TaskOperation existingOperation = taskOperationRepository.selectByGid(gid);
|
||||||
|
Gallery existingGallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
if (existingOperation != null) {
|
||||||
|
// 只有「删除已完成」的墓碑才允许用全新 operationId 重新创建(显式重建)。
|
||||||
|
if (!TaskOperation.STATE_DELETED.equals(existingOperation.getState()))
|
||||||
|
return null;
|
||||||
|
if (existingGallery != null)
|
||||||
|
return null;
|
||||||
|
String operationId = TaskProtocol.newDownloadOperationId();
|
||||||
|
taskOperationRepository.reactivate(gid, operationId, TaskOperation.STATE_RESERVED, now());
|
||||||
|
insertPlaceholder(gid, placeholderName, downloaderId);
|
||||||
|
return taskOperationRepository.selectByGid(gid);
|
||||||
|
}
|
||||||
|
if (existingGallery != null)
|
||||||
|
return null;
|
||||||
|
TaskOperation created = new TaskOperation();
|
||||||
|
created.setGid(gid);
|
||||||
|
created.setOperationId(TaskProtocol.newDownloadOperationId());
|
||||||
|
created.setState(TaskOperation.STATE_RESERVED);
|
||||||
|
created.setUpdateTime(now());
|
||||||
|
taskOperationRepository.insert(created);
|
||||||
|
insertPlaceholder(gid, placeholderName, downloaderId);
|
||||||
|
return created;
|
||||||
|
});
|
||||||
|
return operation == null
|
||||||
|
? new Reservation(ReserveOutcome.DUPLICATE, null)
|
||||||
|
: new Reservation(ReserveOutcome.RESERVED, operation);
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
// 数据库竞争/失败发生在任何外部副作用之前,绝不能穿透成 500,也不能继续提交。
|
||||||
|
log.warn("登记创建意图失败,按业务失败处理 gid={} error={}", gid, e.getClass().getSimpleName());
|
||||||
|
return new Reservation(ReserveOutcome.CONTENDED, null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 写入占位任务行:让提交期间的任务在列表里可见。 */
|
||||||
|
private void insertPlaceholder(int gid, String name, int downloaderId) {
|
||||||
|
Gallery placeholder = new Gallery();
|
||||||
|
placeholder.setGid(gid);
|
||||||
|
placeholder.setName(name);
|
||||||
|
placeholder.setStatus(TaskStatus.PENDING_SUBMIT);
|
||||||
|
placeholder.setDownloader(downloaderId);
|
||||||
|
placeholder.setCreateTime(System.currentTimeMillis() / 1000);
|
||||||
|
placeholder.setProceeding(0);
|
||||||
|
galleryMapper.insertGallery(placeholder);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 标记为正在提交,并用 CAS 确认这一代意图仍然有效。
|
||||||
|
*
|
||||||
|
* <p>只有当前状态与 operationId 都匹配(RESERVED → SUBMITTING)才返回 true;
|
||||||
|
* 期间被删除或重建返回 false,调用方必须停下,绝不能再 POST 上游。
|
||||||
|
*
|
||||||
|
* @return true 表示已持久化 SUBMITTING,可以安全产生上游副作用
|
||||||
|
*/
|
||||||
|
boolean markSubmitting(String operationId, int gid) {
|
||||||
|
return inTransaction(() -> taskOperationRepository.compareAndSetCreateState(gid, operationId,
|
||||||
|
TaskOperation.STATE_RESERVED, TaskOperation.STATE_SUBMITTING, now()) == 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 提交结果未知(超时/断连/中断)。保留意图,等待人工对账,不自动重发。
|
||||||
|
*
|
||||||
|
* <p>TaskOperation 与 Gallery 状态在同一个短事务里更新:
|
||||||
|
* 操作落 UNKNOWN 而任务行仍显示「提交中」会让运营侧误以为还能自动重试,
|
||||||
|
* 因此两边必须一致地表达「不确定,需要人工对账」。
|
||||||
|
*/
|
||||||
|
void markUnknown(String operationId, int gid) {
|
||||||
|
inTransaction(() -> {
|
||||||
|
int advanced = taskOperationRepository.compareAndSetCreateState(gid, operationId,
|
||||||
|
TaskOperation.STATE_SUBMITTING, TaskOperation.STATE_UNKNOWN, now());
|
||||||
|
// 只有本次这一代确实从 SUBMITTING 推进到 UNKNOWN 时才同步任务行:
|
||||||
|
// 若期间已被删除/重建(CAS 返回 0),绝不能再改那条记录。
|
||||||
|
if (advanced != 1)
|
||||||
|
return null;
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
if (gallery != null && !TaskStatus.UNKNOWN.equals(gallery.getStatus()))
|
||||||
|
galleryMapper.updateGalleryStatus(gid, TaskStatus.UNKNOWN);
|
||||||
|
return null;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 提交成功后的「一次性短事务」:写入 Gallery 行、累计周用量、把操作推进为 SUBMITTED。
|
||||||
|
* 三者同一个事务,任何一个失败都一起回滚,避免出现「任务存在但用量未计」这类不一致。
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* 提交成功后的落库(带世代校验)。
|
||||||
|
*
|
||||||
|
* @param expectedOperationId 本次提交所归属的 operationId;只有它仍是当前有效意图时才能落库,
|
||||||
|
* 否则(被删除/重建)必须抛错,避免复活已删 gid 或污染新世代
|
||||||
|
*/
|
||||||
|
int persistSubmitted(Gallery gallery, int downloaderId, String expectedOperationId) {
|
||||||
|
return inTransaction(() -> {
|
||||||
|
TaskOperation operation = taskOperationRepository.selectByGid(gallery.getGid());
|
||||||
|
// 严格匹配本次世代:任务被删除、被重建或换了 operationId 时都不能落库。
|
||||||
|
if (operation == null || TaskOperation.STATE_DELETED.equals(operation.getState())
|
||||||
|
|| TaskOperation.STATE_DELETE_PENDING.equals(operation.getState()))
|
||||||
|
throw new IllegalStateException("任务已删除或没有有效创建意图,禁止落库复活 gid=" + gallery.getGid());
|
||||||
|
if (expectedOperationId == null || !expectedOperationId.equals(operation.getOperationId()))
|
||||||
|
throw new IllegalStateException("创建意图世代不匹配,禁止落库 gid=" + gallery.getGid());
|
||||||
|
// 幂等:同一意图重复落库(例如上游已提交后重跑)不得把用量累计两次。
|
||||||
|
boolean alreadyCounted = TaskOperation.STATE_SUBMITTED.equals(operation.getState());
|
||||||
|
gallery.setDownloader(downloaderId);
|
||||||
|
gallery.set_download(true);
|
||||||
|
if (!TaskStatus.SUBMITTED.equals(gallery.getStatus()))
|
||||||
|
gallery.setStatus(TaskStatus.SUBMITTED);
|
||||||
|
Gallery existing = galleryMapper.selectGalleryByGid(gallery.getGid());
|
||||||
|
if (existing == null)
|
||||||
|
galleryMapper.insertGallery(gallery);
|
||||||
|
else
|
||||||
|
galleryMapper.updateGallery(gallery);
|
||||||
|
if (!alreadyCounted)
|
||||||
|
configurationMapper.incrementConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, gallery.getFileSize());
|
||||||
|
taskOperationRepository.updateState(gallery.getGid(), TaskOperation.STATE_SUBMITTED, now());
|
||||||
|
return 1;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 预占式创建:登记 gid 唯一意图(提交任务行占位)→ 只读解析上游 → 标 SUBMITTING → 真正提交。
|
||||||
|
*
|
||||||
|
* <p>上游提交发生在事务之外,且只在 SUBMITTING 已持久化之后执行一次。网络异常一律落 UNKNOWN。
|
||||||
|
*/
|
||||||
|
public Response createTask(int gid, String placeholderName, String link, String targetResolution, int downloaderId) {
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
Reservation reservation = reserveCreateIntent(gid, placeholderName, downloaderId);
|
||||||
|
if (reservation.outcome() == ReserveOutcome.DUPLICATE) {
|
||||||
|
response.failure("任务队列已有此任务,请点击查找任务");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
if (reservation.outcome() == ReserveOutcome.CONTENDED) {
|
||||||
|
// 预占阶段的数据库竞争不产生任何副作用;转业务失败,绝不下发上游、绝不穿透成 500。
|
||||||
|
response.failure("任务提交失败,请稍后重试");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 解析上游是只读步骤;真正的 POST 只在 SUBMITTING 持久化之后发生。
|
||||||
|
// 生命周期约束:RESERVED 阶段被中断是安全的(可释放重来),SUBMITTING 之后一律按 UNKNOWN 处理。
|
||||||
|
return submitPreparedIntent(gid, reservation.operation(), link, targetResolution, downloaderId);
|
||||||
|
}
|
||||||
|
|
||||||
|
private Response submitPreparedIntent(int gid, TaskOperation operation, String link, String targetResolution,
|
||||||
|
int downloaderId) {
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
beforePrepare.run();
|
||||||
|
PreparedGallery prepared;
|
||||||
|
try {
|
||||||
|
prepared = upstreamGateway.prepare(link);
|
||||||
|
} catch (IOException e) {
|
||||||
|
log.warn("解析任务链接失败 gid={}", gid, e);
|
||||||
|
releaseReservation(gid, operation.getOperationId());
|
||||||
|
response.failure("IO错误,可能是网络波动");
|
||||||
|
return response;
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
log.warn("解析任务链接异常 gid={}", gid, e);
|
||||||
|
releaseReservation(gid, operation.getOperationId());
|
||||||
|
response.failure("提交任务失败,未知原因,请检查链接是否正常");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
if (prepared == null || prepared.gallery() == null) {
|
||||||
|
releaseReservation(gid, operation.getOperationId());
|
||||||
|
response.failure("提交任务失败,未知原因,请检查链接是否正常");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
Gallery gallery = prepared.gallery();
|
||||||
|
gallery.setGid(gid);
|
||||||
|
|
||||||
|
// CAS 确认这一代意图仍然有效(未被并发删除/重建),并把状态持久化为 SUBMITTING。
|
||||||
|
// 只有成功之后才允许真正 POST;失败说明任务已不在当前生命周期里,直接放弃。
|
||||||
|
if (!markSubmitting(operation.getOperationId(), gid)) {
|
||||||
|
log.warn("创建意图已失效(可能被并发删除),放弃提交 gid={} operationId={}",
|
||||||
|
gid, operation.getOperationId());
|
||||||
|
response.failure("任务已被删除或状态已变化,请刷新任务列表");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
afterMarkSubmitting.run();
|
||||||
|
|
||||||
|
Gallery submitted;
|
||||||
|
try {
|
||||||
|
submitted = upstreamGateway.submit(prepared, targetResolution);
|
||||||
|
} catch (ResolutionNotMatchException e) {
|
||||||
|
releaseReservation(gid, operation.getOperationId());
|
||||||
|
response.failure("提交失败,分辨率不存在");
|
||||||
|
return response;
|
||||||
|
} catch (IOException | RuntimeException e) {
|
||||||
|
// 超时或断连时上游可能已经受理,无法确认:只能记 UNKNOWN,绝不自动重发。
|
||||||
|
log.warn("上游提交结果未知 gid={} operationId={}", gid, operation.getOperationId(), e);
|
||||||
|
markUnknown(operation.getOperationId(), gid);
|
||||||
|
response.failure("上游提交结果未知,请稍后在任务列表中确认,不要重复提交");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
if (submitted == null || !TaskStatus.SUBMITTED.equals(submitted.getStatus())) {
|
||||||
|
markUnknown(operation.getOperationId(), gid);
|
||||||
|
response.failure("上游提交结果未知,请稍后在任务列表中确认,不要重复提交");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
submitted.setGid(gid);
|
||||||
|
try {
|
||||||
|
persistSubmitted(submitted, downloaderId, operation.getOperationId());
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
// 上游已受理但本地落库失败:保留 SUBMITTING 意图并标 UNKNOWN,任务行仍在列表里可对账。
|
||||||
|
log.error("提交成功后落库失败 gid={} operationId={}", gid, operation.getOperationId(), e);
|
||||||
|
markUnknown(operation.getOperationId(), gid);
|
||||||
|
response.failure("任务已提交,但本地保存失败,请在任务列表中确认");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
// 只有事务提交之后才向节点下发命令;重复下发由节点按 operationId 去重。
|
||||||
|
byte dispatchResult = dispatchDownload(gid, submitted.getName());
|
||||||
|
Gallery current = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
Gallery payload = current == null ? submitted : current;
|
||||||
|
if (dispatchResult == 0)
|
||||||
|
response.success(objectMapper.valueToTree(payload));
|
||||||
|
else
|
||||||
|
response.failure("任务已保存,但节点未确认接收;请刷新任务列表后重试");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 提交成功后的落库 + 下发。落库(含用量)在一个短事务里完成,事务提交后才向节点发命令。
|
||||||
|
*
|
||||||
|
* <p>先解析当前世代再落库:世代不存在(未提交过 / 已删除)时不落库也不下发。
|
||||||
|
*
|
||||||
|
* @return 已持久化的 Gallery;数据库失败或世代无效时抛出异常,调用方回业务失败且不下发节点。
|
||||||
|
*/
|
||||||
|
public Gallery persistAndDispatch(Gallery gallery, int downloaderId) {
|
||||||
|
DispatchDecision decision = resolveDispatch(gallery.getGid());
|
||||||
|
if (!decision.dispatchable())
|
||||||
|
throw new IllegalStateException("任务没有可下发的有效世代 gid=" + gallery.getGid());
|
||||||
|
persistSubmitted(gallery, downloaderId, decision.operationId());
|
||||||
|
dispatchDownload(gallery.getGid(), gallery.getName());
|
||||||
|
return gallery;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 向节点下发 DOWNLOAD 命令(复用当前有效 operationId),返回节点结果码。 */
|
||||||
|
public byte dispatchDownload(int gid, String name) {
|
||||||
|
DispatchDecision decision = resolveDispatch(gid);
|
||||||
|
if (!decision.dispatchable())
|
||||||
|
return -1;
|
||||||
|
return remoteService.sendTaskCommand(TaskCommandMessage.ACTION_DOWNLOAD, decision.operationId(), gid, name);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 判定一个 gid 能否下发给节点,并为迁移前的旧任务补齐稳定 operationId。
|
||||||
|
*
|
||||||
|
* <p>规则:
|
||||||
|
* <ul>
|
||||||
|
* <li>v2 任务只有 {@code SUBMITTED} 可下发;{@code UNKNOWN} 必须人工对账,
|
||||||
|
* 否则未计量的未知任务会在节点侧变成「下载完成却 0 额度」;</li>
|
||||||
|
* <li>预占中的 {@code RESERVED}/{@code SUBMITTING}、待删除与删除墓碑都不下发;</li>
|
||||||
|
* <li>没有任何操作记录、但任务行确实存在且状态已是「已提交」及之后的历史任务,
|
||||||
|
* 补建一个稳定 operationId 后再下发(这是唯一的 legacy 兼容路径)。</li>
|
||||||
|
* </ul>
|
||||||
|
*/
|
||||||
|
public DispatchDecision resolveDispatch(int gid) {
|
||||||
|
TaskOperation operation = taskOperationRepository.selectByGid(gid);
|
||||||
|
if (operation != null && TaskOperation.STATE_SUBMITTED.equals(operation.getState())
|
||||||
|
&& operation.getOperationId() != null)
|
||||||
|
return new DispatchDecision(DispatchDecision.Kind.READY, operation.getOperationId());
|
||||||
|
// 没有操作记录,或只有「旧世代」记录(缺 operationId):必须先确认任务行确实已提交过,
|
||||||
|
// 再补一个稳定 operationId;UNKNOWN/预占/待删除一律拒绝。
|
||||||
|
if (operation != null && !TaskOperation.STATE_SUBMITTED.equals(operation.getState()))
|
||||||
|
return REFUSE;
|
||||||
|
if (!isLegacySubmitted(gid))
|
||||||
|
return REFUSE;
|
||||||
|
String freshId = ensureStableOperationId(gid);
|
||||||
|
return freshId == null ? REFUSE : new DispatchDecision(DispatchDecision.Kind.LEGACY, freshId);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 迁移前的旧任务行:没有操作记录,但状态表明上游确实已受理过下载,可安全重投。 */
|
||||||
|
private boolean isLegacySubmitted(int gid) {
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
if (gallery == null || gallery.getStatus() == null)
|
||||||
|
return false;
|
||||||
|
return switch (gallery.getStatus()) {
|
||||||
|
case TaskStatus.SUBMITTED, TaskStatus.DOWNLOADING, TaskStatus.WAITING_COMPRESS,
|
||||||
|
TaskStatus.COMPRESSING -> true;
|
||||||
|
// 提交未知 / 提交中 / 待删除 / 删除失败 一律不算已提交,不能下发。
|
||||||
|
default -> false;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 为一个已经真正提交过的旧任务持久化稳定 operationId。
|
||||||
|
*
|
||||||
|
* <p>必须是短事务写入、先落库后返回,之后重投/重连才复用同一个 ID;
|
||||||
|
* 用 {@code insert or ignore} + 回读,避免与并发创建互相覆盖。
|
||||||
|
*
|
||||||
|
* @return 稳定 operationId;无法安全建立时返回 null(调用方不下发)
|
||||||
|
*/
|
||||||
|
String ensureStableOperationId(int gid) {
|
||||||
|
// 与创建预占共用同一把短锁(锁内只有数据库写),并发为同一 gid 补 ID 时只能有一个赢家。
|
||||||
|
synchronized (reservationLock) {
|
||||||
|
String candidate = TaskProtocol.newDownloadOperationId();
|
||||||
|
return inTransaction(() -> {
|
||||||
|
if (taskOperationRepository.selectByGid(gid) == null)
|
||||||
|
taskOperationRepository.insertIfAbsent(gid, candidate, TaskOperation.STATE_SUBMITTED, now());
|
||||||
|
else
|
||||||
|
taskOperationRepository.assignOperationIdIfMissing(gid, candidate, now());
|
||||||
|
TaskOperation current = taskOperationRepository.selectByGid(gid);
|
||||||
|
if (current == null || current.getOperationId() == null)
|
||||||
|
return null;
|
||||||
|
// 只有确实处于 SUBMITTED 的世代才可下发;并发写入的其他状态一律拒绝。
|
||||||
|
return TaskOperation.STATE_SUBMITTED.equals(current.getState()) ? current.getOperationId() : null;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 取消预占(解析失败、分辨率不存在等:还没有产生任何上游副作用)。
|
||||||
|
*
|
||||||
|
* <p>只删除与本次创建 intent 匹配的 RESERVED 记录:若期间该 gid 已被删除并重建
|
||||||
|
* (新的 operationId / 墓碑),这里绝不能把新世代记录抹掉。占位行同理只删「提交中」。
|
||||||
|
*/
|
||||||
|
private void releaseReservation(int gid, String operationId) {
|
||||||
|
inTransaction(() -> {
|
||||||
|
// 只删与本代 operationId 匹配且仍停在 RESERVED/SUBMITTING 的记录。
|
||||||
|
int removed = taskOperationRepository.deleteReservationIfSameGeneration(gid, operationId);
|
||||||
|
// 只有确实删掉了本代预占记录,才能删对应的占位行:
|
||||||
|
// 若期间已出现新的 RESERVED 代,删除返回 0,此时绝不能误删新占位。
|
||||||
|
if (removed == 1) {
|
||||||
|
Gallery existing = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
if (existing != null && TaskStatus.PENDING_SUBMIT.equals(existing.getStatus()))
|
||||||
|
galleryMapper.deleteGalleryByGid(gid);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 删除的持久化部分。
|
||||||
|
*
|
||||||
|
* <p>正在创建意图(RESERVED/SUBMITTING)的任务不能删除:此刻 DELETE 与创建 POST 会同时发生,
|
||||||
|
* 随后创建落库还会把已删/重建的 gid 写回。这里直接拒绝,由用户在提交结果明确后再操作。
|
||||||
|
*
|
||||||
|
* @return 显式结果:被拒时调用方必须回业务失败,不能退避到裸删除
|
||||||
|
*/
|
||||||
|
public DeleteIntent beginDelete(int gid) {
|
||||||
|
return inTransaction(() -> {
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
if (gallery == null)
|
||||||
|
return DeleteIntent.refused("删除失败,该图片不存在");
|
||||||
|
TaskOperation operation = taskOperationRepository.selectByGid(gid);
|
||||||
|
if (operation != null && (TaskOperation.STATE_RESERVED.equals(operation.getState())
|
||||||
|
|| TaskOperation.STATE_SUBMITTING.equals(operation.getState())))
|
||||||
|
return DeleteIntent.refused("任务正在提交中,请等待提交结果明确后再删除");
|
||||||
|
if (operation == null) {
|
||||||
|
// 历史任务行没有操作记录:直接落一条 DELETE_PENDING 意图,避免中间态缺字段。
|
||||||
|
TaskOperation fresh = new TaskOperation();
|
||||||
|
fresh.setGid(gid);
|
||||||
|
fresh.setOperationId(TaskProtocol.newDownloadOperationId());
|
||||||
|
fresh.setDeleteOperationId(TaskProtocol.newDeleteOperationId());
|
||||||
|
fresh.setState(TaskOperation.STATE_DELETE_PENDING);
|
||||||
|
fresh.setUpdateTime(now());
|
||||||
|
taskOperationRepository.insert(fresh);
|
||||||
|
galleryMapper.updateGalleryStatus(gid, TaskStatus.DELETE_PENDING);
|
||||||
|
return DeleteIntent.accepted(fresh.getDeleteOperationId());
|
||||||
|
}
|
||||||
|
if (operation.getDeleteOperationId() == null) {
|
||||||
|
operation.setDeleteOperationId(TaskProtocol.newDeleteOperationId());
|
||||||
|
operation.setUpdateTime(now());
|
||||||
|
}
|
||||||
|
taskOperationRepository.updateDeleteIntent(gid, operation.getDeleteOperationId(),
|
||||||
|
TaskOperation.STATE_DELETE_PENDING, now());
|
||||||
|
galleryMapper.updateGalleryStatus(gid, TaskStatus.DELETE_PENDING);
|
||||||
|
return DeleteIntent.accepted(operation.getDeleteOperationId());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 删除的完成部分:节点已确认(或文件已不存在)后,在一个短事务里删任务行、收藏并留墓碑。
|
||||||
|
*
|
||||||
|
* <p>事务提交前若失败,任务行与 DELETE_PENDING 意图都保留,可以继续重试。
|
||||||
|
* 删除意图 ID 必须匹配(防止并发删除/重建时抹掉别的一代),随后只把 DELETE_PENDING
|
||||||
|
* 世代推进为 DELETED;墓碑(DELETED 且无删除意图)不允许在这里被重新激活。
|
||||||
|
*
|
||||||
|
* @return true 表示本次删除意图已完成并留下墓碑
|
||||||
|
*/
|
||||||
|
public boolean completeDelete(int gid, String deleteOperationId) {
|
||||||
|
return inTransaction(() -> {
|
||||||
|
TaskOperation operation = taskOperationRepository.selectByGid(gid);
|
||||||
|
if (operation == null || operation.getDeleteOperationId() == null
|
||||||
|
|| !operation.getDeleteOperationId().equals(deleteOperationId))
|
||||||
|
return false;
|
||||||
|
// 先清收藏再删任务行:收藏一侧在部分历史库上带有指向 gallery 的外键。
|
||||||
|
collectMapper.deleteByGid(gid);
|
||||||
|
galleryMapper.deleteGalleryByGid(gid);
|
||||||
|
int updated = taskOperationRepository.completeDeleteIfSameGeneration(gid, deleteOperationId,
|
||||||
|
TaskOperation.STATE_DELETED, now());
|
||||||
|
return updated == 1;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 删除失败:保留任务行并记一条可重试的失败状态。 */
|
||||||
|
public void failDelete(int gid) {
|
||||||
|
inTransaction(() -> {
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(gid);
|
||||||
|
if (gallery == null)
|
||||||
|
return null;
|
||||||
|
gallery.setStatus(TaskStatus.DELETE_FAILED);
|
||||||
|
galleryMapper.updateGallery(gallery);
|
||||||
|
return null;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 启动恢复:把上次中断在 SUBMITTING 的意图判为 UNKNOWN,并安全释放还停在 RESERVED 的预占。
|
||||||
|
*
|
||||||
|
* <p>这段恢复必须发生在任何自动重投之前:
|
||||||
|
* <ul>
|
||||||
|
* <li>{@code SUBMITTING}:上游可能已经受理但本地无法确认,只能落 UNKNOWN(操作与任务行同时
|
||||||
|
* 更新为「提交未知」),交人工对账,绝不自动再 POST;</li>
|
||||||
|
* <li>{@code RESERVED}:还没有产生任何上游副作用,重启后可以直接释放预占
|
||||||
|
* (删掉匹配的 RESERVED 记录与「提交中」占位行),让用户重新提交。</li>
|
||||||
|
* </ul>
|
||||||
|
*
|
||||||
|
* @return 本次恢复(判未知 + 释放预占)处理的任务数
|
||||||
|
*/
|
||||||
|
public int recoverInterruptedSubmissions() {
|
||||||
|
return inTransaction(() -> {
|
||||||
|
int recovered = 0;
|
||||||
|
for (TaskOperation operation : taskOperationRepository.selectByState(TaskOperation.STATE_SUBMITTING)) {
|
||||||
|
int advanced = taskOperationRepository.compareAndSetCreateState(operation.getGid(), operation.getOperationId(),
|
||||||
|
TaskOperation.STATE_SUBMITTING, TaskOperation.STATE_UNKNOWN, now());
|
||||||
|
if (advanced != 1)
|
||||||
|
continue;
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(operation.getGid());
|
||||||
|
if (gallery != null && !TaskStatus.UNKNOWN.equals(gallery.getStatus()))
|
||||||
|
galleryMapper.updateGalleryStatus(operation.getGid(), TaskStatus.UNKNOWN);
|
||||||
|
recovered++;
|
||||||
|
}
|
||||||
|
for (TaskOperation operation : taskOperationRepository.selectByState(TaskOperation.STATE_RESERVED)) {
|
||||||
|
// 还没 POST:释放匹配的预占记录与占位行,允许重启后重新提交。
|
||||||
|
// 只有确实删掉了这条本代 RESERVED 记录,才删其占位行;否则不动(可能已换代)。
|
||||||
|
int removed = taskOperationRepository.deleteReservationIfSameGeneration(operation.getGid(),
|
||||||
|
operation.getOperationId());
|
||||||
|
if (removed == 1) {
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(operation.getGid());
|
||||||
|
if (gallery != null && TaskStatus.PENDING_SUBMIT.equals(gallery.getStatus()))
|
||||||
|
galleryMapper.deleteGalleryByGid(operation.getGid());
|
||||||
|
}
|
||||||
|
recovered++;
|
||||||
|
}
|
||||||
|
return recovered;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 已完成状态上报的处理:丢弃旧 revision、旧 operation、待删除任务与重复上报。 */
|
||||||
|
@Override
|
||||||
|
public CommittedReport apply(TaskReportMessage report) {
|
||||||
|
if (report == null || report.getReportId() == null || report.getGalleryTasks() == null)
|
||||||
|
return null;
|
||||||
|
Applied applied;
|
||||||
|
try {
|
||||||
|
applied = inTransaction(() -> applyInTransaction(report));
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
// 数据库失败:不返回可回执结果,节点会保留快照并稍后重投;失败写入绝不清任务。
|
||||||
|
log.warn("应用节点上报失败,未回执 reportId={}", report.getReportId(), e);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (applied == null)
|
||||||
|
return null;
|
||||||
|
return new CommittedReport(report.getReportId(), applied.newlyCompleted(), applied.acceptedTasks());
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 事务内应用上报的结果:新完成的任务与真正被接受的进度更新。 */
|
||||||
|
private record Applied(List<Gallery> newlyCompleted, List<GalleryTask> acceptedTasks) {
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 在一个事务里应用上报。
|
||||||
|
*
|
||||||
|
* @return 非 null 表示已经提交(可以回 ACK,重复上报也会返回结果以停止重放)。
|
||||||
|
*/
|
||||||
|
private Applied applyInTransaction(TaskReportMessage report) {
|
||||||
|
long revision = report.getRevision();
|
||||||
|
List<Gallery> newlyCompleted = new ArrayList<>();
|
||||||
|
List<GalleryTask> acceptedTasks = new ArrayList<>();
|
||||||
|
for (GalleryTask task : report.getGalleryTasks()) {
|
||||||
|
if (task == null)
|
||||||
|
continue;
|
||||||
|
TaskOperation operation = taskOperationRepository.selectByGid(task.getGid());
|
||||||
|
Gallery gallery = galleryMapper.selectGalleryByGid(task.getGid());
|
||||||
|
if (gallery == null) {
|
||||||
|
// 未知 gid(任务已删除):忽略但 ACK,让节点停止重放。
|
||||||
|
log.debug("忽略节点上报的未知任务 gid={}", task.getGid());
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
String expectedOperationId = report.getOperationIds() == null
|
||||||
|
? null : report.getOperationIds().get(task.getGid());
|
||||||
|
if (operation == null) {
|
||||||
|
// 旧任务行(迁移前创建)没有操作记录:按显式生成策略补建并采纳上报里的 operationId。
|
||||||
|
operation = adoptGeneration(task.getGid(), expectedOperationId);
|
||||||
|
} else if (TaskOperation.STATE_DELETED.equals(operation.getState())) {
|
||||||
|
// 墓碑必须永久忽略旧上报:删除已完成的任务只能由显式 CREATE 重新建立新世代。
|
||||||
|
// (gallery 已被删除时上面的 null 分支已经提前忽略,这里防御性兜底。)
|
||||||
|
log.debug("忽略已删除 gid 的迟到上报 gid={}", task.getGid());
|
||||||
|
continue;
|
||||||
|
} else if (TaskOperation.STATE_DELETE_PENDING.equals(operation.getState())) {
|
||||||
|
log.info("忽略待删除任务的迟到上报 gid={}", task.getGid());
|
||||||
|
continue;
|
||||||
|
} else if (operation.getLastRevision() > 0 && revision <= operation.getLastRevision()) {
|
||||||
|
log.debug("丢弃过期上报 gid={} revision={} lastRevision={}",
|
||||||
|
task.getGid(), revision, operation.getLastRevision());
|
||||||
|
continue;
|
||||||
|
} else if (operation.getOperationId() != null) {
|
||||||
|
// v2 任务必须有归属:缺 operationId 或与当前世代不符都不能改当前任务,
|
||||||
|
// 否则「删除后重建」的新世代会被旧上报污染。
|
||||||
|
if (expectedOperationId == null || !operation.getOperationId().equals(expectedOperationId)) {
|
||||||
|
log.info("丢弃无法归属/旧世代的迟到上报 gid={} 上报={} 当前={}",
|
||||||
|
task.getGid(), expectedOperationId, operation.getOperationId());
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
String status = TaskStatus.fromReport(task.getStatus());
|
||||||
|
if (status == null) {
|
||||||
|
log.warn("未知上报状态码 gid={} status={}", task.getGid(), task.getStatus());
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
gallery.setProceeding(task.getProceeding());
|
||||||
|
if (task.getName() != null)
|
||||||
|
gallery.setName(task.getName());
|
||||||
|
boolean justCompleted = TaskStatus.COMPLETE.equals(status)
|
||||||
|
&& !TaskStatus.COMPLETE.equals(gallery.getStatus());
|
||||||
|
gallery.setStatus(status);
|
||||||
|
galleryMapper.updateGallery(gallery);
|
||||||
|
taskOperationRepository.updateReportReceipt(task.getGid(), revision, report.getReportId(), now());
|
||||||
|
acceptedTasks.add(task);
|
||||||
|
if (justCompleted)
|
||||||
|
newlyCompleted.add(gallery);
|
||||||
|
}
|
||||||
|
return new Applied(newlyCompleted, acceptedTasks);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 为迁移前创建、尚无操作记录的历史任务建立一次「世代」:优先采纳上报里的 operationId;
|
||||||
|
* 旧节点不带 operationId 时保留 null,让后续同世代的进度上报仍能被接受。
|
||||||
|
*
|
||||||
|
* <p>不在这里凭空生成随机 ID:那会让第一份进度上报之后的所有旧上报都因「无法归属」被丢弃。
|
||||||
|
* 真正的稳定 operationId 会在首次自动下发时用 {@code ensureStableOperationId} 落库,
|
||||||
|
* 之后该世代即可被严格匹配。不会覆盖任何已有任务状态。
|
||||||
|
*/
|
||||||
|
private TaskOperation adoptGeneration(int gid, String reportedOperationId) {
|
||||||
|
TaskOperation operation = new TaskOperation();
|
||||||
|
operation.setGid(gid);
|
||||||
|
operation.setOperationId(reportedOperationId);
|
||||||
|
operation.setState(TaskOperation.STATE_SUBMITTED);
|
||||||
|
operation.setUpdateTime(now());
|
||||||
|
taskOperationRepository.insert(operation);
|
||||||
|
return operation;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static long now() {
|
||||||
|
return System.currentTimeMillis() / 1000;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 供装配层解析当前有效 operationId(避免装配代码访问包私有字段)。 */
|
||||||
|
public String currentOperationId(int gid) {
|
||||||
|
TaskOperation operation = taskOperationRepository.selectByGid(gid);
|
||||||
|
return operation == null ? null : operation.getOperationId();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,190 @@
|
|||||||
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
|
||||||
|
import com.lion.lionwebsite.Dao.normal.GalleryMapper;
|
||||||
|
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
||||||
|
import com.lion.lionwebsite.Util.CustomUtil;
|
||||||
|
import com.lion.lionwebsite.Util.GalleryUtil;
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.apache.hc.client5.http.classic.methods.HttpGet;
|
||||||
|
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
|
||||||
|
import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
|
||||||
|
import org.apache.hc.client5.http.impl.classic.HttpClients;
|
||||||
|
import org.apache.hc.core5.http.HttpEntity;
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
|
import org.springframework.scheduling.annotation.Scheduled;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.io.*;
|
||||||
|
import java.net.URLDecoder;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.nio.file.*;
|
||||||
|
import java.nio.file.attribute.BasicFileAttributes;
|
||||||
|
import java.nio.file.attribute.FileTime;
|
||||||
|
import java.time.LocalDateTime;
|
||||||
|
import java.time.format.DateTimeFormatter;
|
||||||
|
import java.util.*;
|
||||||
|
import java.util.regex.Matcher;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
import static java.nio.file.FileVisitResult.CONTINUE;
|
||||||
|
|
||||||
|
@Service
|
||||||
|
@Data
|
||||||
|
@Slf4j
|
||||||
|
public class LocalService{
|
||||||
|
@Value("${local.dou-nai-clash:https://aaaa.gay/link/X7zEqkIx5gtIGugO?client=clashmeta}")
|
||||||
|
String DouNaiClash;
|
||||||
|
|
||||||
|
@Value("${local.dou-nai-v2ray:https://aaaa.gay/link/X7zEqkIx5gtIGugO?client=v2}")
|
||||||
|
String DouNaiV2ray;
|
||||||
|
|
||||||
|
private static final CloseableHttpClient httpClient = HttpClients.createDefault();
|
||||||
|
|
||||||
|
final CustomConfigurationMapper configurationMapper;
|
||||||
|
|
||||||
|
final GalleryMapper galleryMapper;
|
||||||
|
|
||||||
|
final PushService pushService;
|
||||||
|
|
||||||
|
final RemoteService remoteService;
|
||||||
|
|
||||||
|
final SubscriptionRefreshScheduler subscriptionRefreshScheduler;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 检查连接是否有效,如果无效自动重连
|
||||||
|
*/
|
||||||
|
@Scheduled(cron = "0 0/30 * * * *")
|
||||||
|
public void CheckConnectionAvailability(){
|
||||||
|
if (remoteService.isDead()){
|
||||||
|
remoteService.initChannel();
|
||||||
|
pushService.sendToMe("主动检测连接已断开,自动进行重连");
|
||||||
|
log.warn("主动检测连接已断开,自动进行重连");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// -1 为对方没有返回
|
||||||
|
if (remoteService.checkAvailability() == -1) {
|
||||||
|
String result = switch (remoteService.reconnect()){
|
||||||
|
case 0 -> "重连成功";
|
||||||
|
case -1 -> "重连失败";
|
||||||
|
case -2 -> "当前未连接,不进行重连";
|
||||||
|
default -> "未知错误";
|
||||||
|
};
|
||||||
|
pushService.sendToMe("主动检测连接无数据返回,自动进行重连:" + result);
|
||||||
|
log.warn("主动检测连接无数据返回,自动进行重连:{}", result);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 每周周一四点重置额度
|
||||||
|
*/
|
||||||
|
@Scheduled(cron = "0 0 4 * * MON")
|
||||||
|
public void reset() {
|
||||||
|
configurationMapper.updateConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, "0");
|
||||||
|
configurationMapper.updateConfiguration(CustomConfiguration.LAST_RESET_AMOUNT_TIME, CustomUtil.now());
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 每天凌晨测试e-hentai cookie是否过期
|
||||||
|
*/
|
||||||
|
@Scheduled(cron = "0 0 0 * * *")
|
||||||
|
public void verifyCookie(){
|
||||||
|
try {
|
||||||
|
String content = GalleryUtil.requests("https://exhentai.org", "GET", null, null);
|
||||||
|
if(content.trim().isEmpty())
|
||||||
|
pushService.sendToMe("cookie过期");
|
||||||
|
} catch (IOException e) {
|
||||||
|
pushService.sendToMe("检测cookie异常:" + e.getMessage());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 手动「更新订阅」:立即刷新全部启用账号,供管理页按钮调用。
|
||||||
|
*
|
||||||
|
* <p>定时刷新已移交 {@link SubscriptionRefreshScheduler}。原先这里还有一个
|
||||||
|
* {@code @Scheduled(fixedRate = 86400000)} 的定时入口,它会一次性刷新全部账号
|
||||||
|
* (十几个账号在十几秒内打满),且因为没有 initialDelay,每次重启都会立刻重刷一遍,
|
||||||
|
* 使上游看到的请求密度取决于部署频率。该入口已移除。
|
||||||
|
*
|
||||||
|
* <p>不再保留 {@code isManual} 参数:定时路径已不存在,留着它只会让人以为
|
||||||
|
* 「传 false 就是不刷新」,而实际语义是「什么都不做却报成功」。
|
||||||
|
*/
|
||||||
|
public boolean updateSub() {
|
||||||
|
boolean success = subscriptionRefreshScheduler.refreshAllNow();
|
||||||
|
remoteService.requestSubscriptionSync();
|
||||||
|
if (success)
|
||||||
|
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME,
|
||||||
|
CustomUtil.dateTimeFormatter().format(LocalDateTime.now()));
|
||||||
|
return success;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 封装好的get方法
|
||||||
|
* @param url url
|
||||||
|
* @return 请求结果
|
||||||
|
* @throws IOException 网络异常
|
||||||
|
*/
|
||||||
|
public static ArrayList<String> Get(String url) throws IOException {
|
||||||
|
CloseableHttpResponse httpResponse;
|
||||||
|
HttpGet httpGet = new HttpGet(url);
|
||||||
|
|
||||||
|
httpResponse = httpClient.execute(httpGet);
|
||||||
|
|
||||||
|
HttpEntity responseEntity = httpResponse.getEntity();
|
||||||
|
int statusCode = httpResponse.getCode();
|
||||||
|
ArrayList<String> temp = new ArrayList<>();
|
||||||
|
|
||||||
|
if (statusCode == 200) {
|
||||||
|
BufferedReader reader = new BufferedReader(new InputStreamReader(responseEntity.getContent()));
|
||||||
|
String str;
|
||||||
|
while ((str = reader.readLine()) != null)
|
||||||
|
temp.add(str);
|
||||||
|
}
|
||||||
|
|
||||||
|
httpResponse.close();
|
||||||
|
return temp;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 每周,查看缩略图数量,按照访问时间排序,删除超出1w的部分
|
||||||
|
*/
|
||||||
|
@Scheduled(cron = "0 0 4 1 * *")
|
||||||
|
public void clearThumbnailCache(){
|
||||||
|
String cachePath = "/storage/hentaiCache/";
|
||||||
|
File directory = new File(cachePath);
|
||||||
|
// 排序需要文件的最后访问时间。原先在比较器里每次重新读属性(n log n 次系统调用),
|
||||||
|
// 且属性读取失败会让整个排序抛 RuntimeException;这里在遍历时一次性带出属性。
|
||||||
|
List<Map.Entry<Path, FileTime>> files = new ArrayList<>();
|
||||||
|
try {
|
||||||
|
Files.walkFileTree(directory.toPath(), new SimpleFileVisitor<>() {
|
||||||
|
@Override
|
||||||
|
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
|
||||||
|
if (attrs.isRegularFile()) {
|
||||||
|
files.add(Map.entry(file, attrs.lastAccessTime()));
|
||||||
|
}
|
||||||
|
return CONTINUE;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
} catch (IOException e) {
|
||||||
|
throw new RuntimeException(e);
|
||||||
|
}
|
||||||
|
|
||||||
|
files.sort(Map.Entry.comparingByValue());
|
||||||
|
|
||||||
|
if (files.size() > 10000) {
|
||||||
|
List<Map.Entry<Path, FileTime>> toDelete = files.subList(0, files.size() - 10000);
|
||||||
|
for (Map.Entry<Path, FileTime> entry : toDelete) {
|
||||||
|
try {
|
||||||
|
Files.delete(entry.getKey());
|
||||||
|
}catch (IOException e){
|
||||||
|
log.warn("删除缩略图缓存文件失败: {}", entry.getKey(), e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
log.info("Deleted {} files", toDelete.size());
|
||||||
|
} else {
|
||||||
|
log.info("No files to delete");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,297 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Service;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Dao.CustomConfigurationMapper;
|
|
||||||
import com.lion.lionwebsite.Dao.GalleryMapper;
|
|
||||||
import com.lion.lionwebsite.Dao.ShareFileMapper;
|
|
||||||
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
|
||||||
import com.lion.lionwebsite.Domain.ShareFile;
|
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import lombok.Data;
|
|
||||||
import org.apache.http.HttpEntity;
|
|
||||||
import org.apache.http.client.methods.CloseableHttpResponse;
|
|
||||||
import org.apache.http.client.methods.HttpGet;
|
|
||||||
import org.apache.http.impl.client.CloseableHttpClient;
|
|
||||||
import org.apache.http.impl.client.HttpClients;
|
|
||||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
|
||||||
import org.springframework.scheduling.annotation.Scheduled;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
import java.io.*;
|
|
||||||
import java.net.URLDecoder;
|
|
||||||
import java.nio.charset.StandardCharsets;
|
|
||||||
import java.nio.file.*;
|
|
||||||
import java.nio.file.attribute.BasicFileAttributes;
|
|
||||||
import java.time.LocalDateTime;
|
|
||||||
import java.time.format.DateTimeFormatter;
|
|
||||||
import java.util.*;
|
|
||||||
|
|
||||||
import static java.nio.file.FileVisitResult.CONTINUE;
|
|
||||||
|
|
||||||
@Service
|
|
||||||
@ConfigurationProperties(prefix = "local-service")
|
|
||||||
@Data
|
|
||||||
public class LocalServiceImpl{
|
|
||||||
String fires;
|
|
||||||
String DouNaiClash;
|
|
||||||
String DouNaiV2ray;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
CustomConfigurationMapper configurationMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
ShareFileMapper shareFIleMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
GalleryMapper galleryMapper;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 每周周一四点重置额度
|
|
||||||
*/
|
|
||||||
@Scheduled(cron = "0 0 4 * * MON")
|
|
||||||
public void reset() {
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.WEEK_USED_AMOUNT, "0");
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.LAST_RESET_AMOUNT_TIME, CustomUtil.now());
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 定时更新订阅
|
|
||||||
* @throws IOException 下载以及保存异常
|
|
||||||
*/
|
|
||||||
@Scheduled(fixedRate = 86400000)
|
|
||||||
public void updateSubScheduler() throws IOException {
|
|
||||||
updateSub(false);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 更新订阅链接的实际方法
|
|
||||||
*/
|
|
||||||
public boolean updateSub(boolean isManual) throws IOException {
|
|
||||||
DateTimeFormatter dateTimeFormatter = CustomUtil.dateTimeFormatter();
|
|
||||||
CustomConfiguration customConfiguration = configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME);
|
|
||||||
|
|
||||||
//如果不是手动,则判断更新间隔是否满足,不满足则取消更新
|
|
||||||
if (!isManual) {
|
|
||||||
LocalDateTime lastUpdate = LocalDateTime.parse(customConfiguration.getValue(), CustomUtil.dateTimeFormatter());
|
|
||||||
LocalDateTime now = LocalDateTime.now();
|
|
||||||
now = now.plusHours(-3);
|
|
||||||
if(now.isBefore(lastUpdate))
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
File mixin = new File("sub/sub.txt");
|
|
||||||
File firesFile = new File("sub/fires.txt");
|
|
||||||
File DouNaiClashFile = new File("sub/DouNaiClash.txt");
|
|
||||||
File DouNaiV2rayFile = new File("sub/DouNaiV2ray.txt");
|
|
||||||
File directory = new File("sub");
|
|
||||||
|
|
||||||
if(!directory.isDirectory())
|
|
||||||
Files.createDirectory(Paths.get("sub"));
|
|
||||||
|
|
||||||
List<String> fires_profile = null;
|
|
||||||
List<String> DouNaiClash_profile = null;
|
|
||||||
OutputStream outputStream;
|
|
||||||
|
|
||||||
//下载薯条订阅
|
|
||||||
try(FileWriter writer = new FileWriter(firesFile)) {
|
|
||||||
fires_profile = Get(fires);
|
|
||||||
for (String i : fires_profile)
|
|
||||||
writer.write(i + "\n");
|
|
||||||
|
|
||||||
System.out.println("load fires complete");
|
|
||||||
}catch (IOException e) {
|
|
||||||
e.printStackTrace();
|
|
||||||
System.out.println("load fires failure");
|
|
||||||
}
|
|
||||||
|
|
||||||
//下载豆奶v2ray订阅
|
|
||||||
try(FileWriter writer = new FileWriter(DouNaiV2rayFile)) {
|
|
||||||
String DouNaiV2rayRaw = Get(DouNaiV2ray).get(0);
|
|
||||||
String[] v2rayPlain = new String(Base64.getDecoder().decode(DouNaiV2rayRaw)).split("\n");
|
|
||||||
StringBuilder stringBuilder = new StringBuilder();
|
|
||||||
//过滤高倍率节点
|
|
||||||
for(String node: v2rayPlain){
|
|
||||||
String name = URLDecoder.decode(node.split("#")[1], StandardCharsets.UTF_8);
|
|
||||||
if(name.startsWith("①") && name.contains("流量")){
|
|
||||||
float ratio = Float.parseFloat(name.substring(name.indexOf("(") + 1, name.indexOf(")")).replace("倍流量", ""));
|
|
||||||
if(ratio <= 1)
|
|
||||||
stringBuilder.append(node).append("\n");
|
|
||||||
}
|
|
||||||
else{
|
|
||||||
stringBuilder.append(node).append("\n");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
writer.write(new String(Base64.getEncoder().encode(stringBuilder.toString().getBytes(StandardCharsets.UTF_8))));
|
|
||||||
|
|
||||||
System.out.println("load DouNai v2ray complete");
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
System.out.println("load DouNai v2ray failure");
|
|
||||||
}
|
|
||||||
|
|
||||||
//下载豆奶clash订阅
|
|
||||||
try(FileWriter writer = new FileWriter(DouNaiClashFile)) {
|
|
||||||
DouNaiClash_profile = Get(DouNaiClash);
|
|
||||||
//过滤高倍率节点
|
|
||||||
ArrayList<String> clashProcessed = new ArrayList<>();
|
|
||||||
boolean isProxies = false;
|
|
||||||
boolean skip = false;
|
|
||||||
for(String line: DouNaiClash_profile){
|
|
||||||
if(line.equals("proxies:"))
|
|
||||||
isProxies = true;
|
|
||||||
else if(line.equals("proxy-groups:") && isProxies)
|
|
||||||
isProxies = false;
|
|
||||||
|
|
||||||
if(isProxies) {
|
|
||||||
if (line.contains("name"))
|
|
||||||
skip = line.contains("流量");
|
|
||||||
if (!skip)
|
|
||||||
clashProcessed.add(line);
|
|
||||||
}
|
|
||||||
else
|
|
||||||
if (!line.contains("流量"))
|
|
||||||
clashProcessed.add(line);
|
|
||||||
}
|
|
||||||
|
|
||||||
for(String line: clashProcessed)
|
|
||||||
writer.write(line + "\n");
|
|
||||||
System.out.println("load DouNai clash complete");
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
System.out.println("load DouNai clash failure");
|
|
||||||
}
|
|
||||||
|
|
||||||
//处理合并
|
|
||||||
assert DouNaiClash_profile != null;
|
|
||||||
int side_start = DouNaiClash_profile.indexOf("proxies:");
|
|
||||||
int side_end = DouNaiClash_profile.indexOf("proxy-groups:");
|
|
||||||
|
|
||||||
List<String> side_proxy = new ArrayList<>();
|
|
||||||
List<String> side_name = new ArrayList<>();
|
|
||||||
for (int i = side_start + 1; i < side_end; i++) {
|
|
||||||
side_proxy.add(DouNaiClash_profile.get(i));
|
|
||||||
if (DouNaiClash_profile.get(i).contains("name"))
|
|
||||||
side_name.add(DouNaiClash_profile.get(i).replace("name:", ""));
|
|
||||||
}
|
|
||||||
|
|
||||||
List<String> final_profile = new ArrayList<>();
|
|
||||||
assert fires_profile != null;
|
|
||||||
for (String i : fires_profile) {
|
|
||||||
if (i.equals("proxy-groups:"))
|
|
||||||
final_profile.addAll(side_proxy);
|
|
||||||
|
|
||||||
final_profile.add(i);
|
|
||||||
if (i.equals("proxy-groups:")) {
|
|
||||||
final_profile.add(" -");
|
|
||||||
final_profile.add(" name: e站流量");
|
|
||||||
final_profile.add(" type: select");
|
|
||||||
final_profile.add(" proxies:");
|
|
||||||
for (String x : side_name)
|
|
||||||
final_profile.add(" " + x);
|
|
||||||
|
|
||||||
}
|
|
||||||
if (i.equals("rules:"))
|
|
||||||
final_profile.add(" - DOMAIN-KEYWORD,hentai,e站流量");
|
|
||||||
}
|
|
||||||
|
|
||||||
outputStream = Files.newOutputStream(mixin.toPath());
|
|
||||||
for(String i: final_profile)
|
|
||||||
outputStream.write((i + "\n").getBytes(StandardCharsets.UTF_8));
|
|
||||||
outputStream.close();
|
|
||||||
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME, dateTimeFormatter.format(LocalDateTime.now()));
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 封装好的get方法
|
|
||||||
* @param url url
|
|
||||||
* @return 请求结果
|
|
||||||
* @throws IOException 网络异常
|
|
||||||
*/
|
|
||||||
public static ArrayList<String> Get(String url) throws IOException {
|
|
||||||
CloseableHttpClient httpClient = HttpClients.createDefault();
|
|
||||||
CloseableHttpResponse httpResponse;
|
|
||||||
HttpGet httpGet = new HttpGet(url);
|
|
||||||
|
|
||||||
httpResponse = httpClient.execute(httpGet);
|
|
||||||
|
|
||||||
HttpEntity responseEntity = httpResponse.getEntity();
|
|
||||||
int statusCode = httpResponse.getStatusLine().getStatusCode();
|
|
||||||
ArrayList<String> temp = new ArrayList<>();
|
|
||||||
|
|
||||||
if (statusCode == 200) {
|
|
||||||
BufferedReader reader = new BufferedReader(new InputStreamReader(responseEntity.getContent()));
|
|
||||||
String str;
|
|
||||||
while ((str = reader.readLine()) != null)
|
|
||||||
temp.add(str);
|
|
||||||
}
|
|
||||||
|
|
||||||
httpClient.close();
|
|
||||||
httpResponse.close();
|
|
||||||
return temp;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 每天四点,清理过期分享码
|
|
||||||
*/
|
|
||||||
@Scheduled(cron = "0 0 4 * * *")
|
|
||||||
public void checkShareCode(){
|
|
||||||
ShareFile[] shareFiles = shareFIleMapper.selectAllShareFile();
|
|
||||||
Calendar now;
|
|
||||||
Calendar expireTime;
|
|
||||||
for(ShareFile shareFile: shareFiles){
|
|
||||||
now = Calendar.getInstance();
|
|
||||||
expireTime = Calendar.getInstance();
|
|
||||||
expireTime.setTime(shareFile.getExpireTime());
|
|
||||||
if(now.after(expireTime))
|
|
||||||
shareFIleMapper.deleteShareFile(shareFile.getShareCode());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 每周,查看缩略图数量,按照访问时间排序,删除超出1w的部分
|
|
||||||
*/
|
|
||||||
@Scheduled(cron = "0 0 4 1 * *")
|
|
||||||
public void clearThumbnailCache(){
|
|
||||||
String cachePath = "/storage/hentaiCache/";
|
|
||||||
File directory = new File(cachePath);
|
|
||||||
List<Path> files = new ArrayList<>();
|
|
||||||
try {
|
|
||||||
Files.walkFileTree(directory.toPath(), new SimpleFileVisitor<>() {
|
|
||||||
@Override
|
|
||||||
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
|
|
||||||
if (attrs.isRegularFile()) {
|
|
||||||
files.add(file);
|
|
||||||
}
|
|
||||||
return CONTINUE;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} catch (IOException e) {
|
|
||||||
throw new RuntimeException(e);
|
|
||||||
}
|
|
||||||
|
|
||||||
files.sort((f1, f2) -> {
|
|
||||||
try {
|
|
||||||
return Files.readAttributes(f1, BasicFileAttributes.class).lastAccessTime()
|
|
||||||
.compareTo(Files.readAttributes(f2, BasicFileAttributes.class).lastAccessTime());
|
|
||||||
} catch (IOException e) {
|
|
||||||
throw new RuntimeException(e);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
if (files.size() > 10000) {
|
|
||||||
List<Path> toDelete = files.subList(0, files.size() - 10000);
|
|
||||||
for (Path file : toDelete) {
|
|
||||||
try {
|
|
||||||
Files.delete(file);
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
System.out.println("Deleted " + toDelete.size() + " files");
|
|
||||||
} else {
|
|
||||||
System.out.println("No files to delete");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
|
||||||
|
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
||||||
|
import com.lion.lionwebsite.Util.Response;
|
||||||
|
|
||||||
|
import lombok.Data;
|
||||||
|
import lombok.extern.slf4j.Slf4j;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.util.HashMap;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
import static com.lion.lionwebsite.Util.CustomUtil.objectMapper;
|
||||||
|
|
||||||
|
|
||||||
|
@Service
|
||||||
|
@Data
|
||||||
|
@Slf4j
|
||||||
|
public class PersonalService{
|
||||||
|
final CustomConfigurationMapper configurationMapper;
|
||||||
|
|
||||||
|
final PushService pushService;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 获取订阅文件上次更新时间
|
||||||
|
* @return 订阅文件上次更新时间
|
||||||
|
*/
|
||||||
|
public String lastUpdate() {
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
CustomConfiguration configuration = configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME);
|
||||||
|
response.success(configuration.getValue());
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 获取家里的ip
|
||||||
|
* @return 家里的ip
|
||||||
|
*/
|
||||||
|
public String getIp(){
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
Map<String, String> jsonObject = new HashMap<>();
|
||||||
|
String ip = configurationMapper.selectConfiguration(CustomConfiguration.CURRENT_IP_ADDRESS).getValue();
|
||||||
|
String updateTime = configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME).getValue();
|
||||||
|
|
||||||
|
jsonObject.put("ip", ip);
|
||||||
|
jsonObject.put("lastUpdateTime", updateTime);
|
||||||
|
response.success(objectMapper.valueToTree(jsonObject).toString().replace("\"", " ").replace("\\", " "));
|
||||||
|
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
|
public String message2me(String message){
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
pushService.sendToMe(message);
|
||||||
|
return response.success().toJSONString();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,365 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Service;
|
|
||||||
|
|
||||||
|
|
||||||
import cn.hutool.core.io.FileUtil;
|
|
||||||
import cn.hutool.core.util.RandomUtil;
|
|
||||||
import com.lion.lionwebsite.Dao.CustomConfigurationMapper;
|
|
||||||
import com.lion.lionwebsite.Dao.ShareFileMapper;
|
|
||||||
import com.lion.lionwebsite.Dao.UserMapper;
|
|
||||||
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
|
||||||
import com.lion.lionwebsite.Domain.ShareFile;
|
|
||||||
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
|
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
|
||||||
import com.lion.lionwebsite.Util.FileDownload;
|
|
||||||
import com.lion.lionwebsite.Util.Response;
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
|
||||||
import lombok.Data;
|
|
||||||
import lombok.extern.slf4j.Slf4j;
|
|
||||||
import org.apache.commons.compress.archivers.ArchiveEntry;
|
|
||||||
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
|
|
||||||
import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream;
|
|
||||||
import org.apache.tomcat.util.http.fileupload.IOUtils;
|
|
||||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
import org.springframework.web.multipart.MultipartFile;
|
|
||||||
|
|
||||||
import java.io.BufferedOutputStream;
|
|
||||||
import java.io.File;
|
|
||||||
import java.io.IOException;
|
|
||||||
import java.io.OutputStream;
|
|
||||||
import java.net.URLDecoder;
|
|
||||||
import java.nio.charset.StandardCharsets;
|
|
||||||
import java.nio.file.*;
|
|
||||||
import java.nio.file.attribute.BasicFileAttributes;
|
|
||||||
import java.time.ZoneId;
|
|
||||||
import java.time.format.DateTimeFormatter;
|
|
||||||
import java.util.*;
|
|
||||||
import java.util.concurrent.ExecutorService;
|
|
||||||
import java.util.concurrent.Executors;
|
|
||||||
|
|
||||||
|
|
||||||
@Service
|
|
||||||
@Data
|
|
||||||
@ConfigurationProperties(prefix = "personal-service")
|
|
||||||
@Slf4j
|
|
||||||
public class PersonalServiceImpl{
|
|
||||||
@Resource
|
|
||||||
CustomConfigurationMapper configurationMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
UserMapper userMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
ShareFileMapper shareFileMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
TaskHandlerInterceptor taskHandlerInterceptor;
|
|
||||||
|
|
||||||
String StoragePath;
|
|
||||||
|
|
||||||
DateTimeFormatter dateTimeFormatter = CustomUtil.dateTimeFormatter();
|
|
||||||
|
|
||||||
ExecutorService compressThreadPool;
|
|
||||||
|
|
||||||
PersonalServiceImpl(){
|
|
||||||
compressThreadPool = Executors.newFixedThreadPool(1);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 获取文件列表,同时带上分享码以及过期时间
|
|
||||||
* @param path 路径
|
|
||||||
* @return 文件列表
|
|
||||||
*/
|
|
||||||
public String getFiles(String path) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
File root_file = new File(StoragePath + path);
|
|
||||||
Calendar now = Calendar.getInstance();
|
|
||||||
Calendar expireTime = Calendar.getInstance();
|
|
||||||
|
|
||||||
//如果目标路径是文件夹
|
|
||||||
if(root_file.isDirectory()) {
|
|
||||||
ArrayList<Map<String, String>> fileMaps = new ArrayList<>();
|
|
||||||
File[] originalFiles = root_file.listFiles();
|
|
||||||
|
|
||||||
//如果文件夹里的文件不为空
|
|
||||||
if (originalFiles != null) {
|
|
||||||
originalFiles = Arrays.stream(originalFiles).sorted(Comparator.comparing(File::getName)).toArray(File[]::new);
|
|
||||||
ArrayList<ShareFile> shareFiles = shareFileMapper.selectShareFilesByFilePath(root_file.getAbsolutePath());
|
|
||||||
|
|
||||||
//遍历文件,放入文件信息以及查询对应的分享码
|
|
||||||
for (File file : originalFiles) {
|
|
||||||
Map<String, String> fileMap = new LinkedHashMap<>();
|
|
||||||
fileMap.put("name", file.getName());
|
|
||||||
fileMap.put("path", file.getAbsolutePath());
|
|
||||||
|
|
||||||
if (file.isDirectory()) {
|
|
||||||
fileMap.put("type", "FOLDER");
|
|
||||||
} else if (file.isFile()) {
|
|
||||||
fileMap.put("type", "FILE"); //处理文件大小单位
|
|
||||||
String fileSize = CustomUtil.fileSizeToString(file.length());
|
|
||||||
fileMap.put("size", fileSize);
|
|
||||||
|
|
||||||
Iterator<ShareFile> iterator = shareFiles.iterator();
|
|
||||||
while(iterator.hasNext()){
|
|
||||||
ShareFile shareFile = iterator.next();
|
|
||||||
if(shareFile.getFilePath().equals(file.getAbsolutePath())){
|
|
||||||
expireTime.setTime(shareFile.getExpireTime());
|
|
||||||
if(now.after(expireTime)){
|
|
||||||
shareFileMapper.deleteShareFile(shareFile.getShareCode());
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
fileMap.put("shareCode", shareFile.getShareCode());
|
|
||||||
fileMap.put("expireTime", dateTimeFormatter.format(shareFile.getExpireTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
|
|
||||||
}
|
|
||||||
iterator.remove();
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
fileMaps.add(fileMap);
|
|
||||||
}
|
|
||||||
response.success(new ObjectMapper().valueToTree(fileMaps).toString());
|
|
||||||
}
|
|
||||||
else
|
|
||||||
response.failure("文件夹为空");
|
|
||||||
}
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 下载文件
|
|
||||||
* @param request 请求对象
|
|
||||||
* @param response 响应对象
|
|
||||||
* @param path 目标路径
|
|
||||||
*/
|
|
||||||
public void download(HttpServletRequest request, HttpServletResponse response, String path){
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
File file = new File(path);
|
|
||||||
if(file.exists())
|
|
||||||
FileDownload.export(request, response, path);
|
|
||||||
|
|
||||||
else
|
|
||||||
try{
|
|
||||||
response.getWriter().print("404 NOT FOUND");
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 上传文件
|
|
||||||
* @param path 目标路径
|
|
||||||
* @param fileName 文件名称
|
|
||||||
* @param file 文件对象
|
|
||||||
* @return 上传结果
|
|
||||||
*/
|
|
||||||
public String uploadFile(String path, String fileName, MultipartFile file) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
log.info("上传文件:{}, 目标路径:{}", fileName, path);
|
|
||||||
|
|
||||||
if(path == null || fileName == null || file == null){
|
|
||||||
response.failure("参数不完整");
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
File directory = new File(StoragePath + path);
|
|
||||||
if(directory.isDirectory()){
|
|
||||||
File targetFile = new File(StoragePath + path, fileName);
|
|
||||||
|
|
||||||
if(targetFile.exists())
|
|
||||||
response.failure("目标文件已存在");
|
|
||||||
else
|
|
||||||
try {
|
|
||||||
file.transferTo(Path.of(StoragePath + path, fileName));
|
|
||||||
response.success("上传成功");
|
|
||||||
} catch (IOException e) {
|
|
||||||
response.failure("上传失败");
|
|
||||||
e.printStackTrace();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
else
|
|
||||||
response.failure("该路径不存在或者不是文件夹");
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 创建分享码
|
|
||||||
* @param path 目标路径
|
|
||||||
* @param expireHour 过期时间
|
|
||||||
* @return 如果成功则是分享码以及过期时间,失败则是失败原因
|
|
||||||
*/
|
|
||||||
public String shareFile(String path, Integer expireHour) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
Map<String, String> jsonObject = new HashMap<>();
|
|
||||||
File file = new File(path);
|
|
||||||
if(file.isFile()){ //如果是文件,则生成分享码,调用此接口时不需要考虑分享已分享文件以及分享已过期文件
|
|
||||||
String ShareCode;
|
|
||||||
ShareCode = RandomUtil.randomString(8);
|
|
||||||
Calendar expireTime = Calendar.getInstance();
|
|
||||||
|
|
||||||
expireTime.add(Calendar.HOUR, expireHour);
|
|
||||||
shareFileMapper.insertShareFile(ShareCode, path, expireTime.getTime());
|
|
||||||
jsonObject.put("shareCode", ShareCode);
|
|
||||||
jsonObject.put("expireTime", dateTimeFormatter.format(expireTime.getTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
|
|
||||||
response.success(new ObjectMapper().valueToTree(jsonObject).toString());
|
|
||||||
}
|
|
||||||
else
|
|
||||||
response.failure("此路径为文件夹或不存在");
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 延长分享时间
|
|
||||||
* @param path 目标文件路径
|
|
||||||
* @param extendHour 延长小时数
|
|
||||||
* @return 延长结果
|
|
||||||
*/
|
|
||||||
public String extendShareTime(String path, Integer extendHour) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
Map<String, String> data = new LinkedHashMap<>();
|
|
||||||
ShareFile shareFile = shareFileMapper.selectShareFileByFilePath(path);
|
|
||||||
|
|
||||||
if(shareFile != null){
|
|
||||||
Calendar calendar = Calendar.getInstance();
|
|
||||||
calendar.setTime(shareFile.getExpireTime());
|
|
||||||
|
|
||||||
calendar.add(Calendar.HOUR, extendHour);
|
|
||||||
shareFile.setExpireTime(calendar.getTime());
|
|
||||||
shareFileMapper.updateShareFile(shareFile);
|
|
||||||
data.put("expireTime", dateTimeFormatter.format(shareFile.getExpireTime().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime()));
|
|
||||||
data.put("path", shareFile.getFilePath());
|
|
||||||
|
|
||||||
response.success(new ObjectMapper().valueToTree(data).toString());
|
|
||||||
}
|
|
||||||
else{
|
|
||||||
response.failure("该文件未被分享");
|
|
||||||
}
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 取消分享
|
|
||||||
* @param path 目标路径
|
|
||||||
* @return 取消结果
|
|
||||||
*/
|
|
||||||
public String cancelShare(String path) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
ShareFile shareFile = shareFileMapper.selectShareFileByFilePath(path);
|
|
||||||
|
|
||||||
if(shareFile != null){
|
|
||||||
shareFileMapper.deleteShareFile(shareFile.getShareCode());
|
|
||||||
response.success("取消分享成功");
|
|
||||||
}
|
|
||||||
else{
|
|
||||||
response.failure("该文件未被分享");
|
|
||||||
}
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 获取订阅文件上次更新时间
|
|
||||||
* @return 订阅文件上次更新时间
|
|
||||||
*/
|
|
||||||
public String lastUpdate() {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
CustomConfiguration configuration = configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_SUB_TIME);
|
|
||||||
response.success(configuration.getValue());
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 获取家里的ip
|
|
||||||
* @return 家里的ip
|
|
||||||
*/
|
|
||||||
public String getIp(){
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
Map<String, String> jsonObject = new HashMap<>();
|
|
||||||
String ip = configurationMapper.selectConfiguration(CustomConfiguration.CURRENT_IP_ADDRESS).getValue();
|
|
||||||
String updateTime = configurationMapper.selectConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME).getValue();
|
|
||||||
|
|
||||||
jsonObject.put("ip", ip);
|
|
||||||
jsonObject.put("lastUpdateTime", updateTime);
|
|
||||||
response.success(new ObjectMapper().valueToTree(jsonObject).toString().replace("\"", " ").replace("\\", " "));
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 打包文件
|
|
||||||
* @param path 目标路径
|
|
||||||
* @return 响应提交成功,因为该方法为异步执行,未完成时后辍为undone
|
|
||||||
*/
|
|
||||||
public String compress(String path) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
File file = new File(path);
|
|
||||||
String finalPath = path;
|
|
||||||
|
|
||||||
if(!file.isDirectory() || !file.exists()){
|
|
||||||
response.failure("选中的路径不是文件夹");
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
compressThreadPool.submit(() -> {
|
|
||||||
try(OutputStream bos = new BufferedOutputStream(Files.newOutputStream(Paths.get(finalPath + ".tar***undone")));
|
|
||||||
TarArchiveOutputStream aos = new TarArchiveOutputStream(bos)) {
|
|
||||||
aos.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU); //解除文件名长度限制
|
|
||||||
Path dirPath = Paths.get(finalPath);
|
|
||||||
Files.walkFileTree(dirPath, new SimpleFileVisitor<>() {
|
|
||||||
@Override
|
|
||||||
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
|
|
||||||
ArchiveEntry entry = new TarArchiveEntry(dir.toFile(), dirPath.relativize(dir).toString());
|
|
||||||
aos.putArchiveEntry(entry);
|
|
||||||
aos.closeArchiveEntry();
|
|
||||||
return super.preVisitDirectory(dir, attrs);
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
|
|
||||||
ArchiveEntry entry = new TarArchiveEntry(file.toFile(), dirPath.relativize(file).toString());
|
|
||||||
aos.putArchiveEntry(entry);
|
|
||||||
IOUtils.copy(Files.newInputStream(file.toFile().toPath()), aos);
|
|
||||||
aos.closeArchiveEntry();
|
|
||||||
return super.visitFile(file, attrs);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
File targetFile = new File(finalPath + ".tar***undone");
|
|
||||||
log.info("打包成功,重命名:" + targetFile.renameTo(new File(finalPath + ".tar")));
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
log.info("打包失败,删除文件结果:" + new File(finalPath + ".tar***undone").delete());
|
|
||||||
}
|
|
||||||
});
|
|
||||||
response.success("加入队列成功");
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 删除文件
|
|
||||||
* @param path 目标路径
|
|
||||||
* @return 删除结果
|
|
||||||
*/
|
|
||||||
public String deleteFile(String path) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
path = URLDecoder.decode(path, StandardCharsets.UTF_8);
|
|
||||||
File file = new File(path);
|
|
||||||
|
|
||||||
if(FileUtil.del(file))
|
|
||||||
response.success("删除成功");
|
|
||||||
else
|
|
||||||
response.failure("删除失败");
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
package com.lion.lionwebsite.Service;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.normal.CustomConfigurationMapper;
|
||||||
|
|
||||||
|
import com.lion.lionwebsite.Dao.normal.UserMapper;
|
||||||
|
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
||||||
|
import com.lion.lionwebsite.Domain.User;
|
||||||
|
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
|
||||||
|
import com.lion.lionwebsite.Util.CustomUtil;
|
||||||
|
import com.lion.lionwebsite.Util.Response;
|
||||||
|
|
||||||
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
|
import lombok.RequiredArgsConstructor;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
@Service
|
||||||
|
@RequiredArgsConstructor
|
||||||
|
public class PublicService {
|
||||||
|
|
||||||
|
final CustomConfigurationMapper configurationMapper;
|
||||||
|
|
||||||
|
final UserMapper userMapper;
|
||||||
|
|
||||||
|
final TaskHandlerInterceptor taskHandlerInterceptor;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 记录家里ip地址
|
||||||
|
* @param ip ip地址
|
||||||
|
*/
|
||||||
|
public void logIpAddress(String ip) {
|
||||||
|
configurationMapper.updateConfiguration(CustomConfiguration.CURRENT_IP_ADDRESS, ip);
|
||||||
|
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME, CustomUtil.now());
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 修改授权码,如果能够执行此方法,则授权码一定存在
|
||||||
|
* @param AuthCode 原来的授权码
|
||||||
|
* @param newAuthCode 新的授权码
|
||||||
|
* @return 修改结果
|
||||||
|
*/
|
||||||
|
public String alterAuthCode(String AuthCode, String newAuthCode) {
|
||||||
|
Response response = Response.generateResponse();
|
||||||
|
|
||||||
|
userMapper.updateAuthCode(AuthCode, newAuthCode);
|
||||||
|
taskHandlerInterceptor.updateAuthCodes();
|
||||||
|
response.success("修改成功");
|
||||||
|
|
||||||
|
return response.toJSONString();
|
||||||
|
}
|
||||||
|
|
||||||
|
public User getUserId(String AuthCode){
|
||||||
|
return userMapper.selectUserByAuthCode(AuthCode);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,161 +0,0 @@
|
|||||||
package com.lion.lionwebsite.Service;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Dao.CustomConfigurationMapper;
|
|
||||||
|
|
||||||
import com.lion.lionwebsite.Dao.ShareFileMapper;
|
|
||||||
import com.lion.lionwebsite.Dao.UserMapper;
|
|
||||||
import com.lion.lionwebsite.Domain.CustomConfiguration;
|
|
||||||
import com.lion.lionwebsite.Domain.ShareFile;
|
|
||||||
import com.lion.lionwebsite.Domain.User;
|
|
||||||
import com.lion.lionwebsite.Interceptor.TaskHandlerInterceptor;
|
|
||||||
import com.lion.lionwebsite.Util.CustomUtil;
|
|
||||||
import com.lion.lionwebsite.Util.FileDownload;
|
|
||||||
import com.lion.lionwebsite.Util.Response;
|
|
||||||
|
|
||||||
import jakarta.annotation.Resource;
|
|
||||||
import jakarta.servlet.ServletOutputStream;
|
|
||||||
import jakarta.servlet.http.HttpServletRequest;
|
|
||||||
import jakarta.servlet.http.HttpServletResponse;
|
|
||||||
import org.springframework.stereotype.Service;
|
|
||||||
|
|
||||||
|
|
||||||
import java.io.ByteArrayOutputStream;
|
|
||||||
import java.io.File;
|
|
||||||
import java.io.IOException;
|
|
||||||
import java.io.InputStream;
|
|
||||||
import java.net.URL;
|
|
||||||
import java.nio.charset.StandardCharsets;
|
|
||||||
import java.util.Calendar;
|
|
||||||
|
|
||||||
import static com.lion.lionwebsite.Util.CustomUtil.fourZeroFour;
|
|
||||||
|
|
||||||
|
|
||||||
@Service
|
|
||||||
public class PublicServiceImpl {
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
CustomConfigurationMapper configurationMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
ShareFileMapper shareFIleMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
UserMapper userMapper;
|
|
||||||
|
|
||||||
@Resource
|
|
||||||
TaskHandlerInterceptor taskHandlerInterceptor;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 记录家里ip地址
|
|
||||||
* @param ip ip地址
|
|
||||||
*/
|
|
||||||
public void logIpAddress(String ip) {
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.CURRENT_IP_ADDRESS, ip);
|
|
||||||
configurationMapper.updateConfiguration(CustomConfiguration.LAST_UPDATE_IP_ADDRESS_TIME, CustomUtil.now());
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 通过分享码获取文件
|
|
||||||
* @param httpRequest 请求对象
|
|
||||||
* @param httpResponse 响应对象
|
|
||||||
* @param ShareCode 分享码
|
|
||||||
* @throws IOException 响应时的异常
|
|
||||||
*/
|
|
||||||
public boolean GetFile(HttpServletRequest httpRequest, HttpServletResponse httpResponse, String ShareCode) throws IOException {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
|
|
||||||
//参数为空的情况
|
|
||||||
if(ShareCode == null) {
|
|
||||||
response.failure("ShareCode invalid");
|
|
||||||
} else {
|
|
||||||
ShareFile shareFile = shareFIleMapper.selectShareFileByShareCode(ShareCode);
|
|
||||||
Calendar ExpireTime = Calendar.getInstance();
|
|
||||||
Calendar now = Calendar.getInstance();
|
|
||||||
|
|
||||||
if (shareFile != null) {
|
|
||||||
ExpireTime.setTime(shareFile.getExpireTime());
|
|
||||||
if (ExpireTime.after(now) && new File(shareFile.getFilePath()).isFile()) {
|
|
||||||
FileDownload.export(httpRequest, httpResponse, shareFile.getFilePath());
|
|
||||||
return true;
|
|
||||||
} else {
|
|
||||||
shareFIleMapper.deleteShareFile(shareFile.getShareCode());
|
|
||||||
response.failure("ShareCode is expired or File is not exist");
|
|
||||||
}
|
|
||||||
} else
|
|
||||||
response.failure("ShareCode is not exist or expired");
|
|
||||||
}
|
|
||||||
|
|
||||||
httpResponse.getOutputStream().write(response.toJSONString().getBytes(StandardCharsets.UTF_8));
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 修改授权码,如果能够执行此方法,则授权码一定存在
|
|
||||||
* @param AuthCode 原来的授权码
|
|
||||||
* @param newAuthCode 新的授权码
|
|
||||||
* @return 修改结果
|
|
||||||
*/
|
|
||||||
public String alterAuthCode(String AuthCode, String newAuthCode) {
|
|
||||||
Response response = Response.generateResponse();
|
|
||||||
|
|
||||||
userMapper.updateAuthCode(AuthCode, newAuthCode);
|
|
||||||
taskHandlerInterceptor.updateAuthCodes();
|
|
||||||
response.success("修改成功");
|
|
||||||
|
|
||||||
return response.toJSONString();
|
|
||||||
}
|
|
||||||
|
|
||||||
public User getUserId(String AuthCode){
|
|
||||||
return userMapper.selectUserByAuthCode(AuthCode);
|
|
||||||
}
|
|
||||||
|
|
||||||
public void getEhThumbnail(String path, HttpServletResponse response){
|
|
||||||
String url;
|
|
||||||
if(!path.contains("/")){
|
|
||||||
fourZeroFour(response);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
url = "https://ehgt.org/" + path;
|
|
||||||
try{
|
|
||||||
byte[] imageBytes = getImageBytesFromUrl(url);
|
|
||||||
ServletOutputStream outputStream = response.getOutputStream();
|
|
||||||
outputStream.write(imageBytes);
|
|
||||||
outputStream.close();
|
|
||||||
}catch (IOException e){
|
|
||||||
e.printStackTrace();
|
|
||||||
try {
|
|
||||||
response.sendError(503);
|
|
||||||
}catch (IOException ex){
|
|
||||||
ex.printStackTrace();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public static byte[] getImageBytesFromUrl(String imageUrl) throws IOException {
|
|
||||||
URL url = new URL(imageUrl);
|
|
||||||
InputStream inputStream = null;
|
|
||||||
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
|
|
||||||
|
|
||||||
try {
|
|
||||||
// 打开URL连接
|
|
||||||
inputStream = url.openStream();
|
|
||||||
|
|
||||||
byte[] buffer = new byte[1024];
|
|
||||||
int bytesRead;
|
|
||||||
// 从输入流读取数据并写入输出流
|
|
||||||
while ((bytesRead = inputStream.read(buffer)) != -1) {
|
|
||||||
outputStream.write(buffer, 0, bytesRead);
|
|
||||||
}
|
|
||||||
} finally {
|
|
||||||
// 关闭流
|
|
||||||
if (inputStream != null) {
|
|
||||||
inputStream.close();
|
|
||||||
}
|
|
||||||
outputStream.close();
|
|
||||||
}
|
|
||||||
|
|
||||||
// 返回图片的字节数组
|
|
||||||
return outputStream.toByteArray();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user